In machine learning, one-class classification, also known as unary classification or class-modelling, tries to identify objects of a specific class amongst all objects, by primarily learning from a training set containing only the objects of that class, although there exist variants of one-class classifiers where counter-examples are used to further refine the classification boundary. This is different from and more difficult than the traditional classification problem, which tries to distinguish between two or more classes with the training set containing objects from all the classes. Examples include the monitoring of helicopter gearboxes, motor failure prediction, or the operational status of a nuclear plant as 'normal': In this scenario, there are few, if any, examples of catastrophic system states; only the statistics of normal operation are known. While many of the above approaches focus on the case of removing a small number of outliers or anomalies, one can also learn the other extreme, where the single class covers a small coherent subset of the data, using an information bottleneck approach.
Overview
The term one-class classification was coined by Moya & Hush and many applications can be found in scientific literature, for example outlier detection, anomaly detection, novelty detection. A feature of OCC is that it uses only sample points from the assigned class, so that a representative sampling is not strictly required for non-target classes.
Introduction
SVM based one-class classification relies on identifying the smallest hypersphere consisting of all the data points. This method is called Support Vector Data Description. Formally, the problem can be defined in the following constrained optimization form, However, the above formulation is highly restrictive, and is sensitive to the presence of outliers. Therefore, a flexible formulation, that allow for the presence of outliers is formulated as shown below, From Karush-Kuhn-Tucker optimality conditions, we get where the 's are the solution to the following optimization problem: subject to, The introduction of kernel function provide additional flexibility to the One-class SVM algorithm.
PU learning
A similar problem is PU learning, in which a binary classifier is learned in a semi-supervised way from only positive and unlabeled sample points. In PU learning, two sets of examples are assumed to be available for training: the positive set and a mixed set, which is assumed to contain both positive and negative samples, but without these being labeled as such. This contrasts with other forms of semisupervised learning, where it is assumed that a labeled set containing examples of both classes is available in addition to unlabeled samples. A variety of techniques exist to adapt supervised classifiers to the PU learning setting, including variants of the EM algorithm. PU learning has been successfully applied to text, time series, bioinformatics tasks, and Remote-Sensing Data.
Approaches
Several approaches have been proposed to solve one-class classification. The approaches can be distinguished into three main categories, density estimation, boundary methods, and reconstruction methods.
Density estimation methods rely on estimating the density of the data points, and set the threshold. These methods rely on assuming distributions, such as Gaussian, or a Poisson distribution. Following which discordancy tests can be used to test the new objects. These methods are robust to scale variance. Gaussian model is one the simplest method to create one-class classifiers. Due to Central Limit Theorem, these methods work best when large number of samples are present, and they are perturbed by small independent error values. The probability distribution for a d-dimensional object is given by: Where, ' is the mean and ' is the covariance matrix. Computing the inverse of covariance matrix is the costliest operation, and in the cases where the data is not scaled properly, or data has singular directions pseudo-inverse is used to approximate the inverse, and is calculated as.
Boundary methods
Boundary methods focus on setting boundaries around a few set of points, called target points. These methods attempt to optimize the volume. Boundary methods rely on distances, and hence are not robust to scale variance. K-centers method, NN-d, and SVDD are some of the key examples. K-centers In K-center algorithm, small balls with equal radius are placed to minimize the maximum distance of all minimum distances between training objects and the centers. Formally, the following error is minimized, The algorithm uses forward search method with random initialization, where the radius is determined by the maximum distance of the object, any given ball should capture. After the centers are determined, for any given test object the distance can be calculated as,
Reconstruction methods
Reconstruction methods use prior knowledge and generating process to build a generating model that best fits the data. New objects can be described in terms of a state of the generating model. Some examples of reconstruction methods for OCC are, k-means clustering, learning vector quantization, self-organizing maps, etc.
Applications
Document classification
The basic Support Vector Machine paradigm is trained using both positive and negative examples, however studies have shown there are many valid reasons for using only positive examples. When the SVM algorithm is modified to only use positive examples, the process is considered one-class classification. One situation where this type of classification might prove useful to the SVM paradigm is in trying to identify a web browser’s sites of interest based only off of the user’s browsing history.
Biomedical studies
One-class classification can be particularly useful in biomedical studies where often data from other classes can be difficult or impossible to obtain. In studying biomedical data it can be difficult and/or expensive to obtain the set of labeled data from the second class that would be necessary to perform a two-class classification. A study from The Scientific World Journal found that the typicality approach is the most useful in analysing biomedical data because it can be applied to any type of dataset. The typicality approach is based on the clustering of data by examining data and placing it into new or existing clusters. To apply typicality to one-class classification for biomedical studies, each new observation,, is compared to the target class,, and identified as an outlier or a member of the target class.
