Bayes classifier


In statistical classification, the Bayes classifier minimizes the probability of misclassification.

Definition

Suppose a pair takes values in, where is the class label of. This means that the conditional distribution of X, given that the label Y takes the value r is given by
where "" means "is distributed as", and where denotes a probability distribution.
A classifier is a rule that assigns to an observation X=x a guess or estimate of what the unobserved label Y=r actually was. In theoretical terms, a classifier is a measurable function, with the interpretation that C classifies the point x to the class C. The probability of misclassification, or risk, of a classifier C is defined as
The Bayes classifier is
In practice, as in most of statistics, the difficulties and subtleties are associated with modeling the probability distributions effectively—in this case,. The Bayes classifier is a useful benchmark in statistical classification.
The excess risk of a general classifier is defined as
Thus this non-negative quantity is important for assessing the performance of different classification techniques. A classifier is said to be consistent if the excess risk converges to zero as the size of the training data set tends to infinity.