Sufficiency, classification, and the class-specific feature theorem

A new proof of the class-specific feature theorem is given. The proof makes use of the observed data as opposed to the set of sufficient statistics as in the original formulation. We prove the theorem for the classical case, in which the parameter vector is deterministic and known, as well as for the Bayesian case, in which the parameter vector is modeled as a random vector with known prior probability density function. The essence of the theorem is that with a suitable normalization the probability density function of the sufficient statistic for each probability density function family can be used for optimal classification. One need not have knowledge of the probability density functions of the data under each hypothesis.