Gaussian Process Classification: Singly vs. Doubly Stochastic Models, and New Computational Schemes

The aim of this paper is to compare four different methods for binary classification with an underlying Gaussian process with respect to theoretical consistency and practical performance. Two of the inference schemes, namely classical indicator kriging and simplicial indicator kriging, are analytically tractable and fast. However, these methods rely on simplifying assumptions which are inappropriate for categorical class labels. A consistent and previously described model extension involves a doubly stochastic process. There, the unknown posterior class probability f(·) is considered a realization of a spatially correlated Gaussian process that has been squashed to the unit interval, and a label at position x is considered an independent Bernoulli realization with success parameter f(x). Unfortunately, inference for this model is not known to be analytically tractable. In this paper, we propose two new computational schemes for the inference in this doubly stochastic model, namely the “Aitchison Maximum Posterior” and the “Doubly Stochastic Gaussian Quadrature”. Both methods are analytical up to a final step where optimization or integration must be carried out numerically. For the comparison of practical performance, the methods are applied to storm forecasts for the Spanish coast based on wave heights in the Mediterranean Sea. While the error rate of the doubly stochastic models is slightly lower, their computational cost is much higher.