Class Probability Estimation via Differential Geometric Regularization

We use differential geometry techniques to estimate the class probability P (y = l|x) for learning both binary and multiclass plug-in classifiers. We propose a geometric regularization technique to find the optimal submanifold corresponding to the estimator of P (y = l|x). The regularization term measures the volume of this submanifold, based on the intuition that overfitting produces fast oscillations and hence large volume of the estimator. We use gradient flow methods to move from an initial estimator towards a minimizer of a penalty function that penalizes both the deviation of the submanifold from the training data and large volume. We establish Bayes consistency for our algorithm under mild initialization assumptions. In experiments for both binary and multiclass classification, our implementation compares favorably to several widely used classification methods.