Discriminative Metric Learning by Neighborhood Gerrymandering

We formulate the problem of metric learning for k nearest neighbor classificationas a large margin structured prediction problem, with a latent variable representingthe choice of neighbors and the task loss directly corresponding to classificationerror. We describe an efficient algorithm for exact loss augmented inference, anda fast gradient descent algorithm for learning in this model. The objective drivesthe metric to establish neighborhood boundaries that benefit the true class labelsfor the training points. Our approach, reminiscent of gerrymandering (redrawingof political boundaries to provide advantage to certain parties), is more direct inits handling of optimizing classification accuracy than those previously proposed.In experiments on a variety of data sets our method is shown to achieve excellentresults compared to current state of the art in metric learning.