Scaling up Structured Multi-Label Prediction using Discriminative Mean Field Networks

Multi-label classification is an important task in many modern machine learning applications. Accurate methods model the correlations and relationships between labels, either by assuming a low-dimensional embedding of the labels or a graph structure of label dependencies. While such interactions can be achieved using feed-forward predictors, problems with tight coupling between labels are better posed as structured prediction problems. Unfortunately, prior applications of graphical models to multi-label classification scale poorly. In response, we introduce discriminative mean field networks, an iterative structured prediction technique applicable to substantially larger label sets. We employ a deep architecture to define an energy function of candidate labels, and form predictions using backpropagation to iteratively optimize the energy with respect to the labels. This deep architecture captures dependencies between labels that would lead to completely intractable graphical models, and enables a form of structure learning by automatically learning discriminative features of the structured output. The technique is effective on a variety of benchmarks, and generalizes easily to other structured prediction applications.