Kuhn meets Rosenblatt: Combinatorial Algorithms for Online Structured Prediction
نویسندگان
چکیده
Online algorithms have been successful at a variety of prediction tasks. In structured prediction settings, the model produced by an online learner is fed as input to some combinatorial algorithm for producing structured outputs. This combinatorial algorithm is predominantly considered a black box, which severely limits the control available to the learner. In this paper, we break open this black box. For each example, it aims to change its model minimally subject to a margin-based optimality condition on the output. We define a flexible linear framework that exploits the combinatorial properties of the desired structured output to achieve this in a convex optimization framework. We demonstrate the efficacy of this framework in two applications: dependency parsing via maximum spanning trees and word alignment via bipartite matching. In addition we formulate the solution for shortest path trees without emperical results.
منابع مشابه
Submodular meets Structured: Finding Diverse Subsets in Exponentially-Large Structured Item Sets
To cope with the high level of ambiguity faced in domains such as Computer Vision or Natural Language processing, robust prediction methods often search for a diverse set of high-quality candidate solutions or proposals. In structured prediction problems, this becomes a daunting task, as the solution space (image labelings, sentence parses, etc.) is exponentially large. We study greedy algorith...
متن کاملOn-line Learning Approach to Ensemble Methods for Structured Prediction
We present a series of algorithms with theoretical guarantees for learning accurate ensembles of several structured prediction rules for which no prior knowledge is assumed. This includes a number of randomized and deterministic algorithms devised by converting on-line learning algorithms to batch ones. We also report the results of experiments with these algorithms on various structured predic...
متن کاملLearning Ensembles of Structured Prediction Rules
We present a series of algorithms with theoretical guarantees for learning accurate ensembles of several structured prediction rules for which no prior knowledge is assumed. This includes a number of randomized and deterministic algorithms devised by converting on-line learning algorithms to batch ones, and a boostingstyle algorithm applicable in the context of structured prediction with a larg...
متن کاملEnsemble Methods for Structured Prediction
We present a series of learning algorithms and theoretical guarantees for designing accurate ensembles of structured prediction tasks. This includes several randomized and deterministic algorithms devised by converting on-line learning algorithms to batch ones, and a boosting-style algorithm applicable in the context of structured prediction with a large number of labels. We give a detailed stu...
متن کاملDifferentiable Dynamic Programming for Structured Prediction and Attention
Dynamic programming (DP) solves a variety of structured combinatorial problems by iteratively breaking them down into smaller subproblems. In spite of their versatility, DP algorithms are usually non-differentiable, which hampers their use as a layer in neural networks trained by backpropagation. To address this issue, we propose to smooth the max operator in the dynamic programming recursion, ...
متن کامل