Scalable Large-Margin Structured Learning: Theory and Algorithms
نویسندگان
چکیده
Much of NLP tries to map structured input (sentences) to some form of structured output (tag sequences, parse trees, semantic graphs, or translated/paraphrased/compressed sentences). Thus structured prediction and its learning algorithm are of central importance to us NLP researchers. However, when applying machine learning to structured domains, we often face scalability issues for two reasons:
منابع مشابه
Scalable Large-Margin Online Learning for Structured Classification
We investigate large-margin online learning algorithms for large-scale structured classification tasks, focusing on a structured-output extension of MIRA, the multi-class classification algorithm of Crammer and Singer [5]. The extension approximates the parameter updates in MIRA using k-best structural decoding. We evaluate the algorithm on several sequential classification tasks, showing that ...
متن کامل(Online) Subgradient Methods for Structured Prediction
Promising approaches to structured learning problems have recently been developed in the maximum margin framework. Unfortunately, algorithms that are computationally and memory efficient enough to solve large scale problems have lagged behind. We propose using simple subgradient-based techniques for optimizing a regularized risk formulation of these problems in both online and batch settings, a...
متن کاملA Generic Online Parallel Learning Framework for Large Margin Models
To speed up the training process, many existing systems use parallel technology for online learning algorithms. However, most research mainly focus on stochastic gradient descent (SGD) instead of other algorithms. We propose a generic online parallel learning framework for large margin models, and also analyze our framework on popular large margin algorithms, including MIRA and Structured Perce...
متن کاملSimple Risk Bounds for Position-Sensitive Max-Margin Ranking Algorithms
R bounds for position-sensitive max-margin ranking algorithms can be derived straightforwardly from a structural result for Rademacher averages presented by [1]. We apply this result to pairwise and listwise hinge loss that are position-sensitive by virtue of rescaling the margin by a pairwise or listwise position-sensitive prediction loss. Similar bounds have recently been presented for probab...
متن کاملLarge Margin Classification with the Progressive Hedging Algorithm
Several learning algorithms in classification and structured prediction are formulated as large scale optimization problems. We show that a generic iterative reformulation and resolving strategy based on the progressive hedging algorithm from stochastic programming results in a highly parallel algorithm when applied to the large margin classification problem with nonlinear kernels. We also unde...
متن کامل