Structured Sparsity in Structured Prediction
نویسندگان
چکیده
Linear models have enjoyed great success in structured prediction in NLP. While a lot of progress has been made on efficient training with several loss functions, the problem of endowing learners with a mechanism for feature selection is still unsolved. Common approaches employ ad hoc filtering or L1regularization; both ignore the structure of the feature space, preventing practicioners from encoding structural prior knowledge. We fill this gap by adopting regularizers that promote structured sparsity, along with efficient algorithms to handle them. Experiments on three tasks (chunking, entity recognition, and dependency parsing) show gains in performance, compactness, and model interpretability.
منابع مشابه
Structured Prediction with Output Embeddings for Semantic Image Annotation
We address the task of annotating images with semantic tuples. Solving this problem requires an algorithm able to deal with hundreds of classes for each argument of the tuple. In such contexts, data sparsity becomes a key challenge. We propose handling this sparsity by incorporating feature representations of both the inputs (images) and outputs (argument classes) into a factorized log-linear m...
متن کاملThis Text Has the Scent of Starbucks: A Laplacian Structured Sparsity Model for Computational Branding Analytics
We propose a Laplacian structured sparsity model to study computational branding analytics. To do this, we collected customer reviews from Starbucks, Dunkin’ Donuts, and other coffee shops across 38 major cities in the Midwest and Northeastern regions of USA. We study the brand related language use through these reviews, with focuses on the brand satisfaction and gender factors. In particular, ...
متن کاملSolving Structured Sparsity Regularization with Proximal Methods
Proximal methods have recently been shown to provide effective optimization procedures to solve the variational problems defining the !1 regularization algorithms. The goal of the paper is twofold. First we discuss how proximal methods can be applied to solve a large class of machine learning algorithms which can be seen as extensions of !1 regularization, namely structured sparsity regularizat...
متن کاملSimple Training of Dependency Parsers via Structured Boosting
Recently, significant progress has been made on learning structured predictors via coordinated training algorithms such as conditional random fields and maximum margin Markov networks. Unfortunately, these techniques are based on specialized training algorithms, are complex to implement, and expensive to run. We present a much simpler approach to training structured predictors by applying a boo...
متن کاملFast Algorithms for Structured Sparsity
Sparsity has become an important tool in many mathematical sciences such as statistics, machine learning, and signal processing. While sparsity is a good model for data in many applications, data often has additional structure that goes beyond the notion of “standard” sparsity. In many cases, we can represent this additional information in a structured sparsity model. Recent research has shown ...
متن کامل