ADMM for Training Sparse Structural SVMs with Augmented ℓ1 Regularizers
نویسندگان
چکیده
The size |Y | of the output space Y is exponential and optimization over the entire space Y is computationally expensive. Hence in the sequential dual optimization method, the optimization of (A.6) is restricted to the set Yi = {y : αiy > 0} maintained for each example. For clarity, we present the sequential dual optimization method to solve (A.2) in Algorithm 3. The algorithm starts with Yi = {yi} ∀i (Step 2 in Algorithm 3). Whenever an example is visited, the set Yi is updated (Steps 8-12 in Algorithm 3) by finding ŷi = arg miny∈Y ∇iyD(α) and by adding ŷi to Yi when the following condition is satisfied:
منابع مشابه
A Block-wise, Asynchronous and Distributed ADMM Algorithm for General Form Consensus Optimization
Many machine learning models, including those with non-smooth regularizers, can be formulated as consensus optimization problems, which can be solved by the alternating direction method of multipliers (ADMM). Many recent efforts have been made to develop asynchronous distributed ADMM to handle large amounts of training data. However, all existing asynchronous distributed ADMM methods are based ...
متن کاملConvergent Iterative CT Reconstruction With Sparsity-Based Regularization
Statistical image reconstruction for X-ray CT can provide improved image quality at reduced patient doses. An important component of statistical reconstruction methods is the regularizer. There has been increased interest in sparsity-based regularization, typically using l1 norms. The non-smooth nature of these regularizers is a challenge for iterative optimization methods and often causes slow...
متن کاملSparse Signal Recovery via Correlated Degradation Model
Sparse signal recovery aims to recover an unknown signal x ∈ R from few non-adaptive, possibly noisy, linear measurements y∈R using a nonlinear sparsity-promoting algorithm, under the assumption that x is sparse or compressible with respect to a known basis or frame [1]. Specifically, y = Ax+ e, where A ∈ Rm×n is the measurement matrix, e ∈ R is the measurement error, and m n. Many of the spars...
متن کاملMulti-Step Stochastic ADMM in High Dimensions: Applications to Sparse Optimization and Matrix Decomposition
In this paper, we consider a multi-step version of the stochastic ADMM method with efficient guarantees for high-dimensional problems. We first analyze the simple setting, where the optimization problem consists of a loss function and a single regularizer (e.g. sparse optimization), and then extend to the multi-block setting with multiple regularizers and multiple variables (e.g. matrix decompo...
متن کاملAn ADMM algorithm for solving ℓ1 regularized MPC
[1] M. Gallieri, J. M. Maciejowski “lasso MPC: Smart Regulation of OverActuated Systems”, to appear in ACC 2012. [2] M. Annergren, A. Hansson, B. Wahlberg “An ADMM Algorithm for Solving l1 Regularized MPC”, submitted. [3] S. Boyd, N. Parikh, E. Chu, B. Peleato, J. Eckstein “Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers”, Foundations and Tr...
متن کامل