Bregman Alternating Direction Method of Multipliers
نویسندگان
چکیده
The mirror descent algorithm (MDA) generalizes gradient descent by using a Bregman divergence to replace squared Euclidean distance. In this paper, we similarly generalize the alternating direction method of multipliers (ADMM) to Bregman ADMM (BADMM), which allows the choice of different Bregman divergences to exploit the structure of problems. BADMM provides a unified framework for ADMM and its variants, including generalized ADMM, inexact ADMM and Bethe ADMM. We establish the global convergence and the O(1/T ) iteration complexity for BADMM. In some cases, BADMM can be faster than ADMM by a factor of O(n/ log(n)). In solving the linear program of mass transportation problem, BADMM leads to massive parallelism and can easily run on GPU. BADMM is several times faster than highly optimized commercial software Gurobi.
منابع مشابه
Applications of Lagrangian-Based Alternating Direction Methods and Connections to Split Bregman
Analogous to the connection between Bregman iteration and the method of multipliers that was pointed out in [59], we show that a similar connection can be made between the split Bregman algorithm [32] and the alternating direction method of multipliers (ADMM) of ([29], [31]). Existing convergence theory for ADMM [23] can therefore be used to justify both the alternating step and inexact minimiz...
متن کاملModified Convex Data Clustering Algorithm Based on Alternating Direction Method of Multipliers
Knowing the fact that the main weakness of the most standard methods including k-means and hierarchical data clustering is their sensitivity to initialization and trapping to local minima, this paper proposes a modification of convex data clustering in which there is no need to be peculiar about how to select initial values. Due to properly converting the task of optimization to an equivalent...
متن کاملAdaptive Stochastic Alternating Direction Method of Multipliers
The Alternating Direction Method of Multipliers (ADMM) has been studied for years. Traditional ADMM algorithms need to compute, at each iteration, an (empirical) expected loss function on all training examples, resulting in a computational complexity proportional to the number of training examples. To reduce the complexity, stochastic ADMM algorithms were proposed to replace the expected loss f...
متن کاملA comparison of the computational performance of Iteratively Reweighted Least Squares and alternating minimization algorithms for ℓ1 inverse problems
Alternating minimization algorithms with a shrinkage step, derived within the Split Bregman (SB) or Alternating Direction Method of Multipliers (ADMM) frameworks, have become very popular for `-regularized problems, including Total Variation and Basis Pursuit Denoising. It appears to be generally assumed that they deliver much better computational performance than older methods such as Iterativ...
متن کاملDistributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers
Many problems of recent interest in statistics and machine learning can be posed in the framework of convex optimization. Due to the explosion in size and complexity of modern datasets, it is increasingly important to be able to solve problems with a very large number of features or training examples. As a result, both the decentralized collection or storage of these datasets as well as accompa...
متن کامل