Maximum a-posteriori (MAP) estimation is an important task in many applications of probabilistic graphical models. Although finding an exact solution is generally intractable, approximations based on linear programming (LP) relaxation often provide good approximate solutions. In this paper we present an algorithm for solving the LP relaxation optimization problem. In order to overcome the lack ...

The alternating direction method of multipliers (ADM or ADMM) breaks a complex optimization problem into much simpler subproblems. The ADM algorithms are typically short and easy to implement yet exhibit (nearly) state-of-the-art performance for large-scale optimization problems. To apply ADM, we first formulate a given problem into the “ADM-ready” form, so the final algorithm depends on the fo...

In this paper, we study the alternating direction method for finding the Dantzig selectors, which are first introduced in [8]. In particular, at each iteration we apply the nonmonotone gradient method proposed in [17] to approximately solve one subproblem of this method. We compare our approach with a first-order method proposed in [3]. The computational results show that our approach usually o...

It is well known that, for a linear program (LP) with constraint matrix A ∈ Rm×n, the Alternating Direction Method of Multiplier converges globally and linearly at a rate O((‖A‖F +mn) log(1/ )). However, such a rate is related to the problem dimension and the algorithm exhibits a slow and fluctuating “tail convergence” in practice. In this paper, we propose a new variable splitting method of LP...

In this paper, we focus on the alternating direction method, which is one of the most effective methods for solving structured variational inequalities(VI). In fact, we propose a proximal parallel alternating direction method which only needs to solve two strongly monotone sub-VI problems at each iteration. Convergence of the new method is proved under mild assumptions. We also present some pre...

The storage and computation requirements of Convolutional Neural Networks (CNNs) can be prohibitive for exploiting these models over low-power or embedded devices. This paper reduces the computational complexity of the CNNs by minimizing an objective function, including the recognition loss that is augmented with a sparsity-promoting penalty term. The sparsity structure of the network is identi...

We propose a new stochastic dual coordinate ascent technique that can be applied to a wide range of regularized learning problems. Our method is based on Alternating Direction Method of Multipliers (ADMM) to deal with complex regularization functions such as structured regularizations. Our method can naturally afford mini-batch update and it gives speed up of convergence. We show that, under mi...

The alternating direction method of multipliers (ADMM) was proposed by Glowinski and Marrocco in 1975; and it has been widely used in a broad spectrum of areas, especially in some sparsitydriven application domains. In 1982, Fortin and Glowinski suggested to enlarge the range of the step size for updating the dual variable in ADMM from 1 to (0, 1+ √ 5 2 ); and this strategy immediately accelera...

The problem of recovering sparse and low-rank components of a given matrix captures a broad spectrum of applications. However, this recovery problem is NP-hard and thus not tractable in general. Recently, it was shown in [3, 6] that this recovery problem can be well approached by solving a convex relaxation problem where the l1-norm and the nuclear norm are used to induce sparse and low-rank st...