First-Order Methods for Nonsmooth Convex Large-Scale Optimization, II: Utilizing Problem’s Structure
نویسنده
چکیده
We present several state-of-the-art first-order methods for well-structured large-scale nonsmooth convex programs. In contrast to their black-boxoriented prototypes considered in Chapter 5, the methods in question utilize the problem structure in order to convert the original nonsmooth minimization problem into a saddle-point problem with a smooth convex-concave cost function. This reformulation allows us to accelerate the solution process significantly. As in Chapter 5, our emphasis is on methods which, under favorable circumstances, exhibit a (nearly) dimension-independent convergence rate. Along with investigating the general well-structured situation, we outline possibilities to further accelerate first-order methods by randomization.
منابع مشابه
First-Order Methods for Nonsmooth Convex Large-Scale Optimization, I: General Purpose Methods
We discuss several state-of-the-art computationally cheap, as opposed to the polynomial time interior-point algorithms, first-order methods for minimizing convex objectives over simple large-scale feasible sets. Our emphasis is on the general situation of a nonsmooth convex objective represented by deterministic/stochastic first-order oracle and on the methods which, under favorable circumstanc...
متن کاملDecomposition Techniques for Bilinear Saddle Point Problems and Variational Inequalities with Affine Monotone Operators
The majority of First Order methods for large-scale convex-concave saddle point problems and variational inequalities with monotone operators are proximal algorithms which at every iteration need to minimize over problem’s domain X the sum of a linear form and a strongly convex function. To make such an algorithm practical, X should be proximal-friendly – admit a strongly convex function with e...
متن کاملLarge-scale randomized-coordinate descent methods with non-separable linear constraints
We develop randomized block coordinate descent (CD) methods for linearly constrained convex optimization. Unlike other large-scale CD methods, we do not assume the constraints to be separable, but allow them be coupled linearly. To our knowledge, ours is the first CD method that allows linear coupling constraints, without making the global iteration complexity have an exponential dependence on ...
متن کاملLevel methods uniformly optimal for composite and structured nonsmooth convex optimization
The main goal of this paper is to develop uniformly optimal first-order methods for large-scale convex programming (CP). By uniform optimality we mean that the first-order methods themselves do not require the input of any problem parameters, but can still achieve the best possible iteration complexity bounds. To this end, we provide a substantial generalization of the accelerated level method ...
متن کاملAn efficient one-layer recurrent neural network for solving a class of nonsmooth optimization problems
Constrained optimization problems have a wide range of applications in science, economics, and engineering. In this paper, a neural network model is proposed to solve a class of nonsmooth constrained optimization problems with a nonsmooth convex objective function subject to nonlinear inequality and affine equality constraints. It is a one-layer non-penalty recurrent neural network based on the...
متن کامل