The affine rank minimization problem, which consists of finding a matrix of minimum rank subject to linear equality constraints, has been proposed in many areas of engineering and science. A specific rank minimization problem is the matrix completion problem, in which we wish to recover a (low-rank) data matrix from incomplete samples of its entries. A recent convex relaxation of the rank minim...

We present a unifying framework for nonsmooth convex minimization bringing together -subgradient algorithms and methods for the convex feasibility problem. This development is a natural step for -subgradient methods in the direction of constrained optimization since the Euclidean projection frequently required in such methods is replaced by an approximate projection, which is often easier to co...

in this paper, we study the problem of minimizing the ratio of two quadratic functions subject to a quadratic constraint. first we introduce a parametric equivalent of the problem. then a bisection and a generalized newton-based method algorithms are presented to solve it. in order to solve the quadratically constrained quadratic minimization problem within both algorithms, a semidefinite optim...

In regularized risk minimization, the associated optimization problem becomes particularly difficult when both the loss and regularizer are nonsmooth. Existing approaches either have slow or unclear convergence properties, are restricted to limited problem subclasses, or require careful setting of a smoothing parameter. In this paper, we propose a continuation algorithm that is applicable to a ...

We analyze stochastic algorithms for optimizing nonconvex, nonsmooth finite-sum problems, where the nonsmooth part is convex. Surprisingly, unlike the smooth case, our knowledge of this fundamental problem is very limited. For example, it is not known whether the proximal stochastic gradient method with constant minibatch converges to a stationary point. To tackle this issue, we develop fast st...

Given a convex optimization problem and its dual, there are many possible firstorder algorithms. In this paper, we show the equivalence between mirror descent algorithms and algorithms generalizing the conditional gradient method. This is done through convex duality and implies notably that for certain problems, such as for supervised machine learning problems with nonsmooth losses or problems ...

Abstract: We consider a semi-infinite optimization problem in Banach spaces, where both the objective functional and the constraint operator are compositions of convex nonsmooth mappings and differentiable mappings. We derive necessary optimality conditions for these problems. Finally, we apply these results to nonconvex stochastic optimization problems with stochastic dominance constraints, ge...

In this paper, we propose a penalty-based recurrent neural network for solving a class of constrained optimization problems with generalized convex objective functions. The model has a simple structure described by using a differential inclusion. It is also applicable for any nonsmooth optimization problem with affine equality and convex inequality constraints, provided that the objective funct...

To the moment we have more or less complete impression of what is the complexity of solving general nonsmooth convex optimization problems and what are the corresponding optimal methods. In this lecture we treat a new topic: optimal methods for smooth convex minimization. We shall start with the simplest case of unconstrained problems with smooth convex objective. Thus, we shall be interested i...