Stochastic First-Order Methods with Random Constraint Projection

Stochastic First-Order Methods with Random Constraint Projection

We consider convex optimization problems with structures that are suitable for sequential treatment or online sampling. In particular, we focus on problems where the objective function is an expected value, and the constraint set is the intersection of a large number of simpler sets. We propose an algorithmic framework for stochastic first-order methods using random projection/proximal updates ...

Random Multi-Constraint Projection: Stochastic Gradient Methods for Convex Optimization with Many Constraints

Consider convex optimization problems subject to a large number of constraints. We focus on stochastic problems in which the objective takes the form of expected values and the feasible set is the intersection of a large number of convex sets. We propose a class of algorithms that perform both stochastic gradient descent and random feasibility updates simultaneously. At every iteration, the alg...

Autoregressive Gaussian Random Vectors of First Order

Stochastic First- and Zeroth-order Methods for Nonconvex Stochastic Programming

In this paper, we introduce a new stochastic approximation (SA) type algorithm, namely the randomized stochastic gradient (RSG) method, for solving an important class of nonlinear (possibly nonconvex) stochastic programming (SP) problems. We establish the complexity of this method for computing an approximate stationary point of a nonlinear programming problem. We also show that this method pos...

Multilevel Projection Methods for First - Order SystemLeast Squares

This paper focuses on the multilevel projection method (PML) applied to numerical solution of the basic equations of uid dynamics formulated as least-squares problems for rst-order systems. The fundamental approach taken here is to pose the uid ow equations in their rst-order form, incorporate additional (usually redundant) equations where they improve the character of the formulation, and deen...