Abstract We study stochastic projection-free methods for constrained optimization of smooth functions on Riemannian manifolds, i.e., with additional constraints beyond the parameter domain being a manifold. Specifically, we introduce Frank–Wolfe (Fw) nonconvex and geodesically convex problems. present algorithms both purely finite-sum For latter, develop variance-reduced methods, including adap...

Quantum annealers aim at solving nonconvex optimization problems by exploiting cooperative tunneling effects to escape local minima. The underlying idea consists of designing a classical energy function whose ground states are the sought optimal solutions of the original optimization problem and add a controllable quantum transverse field to generate tunneling processes. A key challenge is to i...

Regularization has played a key role in deriving sensible estimators in high dimensional statistical inference. A substantial amount of recent works has argued for nonconvex regularizers in favor of their superior theoretical properties and excellent practical performances. In a different but analogous vein, nonconvex loss functions are promoted because of their robustness against “outliers”. H...

Sparse modeling has been highly successful in many realworld applications. While a lot of interests have been on convex regularization, recent studies show that nonconvex regularizers can outperform their convex counterparts in many situations. However, the resulting nonconvex optimization problems are often challenging, especially for composite regularizers such as the nonconvex overlapping gr...

Nonconvex Stochastic Optimization stochastic optimization problems arise in many machine learning problems, including deep learning. The gradient Hamiltonian Monte Carlo (SGHMC) is a variant of gradients with momentum method which controlled and properly scaled Gaussian noise added to the steer iterates toward global minimum. SGHMC has shown empirical success practice for solving nonconvex prob...

We study a class of large-scale, nonsmooth, and nonconvex optimization problems. In particular, we focus on nonconvex problems with composite objectives. This class includes the extensively studied class of convex composite objective problems as a subclass. To solve composite nonconvex problems we introduce a powerful new framework based on asymptotically nonvanishing errors, avoiding the commo...

Abstract. Nonsmooth nonconvex regularization has remarkable advantages for the restoration of piecewise constant images. Constrained optimization can improve the image restoration using a priori information. In this paper, we study regularized nonsmooth nonconvex minimization with box constraints for image restoration. We present a computable positive constant θ for using nonconvex nonsmooth re...

In this paper, we consider a stochastic distributed nonconvex optimization problem with the cost function being over n agents having access only to zeroth-order (ZO) information of cost. This has various machine learning applications. As solution, propose two ZO algorithms, in which at each iteration agent samples local oracle points time-varying smoothing parameter. We show that proposed algor...

Distributed multi-agent optimization finds many applications in distributed learning, control, estimation, etc. Most existing algorithms assume knowledge of first-order information the objective and have been analyzed for convex problems. However, there are situations where is nonconvex, one can only evaluate function values at finitely points. In this paper we consider derivative-free nonconve...