In this paper, we propose an algorithmic framework, dubbed inertial alternating direction methods of multipliers (iADMM), for solving a class nonconvex nonsmooth multiblock composite optimization problems with linear constraints. Our framework employs the general minimization-majorization (MM) principle to update each block variables so as not only unify convergence analysis previous ADMM that ...

While convex optimization for low-light imaging has received some attention by the imaging community, non-convex optimization techniques for photon-limited imaging are still in their nascent stages. In this thesis, we developed a stagebased non-convex approach to recover high-resolution sparse signals from low-dimensional measurements corrupted by Poisson noise. We incorporate gradient-based in...

We study stochastic algorithms for solving non-convex optimization problems with a convex yet possibly non-smooth regularizer, which nd wide applications in many practical machine learning applications. However, compared to asynchronous parallel stochastic gradient descent (AsynSGD), an algorithm targeting smooth optimization, the understanding of the behavior of stochastic algorithms for the n...

This work studies the strong duality of non-convex matrix factorization problems: we show that under certain dual conditions, these problems and its dual have the same optimum. This has been well understood for convex optimization, but little was known for non-convex problems. We propose a novel analytical framework and show that under certain dual conditions, the optimal solution of the matrix...

Non-convex sparsity-inducing penalties have recently received considerable attentions in sparse learning. Recent theoretical investigations have demonstrated their superiority over the convex counterparts in several sparse learning settings. However, solving the non-convex optimization problems associated with non-convex penalties remains a big challenge. A commonly used approach is the Multi-S...

In this paper, we examine a class of non-convex stochastic optimization problems which we call variationally coherent, and which properly includes pseudo-/quasiconvex and star-convex optimization problems. To solve such problems, we focus on the widely used stochastic mirror descent (SMD) family of algorithms (which contains stochastic gradient descent as a special case), and we show that the l...

Conjugate gradient methods are a class of important methods for solving linear equations and nonlinear optimization. In our work, we propose a new stochastic conjugate gradient algorithm with variance reduction (CGVR) and prove its linear convergence with the Fletcher and Revves method for strongly convex and smooth functions. We experimentally demonstrate that the CGVR algorithm converges fast...

Optimization algorithms have been recently applied to solver problems where data possess certain randomness, partly because data themselves contain randomness in a big-data environment or data are randomly sampled from their populations. It has been shown that data randomness typically makes algorithms run faster in the so-called “average behavior analysis”. In this short note, we give an examp...

Compressed sensing is a new sampling technique which can exactly reconstruct sparse signal from a few measurements. In this article, we consider the blocksparse compressed sensing with special structure assumption about the signal. A novel non-convex model is proposed to reconstruct the block-sparse signals. In addition, the conditions of the proposed model for recovering the block-sparse noise...

In this paper the path planning of robots is formulated in the context of an optimization framework. It is shown that generating motion trajectories for mobile robots in real environment is cast into a non-convex problem. Global optimization techniques in the non-convex framework are thus exploited to obtain the desired trajectories. The approach presented in this paper can easily take the robo...