Recently several, so-called, proximal Newton methods were proposed for sparse optimization [6, 11, 8, 3]. These methods construct a composite quadratic approximation using Hessian information, optimize this approximation using a first-order method, such as coordinate descent and employ a line search to ensure sufficient descent. Here we propose a general framework, which includes slightly modif...

The Modal-Shift Transportation Planning Problem (MSTPP) is the problem that finds a feasible schedule for carriers with the minimum total cost when sets of facilities, delivery orders, and carriers are given. In this paper, we propose a fast steepest descent algorithm to solve the MSTPP. Our solution generates a set of candidate routes for each delivery order as a preprocess. Then, it finds a s...

In this paper, based on the efficient Conjugate Descent ({\tt CD}) method, two generalized {\tt CD}algorithms are proposed to solve unconstrained optimization problems.These methods three-term conjugate gradient which generateddirections by using parameters and independent of line searchsatisfy in sufficient descent condition. Furthermore, under strong Wolfe search,the global convergence proved...

The steepest descent method for large linear systems is well-known to often converge very slowly, with the number of iterations required being about the same as that obtained by utilizing a gradient descent method with the best constant step size and growing proportionally to the condition number. Faster gradient descent methods must occasionally resort to significantly larger step sizes, which...

Coordinate descent algorithms solve optimization problems by successively performing approximate minimization along coordinate directions or coordinate hyperplanes. They have been used in applications for many years, and their popularity continues to grow because of their usefulness in data analysis, machine learning, and other areas of current interest. This paper describes the fundamentals of...

In this paper, we make a modification to the LS conjugate gradient method and propose a descent LS method. The method can generates sufficient descent direction for the objective function. We prove that the method is globally convergent with an Armijo-type line search. Moreover, under mild conditions, we show that the method is globally convergent if the Armijo line search or the Wolfe line sea...

A method is proposed for the construction of descent directions for the minimization of energy functionals defined for plane curves. The method is potentially useful in a number of image analysis problems, such as image registration and shape warping, where the standard gradient descent curve evolutions are not always feasible. The descent direction is constructed by taking a weighted average o...

Based on the empirical or theoretical qualitative information about the relationship between response variable and covariates, we propose a new approach to model polynomial regression using a shape restricted regression after estimating the direction by sufficient dimension reduction. The purpose of this paper is to illustrate that in the absence of prior information other than the shape constr...

Cascaded Regression (CR) based methods have been proposed to solve facial landmarks detection problem, which learn a series of descent directions by multiple cascaded regressors separately trained in coarse and fine stages. They outperform the traditional gradient descent based methods in both accuracy and running speed. However, cascaded regression is not robust enough because each regressor’s...