The paper deals with the multilevel solution of elliptic partial differential equations (PDEs) in a finite element setting: uniform ellipticity of the PDE then goes with strict monotonicity of the derivative of a nonlinear convex functional. A Newton multigrid method is advocated, wherein linear residuals are evaluated within the multigrid method for the computation of the Newton corrections. T...

Mixture of experts (ME) is a modular neural network architecture for supervised learning. A double-loop Expectation-Maximization (EM) algorithm has been introduced to the ME architecture for adjusting the parameters and the iteratively reweighted least squares (IRLS) algorithm is used to perform maximization in the inner loop [Jordan, M.I., Jacobs, R.A. (1994). Hierarchical mixture of experts a...

We propose a DC proximal Newton algorithm for solving nonconvex regularized sparse learning problems in high dimensions. Our proposed algorithm integrates the proximal Newton algorithm with multi-stage convex relaxation based on difference of convex (DC) programming, and enjoys both strong computational and statistical guarantees. Specifically, by leveraging a sophisticated characterization of ...

We consider concave minimization problems over nonconvex sets. Optimization problems with this structure arise in sparse principal component analysis. We analyze both a gradient projection algorithm and an approximate Newton algorithm where the Hessian approximation is a multiple of the identity. Convergence results are established. In numerical experiments arising in sparse principal component...

We develop a globally convergent algorithm based on the LP-Newton method, which has been recently proposed for solving constrained equations, possibly nonsmooth and possibly with nonisolated solutions. The new algorithm makes use of linesearch for the natural merit function and preserves the strong local convergence properties of the original LP-Newton scheme. We also present computational expe...

A new registration algorithm based on Newton-Raphson iteration is proposed to align images with rigid body transformation. A set of transformation parameters consisting of translation in x and y and rotation angle around z is calculated by optimizing a specified similarity metric using the Newton-Raphson method. This algorithm has been tested by registering and correlating pairs of topography m...

We propose a DC proximal Newton algorithm for solving nonconvex regularized sparse learning problems in high dimensions. Our proposed algorithm integrates the proximal newton algorithm with multi-stage convex relaxation based on the difference of convex (DC) programming, and enjoys both strong computational and statistical guarantees. Specifically, by leveraging a sophisticated characterization...

Interpolation has wide application in signal processing, numerical integration, Computer Aided Geometric Design (CAGD), engineering technology and electrochemistry. Block based bivariate Newton-like blending rational interpolation can also be calculated based on information matrix algorithm in addition to block divided differences. The paper studied interpolation theorem, dual interpolation of ...

We develop a globally convergent algorithm based on the LP-Newton method which has been recently proposed for solving constrained equations, possibly nonsmooth and possibly with nonisolated solutions. The new algorithm makes use of linesearch for the natural merit function, and preserves the strong local convergence properties of the original LP-Newton scheme. We also present computational expe...

The adjoint Newton algorithm (ANA) is based on the firstand second-order adjoint techniques allowing one to obtain the ‘‘Newton line search direction’’ by integrating a ‘‘tangent linear model’’ backward in time (with negative time steps). Moreover, the ANA provides a new technique to find Newton line search direction without using gradient information. The error present in approximating the Hes...