We investigate modified Levenberg-Marquardt methods coupled with a Kaczmarz strategy for obtaining stable solutions of nonlinear systems of ill-posed operator equations. We show that the proposed method is a convergent regularization method.

We propose a generic method for iteratively approximating various second-order gradient steps - Newton, Gauss-Newton, Levenberg-Marquardt, and natural gradient - in linear time per iteration, using special curvature matrix-vector products that can be computed in O(n). Two recent acceleration techniques for on-line learning, matrix momentum and stochastic meta-descent (SMD), implement this appro...

The resection in 3D space is a common problem surveying engineering and photogrammetry based on observed distances, angles, coordinates. This nonlinear comprises redundant observations which normally solved using the least-squares method an iterative approach. In this paper, we introduce vigorous angular that converges to global minimum even with very challenging starting values of unknowns. de...

A batch training algorithm for feed-forward networks is proposed which uses Newton’s method to estimate a vector of optimal learning factors, one for each hidden unit. Backpropagation, using this learning factor vector, is used to modify the hidden unit’s input weights. Linear equations are then solved for the network’s output weights. Elements of the new method’s Gauss-Newton Hessian matrix ar...

This paper focuses on modelling and controlling an unstable chaotic system defined by a set of input–output data. This approach works in two phases: a model of the system is determined, and then a control mechanism is activated to stabilize unstable fixed point and unstable periodic orbit embedded in the chaotic system. In the modelling phase, a Takagi–Sugeno (T–S) fuzzy system with Gaussian fu...

In this work, two modifications on Levenberg-Marquardt algorithm for feedforward neural networks are studied. One modification is made on performance index, while the other one is on calculating gradient information. The modified algorithm gives a better convergence rate compared to the standard Levenberg-Marquard (LM) method and is less computationally intensive and requires less memory. The p...

An approach to develop response surface approximations based upon artificial neural networks trained using both state and sensitivity information is described in this paper. Compared to previous approaches, this approach does not require weighting the residuals of the targets and gradients and is able to approximate gradient-consistent response surfaces with a relatively compact network archite...

Numerical solutions are proposed to fit the CanDecomp/ParaFac (CP) model of real three-way arrays, when the latter are both nonnegative and symmetric in two modes. In other words, a seminonnegative INDSCAL analysis is performed. The nonnegativity constraint is circumvented by means of changes of variable into squares, leading to an unconstrained problem. In addition, two globalization strategie...