The Levenberg-Marquardt (LM) algorithm is an iterative technique that locates the minimum of a function that is expressed as the sum of squares of nonlinear functions. It has become a standard technique for nonlinear least-squares problems and can be thought of as a combination of steepest descent and the Gauss-Newton method. This document briefly describes the mathematics behind levmar, a free...

When a physical system is modeled by nonlinear function, the unknown parameters can be estimated fitting experimental observations least-squares approach. Newton's method and its variants are often used to solve problems of this type. In paper, we concerned with computation minimal-norm solution an underdetermined problem. We present Gauss-Newton type method, which relies on two relaxation ensu...

The 2010 study of the Shannon entropy of order nine Sudoku and Latin square matrices by Newton and DeSalvo [Proc. Roy. Soc. A 2010] is extended to natural magic and Latin squares up to order nine. We demonstrate that decimal and integer measures of the Singular Value sets, here named SV clans, are a powerful way of comparing different integer squares. Several complete sets of magic and Latin sq...

(1998) have formulated an important and practical problem: How to smooth the noise out of image data while at the same time preserving unsmooth features such as jumps, spikes and edges? They formalize the problem as the estimation of a mostly smooth regression function function, m(x), based on observations Y i that decompose as follows: By assuming m satisses standard nonparametric smoothness a...

This paper reports theoretical and empirical investigations on the use of quasi-Newton methods to minimize the Optimal Bellman Residual (OBR) of zero-sum two-player Markov Games. First, it reveals that state-of-the-art algorithms can be derived by the direct application of Newton’s method to different norms of the OBR. More precisely, when applied to the norm of the OBR, Newton’s method results...

In this paper, we derive and discuss a new adaptive quasi-Newton eigen-estimation algorithm and compare it with the RLS-type adaptive algorithms and the quasi-Newton algorithm proposed by Mathew et al. through experiments with stationary and nonstationary data.

Outline Motivation / algorithmic pairs trading Model setup Detection of local mean-reversion Adaptive estimation 1. RLS with gradient variable forgetting factor 2. RLS with Gauss-Newton variable forgetting factor 3. RLS with beta-Bernoulli forgetting factor Trading strategy Pepsi and Coca Cola example Introduction Statistical arbitrage. Algorithmic pairs trading market neutral trading. Buy low,...

We discuss the solution of numerically ill-posed overdetermined systems of equations using Tikhonov a-priori-based regularization. When the noise distribution on the measured data is available to appropriately weight the fidelity term, and the regularization is assumed to be weighted by inverse covariance information on the model parameters, the underlying cost functional becomes a random varia...

Newton, in an unauthorized textbook, described a process for solving simultaneous equations that later authors applied specifically to linear equations. This method — that Newton did not want to publish, that Euler did not recommend, that Legendre called “ordinary,” and that Gauss called “common” — is now named after Gauss: “Gaussian” elimination. (One suspects, he would not be amused.) Gauss’s...

Newton-step approximations to pseudo maximum likelihood estimates of spatial autoregressive models with a large number parameters are examined, in the sense that parameter space grows slowly as function sample size. These have same asymptotic efficiency properties under Gaussianity but closed form. Hence they computationally simple and free from compactness assumptions, thereby avoiding two not...