Nonlinear least squares and maximum likelihood estimation of a heteroscedastic regression model

Least squares and maximum likelihood

Estimation by Least Squares and by Maximum Likelihood

Although statements to the contrary are often made, application of the principle of least squares is not limited to situations in which p is normally distributed. The GaussMarkov theorem is to the effect that, among unbiased estimates which are linear functions of the observations, those yielded by least squares have minimum variance, and the independence of this property from any assumption re...

" - a Comparison between Maximum Likelihood and Generalized Least Squares in a Heteroscedastic Linear

We consider a linear model with normally distributed but heteroscedastic errors. When the error variances are functionally related to the regression parameter, one can use either maximum likelihood or generalized least squares to estimate the regression parameter. We show that maximum likelihood is much more sensitive to small misspecifications in the functional relationship between the error v...

Maximum Likelihood Recursive Least Squares Estimation for Multivariable Systems

This paper discusses parameter estimation problems of the multivariable systems described by input–output difference equations. We decompose a multivariable system to several subsystems according to the number of the outputs. Based on the maximum likelihood principle, a maximum likelihood-based recursive least squares algorithm is derived to estimate the parameters of each subsystem. Finally, t...

Nonlinear Least-squares Estimation

The paper uses empirical process techniques to study the asymptotics of the least-squares estimator for the fitting of a nonlinear regression function. By combining and extending ideas of Wu and Van de Geer, it establishes new consistency and central limit theorems that hold under only second moment assumptions on the errors. An application to a delicate example of Wu’s illustrates the use of t...