In this paper we study the impact of network size on the performance of incremental least mean square (ILMS) adaptive networks. Specifically, we consider two ILMS networks with different number of nodes and compare their performance in two different cases including (i) ideal links and (ii) noisy links. We show that when the links between nodes are ideal, increasing the network size improves the...

Although the least squares estimate is the linear unbiased estimate with minimum variance, it is possible that a biased estimate will give us a better mean squared error. Consider a case where the true model is Y = β0 + β1X1 + β2X2 + and that X1 and X2 are almost perfectly correlated (statisticians say X1 and X2 are co-linear). What happens if we leave X2 out? Then the model is very well approx...

Abstract Assume that observations are generated from a nonstationary autoregressive (AR) processes of infinite order. We adopt a finite-order approximation model to predict future observations and obtain an asymptotic expression for the mean-squared prediction error (MSPE) of the least squares predictor. This expression provides the first exact assessment of the impacts of nonstationarity, mode...

Shrinkage estimation usually reduces variance at the cost of bias. But when we care only about some parameters of a model, I show that we can reduce variance without incurring bias if we have additional information about the distribution of covariates. In a linear regression model with homoscedastic Normal noise, I consider shrinkage estimation of the nuisance parameters associated with control...

A new method is proposed for estimating derivatives of a nonparametric regression function. By applying Taylor expansion technique to a derived symmetric difference sequence, we obtain a sequence of approximate linear regression representation in which the derivative is just the intercept term. Using locally weighted least squares, we estimate the derivative in the linear regression model. The ...

In recent years, the Weibull distribution has been commonly used and recommended to model the wind speed. Therefore, many estimators have been proposed to find the best method to estimate the parameters of the Weibull distribution. Particularly, the estimator based on regression procedures with the Weibull probability plot are often used because of its computational simplicity and graphical pre...

This paper analyzes the behavior of a variety of tracking algorithms for single layer threshold networks in the presence of random drift. We use a system identiication model to model a target network where weights slowly change and a tracking network. Tracking algorithms are divided into conservative and nonconservative algorithms. For a random drift rate of , we nd upper bounds for the general...

This paper compares performance of finite impulse response (FIR) adaptive linear equalizers based on the recursive least-squares (RLS) and least mean square (LMS) algorithms in nonstationary uncorrelated scattering wireless channels. Simulation results, in terms of steady-state mean-square estimation error (MSE) and average bit-error rate (BER) metrics, are found for the frequency-selective Ray...

The Lasso achieves variance reduction and variable selection by solving an 1-regularized least squares problem. Huang (2003) claims that ‘there always exists an interval of regularization parameter values such that the corresponding mean squared prediction error for the Lasso estimator is smaller than for the ordinary least square estimator’. This result is correct. However, its proof in Huang ...