Non-parametric Sequential Estimation of a Regression Function Based on Dependent Observations

This paper presents a sequential estimation procedure for an unknown regression function. Observed regressors and noises of the model are supposed to be dependent and form sequences of dependent numbers. Two types of estimators are considered. Both estimators are constructed on the basis of Nadaraya–Watson kernel estimators. First, sequential estimators with given bias and mean square error are defined. According to the sequential approach the duration of observations is a special stopping time. Then on the basis of these estimators of a regression function, truncated sequential estimators on a time interval of a fixed length are constructed. At the same time the variance of these estimators is controlled by a (non-asymptotic) bound. In addition to non-asymptotic properties, the limiting behavior of presented estimators is investigated. It is shown, in particular, that by the appropriate chosen bandwidths both estimators have optimal (as compared to the case of independent data) rates of convergence of Nadaraya–Watson kernel estimators.