Multistep kernel regression smoothing by boosting
نویسندگان
چکیده
In this paper we propose a simple multistep regression smoother which is constructed in a boosting fashion, by learning the Nadaraya–Watson estimator with L2Boosting. Differently from the usual approach, we do not focus on L2Boosting for ever. Given a kernel smoother as a learner, we explore the boosting capability to build estimators using a finite number of boosting iterations. This approach appears fruitful since it simplifies the boosting interpretation and application. We find, in both theoretical analysis and simulation experiments, that higher order bias properties emerge. Relationships between our smoother and previous work are explored. Moreover, we suggest a way to successfully employ our method for estimating probability density functions (pdf) and cumulative distribution functions (cdf) via binning procedures and the smoothing of the empirical cumulative distribution function, respectively. The practical performance of the method is illustrated by a large simulation study which shows an encouraging finite sample behaviour paricularly in comparison with other methods.
منابع مشابه
On boosting kernel regression
In this paper we propose a simple multistep regression smoother which is constructed in an iterative manner, by learning the Nadaraya-Watson estimator with L2boosting. We find, in both theoretical analysis and simulation experiments, that the bias converges exponentially fast, and the variance diverges exponentially slow. The first boosting step is analyzed in more detail, giving asymptotic exp...
متن کاملOn boosting kernel density methods for multivariate data: density estimation and classification
Statistical learning is emerging as a promising field where a number of algorithms from machine learning are interpreted as statistical methods and vice–versa. Due to good practical performance, boosting is one of the most studied machine learning techniques. We propose algorithms for multivariate density estimation and classification. They are generated by using the traditional kernel techniqu...
متن کاملKernel Density Classification and Boosting
Kernel density estimation is a commonly used approach to classification. However, most of the theoretical results for kernel methods apply to estimation per se and not necessarily to classification. For example, in estimating the difference between two densities, we show that the optimal smoothing parameters are increasing functions of the sample size of the complementary group. A relative newc...
متن کاملKernel density classification and boosting: an L2 analysis
Kernel density estimation is a commonly used approach to classification. However, most of the theoretical results for kernel methods apply to estimation per se and not necessarily to classification. In this paper we show that when estimating the difference between two densities, the optimal smoothing parameters are increasing functions of the sample size of the complementary group, and we provi...
متن کاملDerivation of Equivalent Kernel for General Spline Smoothing: a Systematic Approach
We consider ®rst the spline smoothing nonparametric estimation with variable smoothing parameter and arbitrary design density function and show that the corresponding equivalent kernel can be approximated by the Green function of a certain linear differential operator. Furthermore, we propose to use the standard (in applied mathematics and engineering) method for asymptotic solution of linear d...
متن کامل