Boosting for High-multivariate Responses in High-dimensional Linear Regression

We propose a boosting method, multivariate L2Boosting, for multivariate linear regression based on some squared error loss for multivariate data. It can be applied to multivariate linear regression with continuous responses and to vector autoregressive time series. We prove, for i.i.d. as well as time series data, that multivariate L2Boosting can consistently recover sparse high-dimensional multivariate linear functions, even when the number of predictor variables pn and the dimension of the response qn grow almost exponentially with sample size n, pn = qn = O(exp(Cn )) (0 < ξ < 1, 0 < C < ∞), but the `1-norm of the true underlying function is finite. Our theory seems to be among the first to address the issue of large dimension of the response variable; the relevance of such settings is briefly outlined. We also identify empirically some cases where our multivariate L2Boosting is better than multiple application of univariate methods to single response components, thus demonstrating that the multivariate approach can be very useful.