On constrained and regularized high-dimensional regression.
نویسندگان
چکیده
High-dimensional feature selection has become increasingly crucial for seeking parsimonious models in estimation. For selection consistency, we derive one necessary and sufficient condition formulated on the notion of degree-of-separation. The minimal degree of separation is necessary for any method to be selection consistent. At a level slightly higher than the minimal degree of separation, selection consistency is achieved by a constrained L0-method and its computational surrogate-the constrained truncated L1-method. This permits up to exponentially many features in the sample size. In other words, these methods are optimal in feature selection against any selection method. In contrast, their regularization counterparts-the L0-regularization and truncated L1-regularization methods enable so under slightly stronger assumptions. More importantly, sharper parameter estimation/prediction is realized through such selection, leading to minimax parameter estimation. This, otherwise, is impossible in absence of a good selection method for high-dimensional analysis.
منابع مشابه
Elementary Estimators for High-Dimensional Linear Regression
We consider the problem of structurally constrained high-dimensional linear regression. This has attracted considerable attention over the last decade, with state of the art statistical estimators based on solving regularized convex programs. While these typically non-smooth convex programs can be solved by the state of the art optimization methods in polynomial time, scaling them to very large...
متن کاملThe effect of boundary conditions on the accuracy and stability of the numerical solution of fluid flows by Lattice-Boltzmann method
The aim of this study is to investigate the effect of boundary conditions on the accuracy and stability of the numerical solution of fluid flows in the context of single relaxation time Lattice Boltzmann method (SRT-LBM). The fluid flows are simulated using regularized, no-slip, Zou-He and bounce back boundary conditions for straight surfaces in a lid driven cavity and the two-dimensional flow ...
متن کاملLikelihood-based selection and sharp parameter estimation.
In high-dimensional data analysis, feature selection becomes one means for dimension reduction, which proceeds with parameter estimation. Concerning accuracy of selection and estimation, we study nonconvex constrained and regularized likelihoods in the presence of nuisance parameters. Theoretically, we show that constrained L(0)-likelihood and its computational surrogate are optimal in that the...
متن کاملCovariance-regularized regression and classification for high-dimensional problems.
In recent years, many methods have been developed for regression in high-dimensional settings. We propose covariance-regularized regression, a family of methods that use a shrunken estimate of the inverse covariance matrix of the features in order to achieve superior prediction. An estimate of the inverse covariance matrix is obtained by maximizing its log likelihood, under a multivariate norma...
متن کاملIterated Local Search Algorithm for the Constrained Two-Dimensional Non-Guillotine Cutting Problem
An Iterated Local Search method for the constrained two-dimensional non-guillotine cutting problem is presented. This problem consists in cutting pieces from a large stock rectangle to maximize the total value of pieces cut. In this problem, we take into account restrictions on the number of pieces of each size required to be cut. It can be classified as 2D-SLOPP (two dimensional single large o...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
برای دانلود متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید
ثبت ناماگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید
ورودعنوان ژورنال:
- Annals of the Institute of Statistical Mathematics
دوره 65 5 شماره
صفحات -
تاریخ انتشار 2013