On Model Selection Consistency of the Elastic Net When p > > n
نویسندگان
چکیده
In this paper, we study the model selection property of the Elastic net. In the classical settings when p (the number of predictors) and q (the number of predictors with non-zero coefficients in the true linear model) are fixed, Yuan and Lin (2007) give a necessary and sufficient condition for the Elastic net to consistently select the true model, which is called the Elastic Irrepresentable Condition (EIC) in this paper. Here we study the general case when p, q and n all go to infinity. For general scalings of p, q and n, when gaussian noise is assumed, sufficient conditions on p, q and n ar given in this paper such that EIC guarantees the Elastic net’s model selection consistency. We show that to make these conditions hold, n should grow at a rate faster than q log(p−q). For the classical case, when p and q are fixed, we also study the relationship between EIC and the Irrepresentable Condition (IC) which is necessary and sufficient for the Lasso to select the true model. Through theoretical results and simulation studies, we provide insights into when and why EIC is weaker than IC and when the Elastic net can consistently select the true model even when the Lasso can not.
منابع مشابه
ON MODEL SELECTION CONSISTENCY OF THE ELASTIC NET WHEN p À n
We study the model selection property of the Elastic Net. In the classical settings when p (the number of predictors) and q (the number of predictors with non-zero coefficients in the true linear model) are fixed, Yuan and Lin (2007) give a necessary and sufficient condition for the Elastic Net to consistently select the true model. They showed that it consistently selects the true model if and...
متن کاملFeature Selection Guided by Structural Information
In generalized linear regression problems with an abundant number of features, lasso-type regularization which imposes an `-constraint on the regression coefficients has become a widely established technique. Crucial deficiencies of the lasso were unmasked when Zhou and Hastie (2005) introduced the elastic net. In this paper, we propose to extend the elastic net by admitting general nonnegative...
متن کاملRegularization and variable selection via the elastic net
We propose the elastic net, a new regularization and variable selection method. Real world data and a simulation study show that the elastic net often outperforms the lasso, while enjoying a similar sparsity of representation. In addition, the elastic net encourages a grouping effect, where strongly correlated predictors tend to be in or out of the model together.The elastic net is particularly...
متن کاملRegularization with the Smooth-Lasso procedure
We consider the linear regression problem. We propose the S-Lasso procedure to estimate the unknown regression parameters. This estimator enjoys sparsity of the representation while taking into account correlation between successive covariates (or predictors). The study covers the case when p ≫ n, i.e. the number of covariates is much larger than the number of observations. In the theoretical p...
متن کاملPre-Selection in Cluster Lasso Methods for Correlated Variable Selection in High-Dimensional Linear Models
We consider variable selection problems in high dimensional sparse regression models with strongly correlated variables. To handle correlated variables, the concept of clustering or grouping variables and then pursuing model fitting is widely accepted. When the dimension is very high, finding an appropriate group structure is as difficult as the original problem. We propose to use Elastic-net a...
متن کامل