Penalized Regression with Model-Based Penalties
نویسنده
چکیده
Nonparametric regression techniques such as spline smoothing and local tting depend implicitly on a parametric model. For instance, the cubic smoothing spline estimate of a regression function based on observations ti; Yi is the minimizer of P(Yi (ti))2 + R ( 00)2. Since R ( 00)2 is zero when is a line, the cubic smoothing spline estimate favors the parametric model (t) = 0+ 1t: Here we consider replacing R ( 00)2 with the more general expression R (L )2 where L is a linear di erential operator with possibly nonconstant coe cients. The resulting estimate of performs well, particularly if L is small. We present O(n) algorithms for the computation of and suggest several methods for the estimation of L. We study our estimates via simulation and apply them to several data sets.
منابع مشابه
Model Selection in Gaussian Regression for High-dimensional Data
We consider model selection in Gaussian regression, where the number of predictors might be even larger than the number of observations. The proposed procedure is based on penalized least square criteria with a complexity penalty on a model size. We discuss asymptotic properties of the resulting estimators corresponding to linear and so-called 2k ln(p/k)-type nonlinear penalties for nearly-orth...
متن کاملGeneralized Linear Model Regression under Distance-to-set Penalties
Estimation in generalized linear models (GLM) is complicated by the presence of constraints. One can handle constraints by maximizing a penalized log-likelihood. Penalties such as the lasso are effective in high dimensions, but often lead to unwanted shrinkage. This paper explores instead penalizing the squared distance to constraint sets. Distance penalties are more flexible than algebraic and...
متن کاملSNP Selection in Genome-Wide and Candidate Gene Studies via Penalized Logistic Regression
Penalized regression methods offer an attractive alternative to single marker testing in genetic association analysis. Penalized regression methods shrink down to zero the coefficient of markers that have little apparent effect on the trait of interest, resulting in a parsimonious subset of what we hope are true pertinent predictors. Here we explore the performance of penalization in selecting ...
متن کاملSplines, Knots, and Penalties
Penalized splines have gained much popularity as a flexible tool for smoothing and semi-parametric models. Two approaches have been advocated: 1) use a B-spline basis, equally-spaced knots and difference penalties (Eilers and Marx, 1996) and 2) use truncated power functions, knots based on quantiles of the independent variable and a ridge penalty (Ruppert, Wand and Carroll, 2003). We compare th...
متن کاملPenalized Regression for Genome-Wide Association Screening of Sequence Data
Whole exome and whole genome sequencing are likely to be potent tools in the study of common diseases and complex traits. Despite this promise, some very difficult issues in data management and statistical analysis must be squarely faced. The number of rare variants identified by sequencing is apt to be much larger than the number of common variants encountered in current association studies. T...
متن کاملPenalized likelihood regression for generalized linear models with nonquadratic penalties
One popular method for fitting a regression function is regularization: minimize an objective function which enforces a roughness penalty in addition to coherence with the data. This is the case when formulating penalized likelihood regression for exponential families. Most smoothing methods employ quadratic penalties, leading to linear estimates, and are in general incapable of recovering disc...
متن کامل