Absolute Penalty and Shrinkage Estimation Strategies in Linear and Partially Linear Models

In this dissertation we studied asymptotic properties of shrinkage estimators, and compared their performance with absolute penalty estimators (APE) in linear and partially linear models (PLM). A robust shrinkage M-estimator is proposed for PLM, and asymptotic properties are investigated, both analytically and through simulation studies. In Chapter 2, we compared the performance of shrinkage and some APEs through prediction error criterion in a multiple linear regression setup. In particular, we compared shrinkage estimators with lasso, adaptive lasso and SCAD estimators. Monte Carlo studies were conducted to compare the estimators in two situations: when p << n, and when p is large yet p < n. Examples using some real data sets are presented to illustrate the usefulness of the suggested methods. In Chapter 3, we developed shrinkage estimators for a PLM. Efficient procedures for simultaneous sub-model selection and shrinkage estimation have been developed and implemented to obtain the parameter estimates where the nonparametric component is estimated using B-spline basis expansion. The proposed shrinkage estimator performed similarly to adaptive lasso estimators. In overall comparison, shrinkage estimators based on B-splines outperformed the lasso for moderate sample sizes and when the nuisance parameter space is large. In Chapter 4, we proposed robust shrinkage M-estimators in a PLM with scaled residuals. Ahmed et al. (2006) considered such an M-estimator in a linear regression setup. We extended their work to a PLM.