We survey and compare model-based approaches to regression for cross-sectional and longitudinal data which extend the classical parametric linear model for Gaussian responses in several aspects and for a variety of settings. Additive models replace the sum of linear functions of regressors by a sum of smooth functions. In dynamic or state space models, still linear in the regressors, coeecients...

We propose a new penalized approach for variable selection using a combination of minimax concave and ridge penalties. The proposed method is designed to deal with p ≥ n problems with highly correlated predictors. We call the propose approach the Mnet method. Similar to the elastic net of Zou and Hastie (2005), the Mnet also tends to select or drop highly correlated predictors together. However...

The present work discusses robust multivariate methods specifically designed for high dimensions. Their implementation in R is presented and their application is illustrated on examples. The first group are algorithms for outlier detection, already introduced elsewhere and implemented in other packages. The value added of the new package is that all methods follow the same design pattern and th...

To reduce the computational complexity of classification systems using tangent distance, Hastie et al. (HSS) developed an algorithm to devise rich models for representing large subsets of the data which computes automatically the "best" associated tangent subspace. Schwenk & Milgram proposed a discriminant modular classification system (Diabolo) based on several autoassociative multilayer perce...

This article describes a fully automated bandwidth selection method for additive models that is applicable to the widely used back tting algorithm of Buja, Hastie and Tibshirani (1989) and does not rely on cross-validation. The proposed plug-in estimator is an extension of the local linear regression estimator of Ruppert, Sheather and Wand (1996) and is shown to achieve the same Op(n 2=7) relat...

1. Use-availability and presence-only analyses are synonyms. Both require two samples (one containing known locations, one containing potential locations), both estimate the same parameters, and both use the same fundamental likelihood. 2. Use-availability and presence-only designs compare characteristics of points where an organism was located to those where the organism could have been locate...

Two important tasks of machine learning are the statistical learning from sample data (SL) and the unsupervised learning from unlabelled data (UL) (Hastie et al., 2001; Theodoridis & Koutroumbas, 2006). The synthesis of the two parts – the unsupervised statistical learning (USL) – is frequently used in the cyclic process of inductive and deductive scientific inference. This applies especially t...

Fithian and Hastie (2014) proposed a new sampling scheme called local case-control (LCC) that achieves stability efficiency by utilizing clever adjustment pertained to the logistic model. It is particularly useful for classification with large imbalanced data. This paper proposes more general based on working principle data points deserve higher probability if they contain information or appear...

Two new methods to select groups of variables have been developed for multiblock data: ”Group Sparse Principal Component Analysis” (GSPCA) for continuous variables and ”Sparse Multiple Correspondence Analysis” (SMCA) for categorical variables. GSPCA is a compromise between Sparse PCA method of Zou, Hastie and Tibshirani and the method ”group Lasso” of Yuan and Lin. PCA is formulated as a regres...

High-dimensional statistics refers to statistical inference when the number of unknown parameters p is much larger than the sample size n. This includes regression and supervised classification models, when the number of covariates is of much larger order than n, and unsupervised settings, such as clustering, with more variables than observations. Image processing, information retrieval in text...