We discuss empirical comparison of analytical methods for model selection. Currently, there is no consensus on the best method for finite-sample estimation problems, even for the simple case of linear estimators. This article presents empirical comparisons between classical statistical methods - Akaike information criterion (AIC) and Bayesian information criterion (BIC) - and the structural ris...

This paper could not have been written without the financial and organizational support from Dieter Paulmann and Jo Hastie respectively. Thanks are also due to 2 anonymous reviewers, whose comments on an earlier version of the manuscript greatly improved the paper. The views expressed in this paper are those of the authors alone and do not represent those of Stellwagen Bank National Marine Sanc...

Latent factor models (LFMs) are a set of unsupervised methods that model observed highdimensional data examples by linear combination of latent factors. To enable efficient processing of large data collections, LFMs aim to find concise descriptions of the members of a data collection while preserving the essential statistical information which is useful for basic tasks such as classification, i...

Table 1: Four simulation scenarios used in the evaluation of the bias-variance decomposition. The simulation scenarios are taken from Zou & Hastie (2005). scenario n p β Structure of X (1) 100 8 (3, 1.5, 0, 0, 2, 0, 0, 0) corr (i, j) = 0.5|i−j| (2) 100 8 0.85 for all j corr (i, j) = 0.5|i−j| (3) 50 40 βj = { 0 j = (1, . . . , 10, 21, . . . , 30) 1 j = (11, . . . , 20, 31, . . . , 40) corr (i, j...

Several nonparametric methods in a regression model are presented. First, the most classical ones: piecewise polynomial estimators, estimation with Spline bases, kernel estimators and projection estimators on orthonormal bases (such as Fourier or wavelet bases). Since these methods suffer from the curse of dimensionality, we also present Generalized Additive Models and CART regression models. T...

Neil Vidmar and Reid Hastie would surely be at the top of anyone's list of the leading scholars in the jury research community, David Schkade and John Payne are major figures in the broader judgment and decision-making community. All four are careful, sophisticated theorists and researchers. Any dispute involving the four of them is bound to be a productive one for the field. I will argue that ...

and submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy (Computer Science) complies with the regulations of this University and meets the accepted standards with respect to originality and quality. The subjects of this thesis are unsupervised learning in general, and principal curves in particular. Principal curves were originally defined by Hastie [Has84...

Algorithms for simultaneous shrinkage and selection in regression and classification provide attractive solutions to knotty old statistical challenges. Nevertheless, as far as we can tell, Tibshirani’s Lasso algorithm has had little impact on statistical practice. Two particular reasons for this may be the relative inefficiency of the original Lasso algorithm and the relative complexity of more...

Modelling context eeects and segmental transitions in speech recognition systems is very important. Explicitly modelling segmental transitions in a RNN framework would circumvent these problems. We present an interesting application of Principal Curves, an algorithm to extract a non-linear summary of p-dimensional data rstly published in 1989 by Hastie/Stuetzle. The algorithm can be used to vis...