Variance Estimates and Model Selection

The large majority of the criteria for model selection are functions of the ̂ 2, the usual variance estimate for a regression model. The validity of the usual variance estimate depends on some assumptions, most critically the validity of the model being estimated. This is often violated in model selection contexts, where model search takes place over invalid models. A cross validated estimate of variance is more robust to speci cation errors (see, for example, Efron(1983)). We consider the e ects of replacing the usual estimate of variance by cross-validated estimates in the function of the Akaike Information Criterion (AIC), which is very popular in practice. Such a replacement can be made to any of the criteria involving ̂2 in their functions such as Schwarz Criterion (SC) and Hannan and Quinn Criterion (HQC). In the paper we show that the improvements obtained from all the replacements are similar to eachother. Our Monte-Carlo study, in the context of selecting the right order for an autoregressive process, shows that these replacements improve the performance of criteria for large sample sizes. Moreover forecasts based on the model selected by the criteria using cross-validated estimates of variance are better for both small and large sample sizes.