A faster model selection criterion for OP-ELM and OP-KNN: Hannan-Quinn criterion
نویسندگان
چکیده
The Optimally Pruned Extreme Learning Machine (OPELM) and Optimally Pruned K-Nearest Neighbors (OP-KNN) algorithms use the a similar methodology based on random initialization (OP-ELM) or KNN initialization (OP-KNN) of a Feedforward Neural Network followed by ranking of the neurons; ranking is used to determine the best combination to retain. This is achieved by Leave-One-Out (LOO) crossvalidation. In this article is proposed to use the Hannan-Quinn (HQ) Criterion as a model selection criterion, instead of LOO. It proved to be efficient and as good as the LOO one for both OP-ELM and OP-KNN, while decreasing computations by factors of four to five for OP-ELM and up to 24 for OP-KNN.
منابع مشابه
Bayesian model selection in ARFIMA models
Keywords: Bayesian model selection Reversible jump Markov chain Monte Carlo Autoregressive fractional integrated moving average models Long memory processes a b s t r a c t Various model selection criteria such as Akaike information criterion (AIC; Akaike, 1973), Bayesian information criterion (BIC; Akaike, 1979) and Hannan–Quinn criterion (HQC; Hannan, 1980) are used for model specification in...
متن کاملA theoretical investigation of several model selection criteria for dimensionality reduction
Based on the problem of determining the hidden dimensionality (or the number of latent factors) of Factor Analysis (FA) model, this paper provides a theoretic comparison on several classical model selection criteria, including Akaike’s Information Criterion (AIC), Bozdogan’s Consistent Akaike’s Information Criterion (CAIC), Hannan–Quinn information criterion (HQC), Schwarz’s Bayesian Informatio...
متن کاملVariance Estimates and Model Selection
The large majority of the criteria for model selection are functions of the ̂ 2, the usual variance estimate for a regression model. The validity of the usual variance estimate depends on some assumptions, most critically the validity of the model being estimated. This is often violated in model selection contexts, where model search takes place over invalid models. A cross validated estimate of...
متن کاملModel selection for integrated autoregressive processes of infinite order
Choosing good predictive models is an important ingredient in a great deal of statistical research. When the true model is relatively simple and can be parameterized by a prescribed finite set of parameters whose values are unknown, it is natural to ask whether a model selection criterion can exclude all redundant parameters, thereby achieving prediction efficiency through the most parsimonious...
متن کاملAn Investigation of Several Typical Model Selection Criteria for Detecting the Number of Signals
Based on the problem of detecting the number of signals, this paper provides a systematic empirical investigation on model selection performances of several classical criteria and recently developed methods (including Akaike’s information criterion (AIC), Schwarz’s Bayesian information criterion, Bozdogan’s consistent AIC, Hannan-Quinn information criterion, Minka’s (MK) principal component ana...
متن کامل