MOTIVATION Estimation of misclassification error has received increasing attention in clinical diagnosis and bioinformatics studies, especially in small sample studies with microarray data. Current error estimation methods are not satisfactory because they either have large variability (such as leave-one-out cross-validation) or large bias (such as resubstitution and leave-one-out bootstrap). W...

Representations are fundamental to artificial intelligence. The performance of a learning system depends on the type of representation used for representing the data. Typically, these representations are hand-engineered using domain knowledge. More recently, the trend is to learn these representations through stochastic gradient descent in multi-layer neural networks, which is called backprop. ...

Predictive models ground many state-of-the-art developments in statistical brain image analysis: decoding, MVPA, searchlight, or extraction of biomarkers. The principled approach to establish their validity and usefulness is cross-validation, testing prediction on unseen data. Here, I would like to raise awareness on error bars of cross-validation, which are often underestimated. Simple experim...

In most cases of QSAR modelling the final model used to make predictions, is not known a priori but has to be selected in a data driven fashion (e.g. selection of principal components, variable selection, selection of the best mathematical modelling technique). Reliable estimation of externally validated prediction errors under this model uncertainty is still a challenge in chemoinformatics. To...

an alumina supported co/ni catalyst was prepared by sol-gel procedure to study the catalytic behavior during fischer-tropsch synthesis in a fixed-bed reactor. the effect of co conversion (10-50%) on hydrocarbon product distribution (ch4, c5+ and c2-c4 olefin selectivities) was studied. selectivity for ch4 decreased, while those of c5+ and olefin selectivities increased with increasing co conver...

Modern steganalysis is a combination of a feature space design and a supervised binary classification. In this report, we assume that the feature space has been already constructed, i.e., the steganalyst has a set of training features and needs to train a binary classifier. Any machine learning tool can be used for this task and its parameters can be tuned through cross-validation, a standard a...

In this short paper we briefly discuss a moment based method that was recently introduced to study the behavior classification algorithms and model validation techniques for finite sample sizes. The method involves accurate and efficient computation of the moments of the generalization error which are over the space of all possible datasets of size N drawn from an underlying distribution. A cla...

purpose: to determine the prevalence of uncorrected refractive errors and its determinants in tehran in 2002. methods: a total of 6497 citizens representing a cross-section of the population of tehran were sampled using a stratified random cluster sampling strategy. of these, 4565 (70.3%) subjects participated in the study and were transferred to a clinic for an extensive eye examination and in...

| The leave-one-out cross-validation scheme for generalization assessment of neu-ral network models is computationally expensive due to replicated training sessions. Linear unlearning of examples has recently been suggested as an approach to approximative cross-validation. Here we brieey review the linear unlearning scheme, dubbed LULOO, and we illustrate it on a system identiication example. F...

Cross-validation is a tried and tested approach to select the number of components in principal component analysis (PCA), however, its main drawback is its computational cost. In a regression (or in a non parametric regression) setting, criteria such as the general cross-validation one (GCV) provide convenient approximations to leave-one-out crossvalidation. They are based on the relation betwe...