Kernel density estimation is a well known method involving smoothing parameter (the bandwidth) that needs to be tuned by the user. Although this has been widely used, bandwidth selection remains challenging issue in terms of balancing algorithmic performance and statistical relevance. The purpose paper study recently developed method, called Penalized Comparison Overfitting (PCO). We first prov...

AdaBoost rarely suffers from overfitting problems in low noise data cases. However, recent studies with highly noisy patterns have clearly shown that overfitting can occur. A natural strategy to alleviate the problem is to penalize the data distribution skewness in the learning process to prevent several hardest examples from spoiling decision boundaries. In this paper, we pursue such a penalty...

The fitting of high-resolution structures into low-resolution densities obtained from techniques such as electron microscopy or small-angle X-ray scattering can yield powerful new insights. While several algorithms for achieving optimal fits have recently been developed, relatively little effort has been devoted to developing objective measures for judging the quality of the resulting fits, in ...

The issue of discrete probability estimation for samples of small size is addressed in this study. The maximum likelihood method often suffers overfitting when insufficient data is available. Although the Bayesian approach can avoid overfitting by using prior distributions, it still has problems with objective analysis. In response to these drawbacks, a new theoretical framework based on thermo...

Hard margin support vector machines (HM-SVMs) have a risk of getting overfitting in the presence of the noise. Soft margin SVMs deal with this problem by the introduction of the capacity control term and obtain the state of the art performance. However, this disposal leads to a relatively high computational cost. In this paper, an alternative method, greedy stagewise algorithm, named GS-SVMs is...

Latent Dirichlet Allocation (LDA) is a generative model for text documents. It is an unsupervised method which can learn latent topics from documents. We investigate the task of topic modeling of documents using LDA, where the parameters are trained with collapsed Gibbs sampling. Since the training process is unsupervised and the true labels of the training documents are absent, it is hard to m...

The high dimensionality of functional magnetic resonance imaging (fMRI) data presents major challenges to fMRI pattern classification. Directly applying standard classifiers often results in overfitting, which limits the generalizability of the results. In this paper, we propose a new group of classifiers, “Generalized Sparse Classifiers” (GSC), to alleviate this overfitting problem. GSC draws ...

When data exhibit imbalance between a large number d of covariates and a small number n of samples, clinical outcome prediction is impaired by overfitting and prohibitive computation demands. Here we study two simple Bayesian prediction protocols that can be applied to data of any dimension and any number of outcome classes. Calculating Bayesian integrals and optimal hyperparameters analyticall...

Evolutionary Learning Classifier Systems (LCSs) are rule based systems that have been used effectively in concept learning. XCS is a prominent LCS that uses genetic algorithms and reinforcement learning techniques. In traditional machine learning, early stopping have been investigated extensively to an extent that it is now a default mechanism in many systems. There has been a belief that EC me...

When the neural element number n of neural networks is larger than the sample size m, the overfitting problem arises since there are more parameters than actual data (more variable than constraints). In order to overcome the overfitting problem, we propose to reduce the number of neural elements by using compressed projection A which does not need to satisfy the condition of Restricted Isometri...