In a broad range of classification and decision-making problems, one is given the advice or predictions of several classifiers, of unknown reliability, over multiple questions or queries. This scenario is different from the standard supervised setting, where each classifier's accuracy can be assessed using available labeled data, and raises two questions: Given only the predictions of several c...

Arabic handwriting word recognition is a challenging problem due to Arabic’s connected letter forms, consonantal diacritics and rich morphology. One way to improve the recognition rates classification task is to improve the accuracy of individual classifiers; another, is to apply ensemble of classifiers methods. To select the best classifier set from a pool of classifiers, the classifier divers...

If classifiers are selected from a hypothesis class to form an ensemble, bounds on average error rate over the selected classifiers include a component for selectivity, which grows as the fraction of hypothesis classifiers selected for the ensemble shrinks, and a component for variety, which grows with the size of the hypothesis class or in-sample data set. We show that the component for select...

Building a classification model from thousands of available predictor variables with a relatively small sample size presents challenges for most traditional classification algorithms. When the number of samples is much smaller than the number of predictors, there can be a multiplicity of good classification models. An ensemble classifier combines multiple single classifiers to improve classific...

For N-, O-, and C-linked glycosylation, we trained ensembles of Support Vector Machine (SVM) classifiers using evolutionary information to predict whether or not a site in a protein sequence is a glycosylation site. An ensemble of SVMs is simply a collection of SVM classifiers, each trained on a balanced subsample of the training data. The prediction of the ensemble is computed from the predict...

In this paper, a novel approach to classifier ensemble creation is presented. While other ensemble creation techniques are based on careful selection of existing classifiers or preprocessing of the data, the presented approach automatically creates an optimal labelling for a number of classifiers, which are then assigned to the original data instances and fed to classifiers. The approach has be...

When making predictions with a voting rule, a basic question arises: “What is the smallest number of votes needed to make a good prediction?” In the context of ensemble classifiers, such as Random Forests or Bagging, this question represents a tradeoff between computational cost and statistical performance. Namely, by paying a larger computational price for more classifiers, the prediction erro...

In this paper we propose an approach for ensemble construction based on the use of supervised projections, both linear and non-linear, to achieve both accuracy and diversity of individual classifiers. The proposed approach uses the philosophy of boosting, putting more effort on difficult instances, but instead of learning the classifier on a biased distribution of the training set, it uses misc...

In this paper, we report an experimental comparison between two widely used combination methods, i.e. sum and product rules, in order to determine the relationship between their performance and classifier diversity. We focus on the behaviour of the considered combination rules for ensembles of classifiers with different performance and level of correlation. To this end, a simulation method is p...