An Ensemble of Fine-Tuned Heterogeneous Bayesian Classifiers

Bayesian network (BN) classifiers use different structures and different training parameters which leads to diversity in classification decisions. This work empirically shows that building an ensemble of several fine-tuned BN classifiers increases the overall classification accuracy. The accuracy of the constituent classifiers can be achieved by fine-tuning each classifier and the diversity is achieved using different BN classifiers. The proposed ensemble combines a Naive Bayes (NB) classifier, five different models of Tree Augmented Naive Bayes (TAN), and four different model of Bayesian Augmented Naive Bayes (BAN). This work also proposes a new Distance-based Diversity Measure (DDM) and uses it to analyze the diversity of the ensembles. The ensemble of fine-tuned classifier achieves better average classification accuracy than any of its constituent classifiers or the ensemble of un-tuned classifiers. Moreover, the empirical experiments present better significant results for many data sets. Keywords—Ensemble classifier; Bayesian Network (BN) classifiers; Fine-tuned BN classifiers; Stacking; Diversity