Sparsity-driven weighted ensemble classifier

In this study, a novel weighted ensemble classifier that improves classification accuracy and minimizes the number of classifiers is proposed. Proposed method uses sparsity techniques therefore it is named sparsity-driven weighted ensemble classifier (SDWEC). In SDWEC, ensemble weight finding problem is modeled as a cost function with following terms: (a) a data fidelity term aiming to decrease misclassification rate, (b) a sparsity term aiming to decrease the number of classifiers, and (c) a non-negativity constraint on the weights of the classifiers. As the proposed cost function is non-convex and hard to solve, convex relaxation techniques and novel approximations are employed to obtain a numerically efficient solution. The efficiency of SDWEC is tested on 11 datasets and compared with state-of-the art classifier ensemble methods. The results show that SDWEC provides better or similar accuracy using fewer classifiers and reduces testing time for ensemble.