Optimal Decision Trees for Nonlinear Metrics

Nonlinear metrics, such as the F1-score, Matthews correlation coefficient, and Fowlkes–Mallows index, are often used to evaluate performance of machine learning models, in particular, when facing imbalanced datasets that contain more samples one class than other. Recent optimal decision tree algorithms have shown remarkable progress producing trees with respect linear criteria, accuracy, but unfortunately nonlinear metrics remain a challenge. To address this gap, we propose novel algorithm based on bi-objective optimisation, which treats misclassifications each binary separate objective. We show that, for large lies Pareto frontier. Consequently, obtain by using our method generate set all nondominated trees. best knowledge, is first compute provably metrics. Our approach leads trade-off compared optimising metrics: resulting may be desirable according given metric at expense higher runtimes. Nevertheless, experiments illustrate runtimes reasonable majority tested datasets.