A bias correction algorithm for the Gini variable importance measure in classification trees

This paper considers a measure of variable importance frequently used in variable selection methods based on decision trees and tree-based ensemble models, like CART, Random Forests and Gradient Boosting Machine. It is defined as the total heterogeneity reduction produced by a given covariate on the response variable when the sample space is recursively partitioned. Some authors showed that this measure is affected by a bias that, under certain conditions, may have potentially dangerous effects on variable selection. The aim of our work is to present a simple and effective method for bias correction, focusing on the easily generalizable case of the Gini index as a measure of heterogeneity.