Option Decision Trees with Majority Votes

We describe an experimental study of Option Decision Trees with majority votes. Option Decision Trees generalize regular decision trees by allowing option nodes in addition to decision nodes; such nodes allow for several possible tests to be conducted instead of the commonly used single test. Our goal was to explore when option nodes are most useful and to control the growth of the trees so that additional complexity of little utility is limited. Option Decision Trees can reduce the error of decision trees on real-world problems by combining multiple options, with the motivation similar to that of voting algorithms that learn multiple models and combine the predictions. However, unlike voting algorithms, an Option Decision Tree provides a single structured classiier (one decision tree), which can be interpreted more easily by humans. Our results show that for the tested problems, we can achieve signiicant reduction in error rates for trees restricted to two levels of option nodes at the top. When very large Option Decision Trees are built, Option Decision Trees outperform Bagging in reducing error, although the trees are much larger and cannot be reasonably interpreted by humans.