Boosting Descriptive ILP for Predictive Learning

Inductive Logic Programming has been very successful in application to multirelational predictive tasks. Sophisticated predictive ILP systems, such as Progol and foil, can achieve high predictive accuracy, while the learning results remain understandable. Although boosting [1] is an established method to promote predictive accuracy of weak algorithms, there have been relatively few efforts to apply it to ILP systems. Some studies include [2], which applied AdaBoost to the ffoil ILP system, and [3], in which MolFea, a domain-specific ILP system, was used as the base learner for AdaBoost. While these studies showed that predictive accuracy of ILP can often be increased by boosting, there is still much room for improvement. In particular, the run-time performance of ILP systems becomes an issue because AdaBoost has to invoke them many times to produce base classifiers. This prevents boosting from running more iterations to achieve higher predictive accuracy. Also, base classifiers from these ILP systems tend to be fairly accurate, which causes boosting to converge quickly, hence it is liable to overfitting, particularly on noisy datasets. Moreover, boosting needs to apply a weighting over training examples when the base algorithm is invoked. As ILP systems are usually not able to handle weighted examples, resampling is adopted, in which low weighted examples may be lost. To attempt to overcome these weaknesses, we have investigated the use of boosting with descriptive ILP systems, which generate first-order classification rules from training data in a class-blind manner. Using two methods to convert clauses from a descriptive ILP method to classifiers, we present the results of this approach for three bioinformatics datasets.