Exploiting Multiple Existing Models and Learning Algorithms

This paper presents MMM and MMC, two methods for combining knowledge from a variety of prediction models. Some of these models may have been created by hand while others may be the result of empirical learning over an available set of data. The approach consists of learning a set of \Referees", one for each prediction model, that characterize the situations in which each of the models is able to make correct predictions. In future instances, these referees are rst consulted to select the most appropriate prediction model, and the prediction of the selected model is then returned. Experimental results on the audiology domain show that using referees can help obtain higher accuracies than those obtained by any of the individual prediction models. are viewed from a single perspective. Theory revision systems make use of two sources of knowledge: an existing imperfect model of a domain and a set of available data. Bias selection systems, on the other hand, make use of available data for a domain and several empirical learning algorithms. We propose two methods (MMM and MMC) that take a collection of existing models and learning algorithms, together with a set of available data, and create a combined model that takes advantage of all of these sources of knowledge. addresses the theory revision problem by evaluating the eeectiveness of the existing model (or domain theory) as a predictor using the available data. The data set available for training is divided into two categories: data on which the existing model is correct, and data on which the existing model is incorrect. This divided data set is used to a build a \Referee" predictor (using our default in-ductive method, C4.5 (Quinlan 1993)) that provides a mechanism for deciding in which situations the existing model should be chosen for the prediction of future instances. Another predictor, the \Data" predictor is built using induction on the available data. This pre-dictor is used on future instances where the \Referee" predictor indicates that the existing model is incorrect. This paper reports on MMM/MMC (Multiple MAI Majority/Multiple MAI Conndence), two extensions of the MAI approach to an arbitrary number of models , of which some may be pre-existing models with no ability to learn; some may be constructed by empirical induction from the available data; and still others may even be theory revision methods that make use of both prior knowledge and empirical learning. From now …