Statistical Variable Interaction : FocusingMultiobjective Optimization in Machine

The statistical notion of variable interaction is relevant to learning diiculty. This is substantiated by an empirical study that examines the relationship between concept variation and predictive accuracy, extended by analizing the variable interaction as a cause of concept variation. One of six learning systems empirically studied, MRP, exhibits a distinctive advantage ascribed to characteristics of its design , which focuses on the problem of variable interaction, and incorporates techniques appropriate for reducing this problem's innuence on learning. Viewed as an optimization task, this system's learning utilizes a multiobjective function diier-ent from the typical minimization of both classiication error/cost and hypothesis complexity. Whereas complexity is often judged by the size of hypotheses representations that use only attributes given with the data, the new optimization function incorporates a broader sense of complexity. Seemingly complex hypotheses may appear simple after a sensible change of the representation used to describe the examples. MRP's exible representation reduces the complexity penalty associated with relations among attributes, and consequently diminishes the eeect of variable interaction in learning. Incorporating such reduced penalty in the optimization criterion becomes essential for learning in diicult domains, where lack of expert knowledge imposes primitive representations. 1 Motivation It is important to discover the fundamental conditions that obstruct learning (Rendell and Cho, 1990). Most concepts are not learnable because they are random but unlike them, real-world concepts do possess structure, which can be induced from data. In addition, concepts suitable for learning have diierent degrees of structural complexity and hence, diierent degrees of learning diiculty. Empirical studies of learning accuracy often nd no signiicant difference among systems as a result of using data customized for learning (Holte, 1993; Shavlik et al., 1991; Weiss and Ioannis, 1989). These data are described using attributes carefully selected and/or constructed by domain experts. Thus, the given attributes are converted into an important source of knowledge, which simpliies learning greatly and blurs system diierences. However, in poorly understood domains, lack of domain knowledge prevents the use of highly informative attributes. Then, even with suucient