Pattern Theoretic Learning

The goal of learning from sample data is to extract a concept that captures the underlying pattern while still representing it in a way useful to the investigator. A new approach based on function decomposition in the Pattern Theory framework is presented here. The objective of this extended abstract is three-fold. The first is to provide an overview of our new approach to learning. Specifically, we wish to show the applicability to discovery. Second, we will demonstrate the correlation of decomposed function cardinality (DFC) and "patterned." Finally, we demonstrate the robustness of this approach by exhibiting experimental results on binary functions with C4.5. This new approach to discovery and learning is a powerful method for finding patterns in a robust manner. 1 The Pattern Theory Approach Pattern Theory is a discipline that arose out of machine learning [2] [6] and switching theory [5]. The original goal was to develop formal methods of algorithm design from specifications. The approach is based on a technique called function decomposition and a measure called decomposed function cardinality (DFC). Since Pattern Theory is able to extrapolate available information based on the inherent structure in the data, it is directly related to scientific discovery. Decomposing a function involves breaking it up into smaller subfunctions. These smaller functions are further broken down until all subfunctions will no longer decompose. For a given function, the number of ways to choose two sets of variables (the partition space) exponential. The decomposition space is even larger, since there are several ways the subfunctions can be *Emaih goldmanj ~aa.wpafb.af.mil Figure 1: Lookup Table X¥~ F(X,Y,Z,W) Figure 2: Decomposition combined and there are several levels of subfunctions possible. The complexity measure that we use to determine the relative predictive power of different function decompositions is called DFC. DFC is calculated by adding the cardinalities of each of the subfunctions in the decomposition. The cardinality of an n-variable binary function is 2n. We illustrate the measure in the above figures. In Figure 1, we have a function on four variables with cardinality 24 = 16. In Figure 2, we show the same function after it has been decomposed. The DFC of this representation for the original function is 22 + 22 + 22 = 12. The DFC measures the relative complexity of a function. When we search through the possible decompositions for a function, we choose one with the smallest DFC. This decomposition is our learned concept. The decomposed representation of the function is one that exhibits more information than the alternative. For example, Figure 1 is essentially a lookup table of inputs and outputs. Figure 2, on the other hand, is a function that is not simply a table. The decomposition, for example, could be two simple functions combined together. Throughout the paper when we refer to a minimal function decomposition, we use "minimal" to mean a decomposition such that the DFC is the smallest possible for the entire set of decompositions. It is noted that a given minimal decomposition is not unique. For a more rigorous explanation of the inner workings of 125 From: AAAI Technical Report SS-95-03. Compilation copyright © 1995, AAAI (www.aaai.org). All rights reserved.