Cached Sufficient Statistics for Efficient Machine Learning with Large Datasets

This paper introduces new algorithms and data st.ruct,ures for quick rounting for machine learning dat.asets. We focus on t,he counting task of constructing contingent:. t.ables, but our approach is also applicahlc t.o counting the number of records in a dataset that match conjunctive queries. Subject to certain assumptionsl t h c rosts of thesr operations ca,n he shown to be independent of the number of rpcords in the data.set a n d loglinear i n llie number of noli-zero entries in the coutingeucy table. We provide a very sparse dat.a st.ructurel the ADtrce. to minimize memory use. We provide analytical worst-case bounds for this structure for sewral inodcls of data distribution. We empirically deino~istrat.e t,hat t,ractably-sized data. st.ruct,tires cau br produced for large real-world datascts by (a) using a sparse tree st.ruclure that n w e r allocates memory for counts of zero! (b j nwer a,llocat.ing memory for rounts t,tia,t, can be deduced from other counts. and ( c ) not. bothering to expand t.he t.rec ful ly near i1.s leaves. 1% show 11ow the ADtree call be used to accelerate Bayes I I C ~ structure liidiiig algorithms, ru le learning algorithms, and feature selection algorithms. and we provide a number of empirical results comparing ADtree met.hods against traditional direct count.ing approaches. We also discuss thc possible uses of AMrees in ot,lier machine lesrniiig methods. and discuss the merits of ADtrees i n cornparisori with altcrnative representations such as M-t.rees, R-trees and frequent sets. 1 Caching sufficient statistics Coinputat.iona1 efficiency is an important concern fur machine leasiiing algorithms. especially when applied t,o large datasets (Fayyatl et al.: 1997: Fa.&yya,d and Uthuriisamy, 1996) or in real-time sceiiarios. In (Moore et al.. 1997) &trees with multiresolution cached regression