Bagging Is a Small-Data-Set Phenomenon

Bagging forms a committee of classijiers by bootstrap aggregation of training sets from a pool of training data. A simple alternative to bagging is to partition the data into disjoint subsets. Experiments on various datasets show that, given the same size partitions and bags, disjoint partitions result in betterperformance than bootstrap aggregates (bags). Many applications (e.g., protein structure prediction) involve use of datasets that are too large to handle in the memory of the typical computer: Our results indicate that, in such applications, the simple approach of creating a committee of classijiers from disjoint partitions is to be preferred over the more complex approach of bagging.