Boosting with Averaged Weight Vectors

AdaBoost !5] is a well-known ensemble learning algorithm that constructs its constituent or base models in sequence. A key step ill AdaBoost is constructing a distribution over the training examples to crette each base model. This distribution, represented as a vector, is constructed to be orthogonal to the vector of mistakes made by tLe previous base model in the sequence [6]. The idea is to make file next base model's errors uncorrelated with those of the previol_s model. Some researchers have pointed out the intuition that it is probably better to construct a distribution that is orthogonal to the mistake vectors of all the previous base models, but that tt_is is not always possible [6]. We present an algorithm that atteml)ts to come as close as possible to this goal in an efficient manner, v_e present experimental results demonstrating significant improveme:lt over AdaBoost and the Totally Corrective boosting algorithm [5], which also attempts to satisfy this goal.