Parallel Boosting with Momentum
نویسندگان
چکیده
We describe a new, simplified, and general analysis of a fusion of Nesterov’s accelerated gradient with parallel coordinate descent. The resulting algorithm, which we call BOOM, for boosting with momentum, enjoys the merits of both techniques. Namely, BOOM retains the momentum and convergence properties of the accelerated gradient method while taking into account the curvature of the objective function. We describe an distributed implementation of BOOM which is suitable for massive high dimensional datasets. We show experimentally that BOOM is especially effective in large scale learning problems with rare yet informative features.
منابع مشابه
Algorithms and hardness results for parallel large margin learning
We consider the problem of learning an unknown large-margin halfspace in the context of parallel computation, giving both positive and negative results. As our main positive result, we give a parallel algorithm for learning a large-margin halfspace, based on an algorithm of Nesterov’s that performs gradient descent with a momentum term. We show that this algorithm can learn an unknown γ-margin ...
متن کاملParallelizing Boosting and Bagging
Bagging and boosting are two general techniques for building predictors based on small samples from a dataset. We show that boosting can be parallelized, and then present performance results for parallelized bagging and boosting using OC1 decision trees and two standard datasets. The main results are that sample sizes limit achievable accuracy, regardless of computational time spent; that paral...
متن کاملComputer extended series and HAM for the solution of non-linear Squeezing flow of Casson fluid between parallel plates
The paper presents analysis of two-dimensional non-Newtonian incompressible viscous flow between parallel plates. The governing problem of momentum equations are reduced to NODE using similarity transformations. The resulting equation is solved using series solution and homotopy analysis method. These methods have advantages over pure numerical methods for obtaining the derived quantities accur...
متن کاملMultiple Boosting: a Combination of Boosting and Bagging
Classiier committee learning approaches have demonstrated great success in increasing the prediction accuracy of classiier learning , which is a key technique for datamining. These approaches generate several classiiers to form a committee by repeated application of a single base learning algorithm. The committee members vote to decide the nal classiication. It has been shown that Boosting and ...
متن کاملMultiple Boosting : A Combination of Boosting
Classiier committee learning approaches have demonstrated great success in increasing the prediction accuracy of classiier learning , which is a key technique for datamining. It has been shown that Boosting and Bagging, as two representative methods of this type, can signiicantly decrease the error rate of decision tree learning. Boosting is generally more accurate than Bagging, but the former ...
متن کامل