In this paper we present CatBoost, a new open-sourced gradient boosting library that successfully handles categorical features and outperforms existing publicly available implementations of gradient boosting in terms of quality on a set of popular publicly available datasets. The library has a GPU implementation of learning algorithm and a CPU implementation of scoring algorithm, which are sign...

In the regression context, boosting and bagging are techniques to build a committee of regressors that may be superior to a single regressor. We use regression trees as fundamental building blocks in bagging committee machines and boosting committee machines. Performance is analyzed on three non-linear functions and the Boston housing database. In all cases, boosting is at least equivalent, and...

We study Schapire's Boosting Algorithm(SBA) for use in practice. SBA is analyzed in terms of its representation and its search. We show that the SBA representation is a piecewise tiling of the domain and that if the weak learner has low coverage ability, SBA's search may fail to boost or may give a sub-optimal solution. We present a rejection boosting algorithm that trades-oo exploration and ex...