Tree based ensemble models regularization by convex optimization
نویسندگان
چکیده
Tree based ensemble methods can be seen as a way to learn a kernel from a sample of input-output pairs. This paper proposes a regularization framework to incorporate non-standard information not used in the kernel learning algorithm, so as to take advantage of incomplete information about output values and/or of some prior information about the problem at hand. To this end a generic convex optimization problem is formulated which is first customized into a manifold regularization approach for semi-supervised learning, then as a way to exploit censored output values, and finally as a generic way to exploit prior information about the problem.
منابع مشابه
Stochastic Particle Gradient Descent for Infinite Ensembles
The superior performance of ensemble methods with infinite models are well known. Most of these methods are based on optimization problems in infinite-dimensional spaces with some regularization, for instance, boosting methods and convex neural networks use L1-regularization with the non-negative constraint. However, due to the difficulty of handling L1-regularization, these problems require ea...
متن کاملA Robust Convex Formulation for Ensemble Clustering
We formulate ensemble clustering as a regularization problem over nuclear norm and cluster-wise group norm, and present an efficient optimization algorithm, which we call Robust Convex Ensemble Clustering (RCEC). A key feature of RCEC allows to remove anomalous cluster assignments obtained from component clustering methods by using the group-norm regularization. Moreover, the proposed method is...
متن کاملEnsemble of M5 Model Tree Based Modelling of Sodium Adsorption Ratio
This work reports the results of four ensemble approaches with the M5 model tree as the base regression model to anticipate Sodium Adsorption Ratio (SAR). Ensemble methods that combine the output of multiple regression models have been found to be more accurate than any of the individual models making up the ensemble. In this study additive boosting, bagging, rotation forest and random subspace...
متن کاملOptimum Shape Design of a Radiant Oven by the Conjugate Gradient Method and a Grid Regularization Approach
This study presents an optimization problem for shape design of a 2-D radiant enclosure with transparent medium and gray-diffuse surfaces. The aim of the design problem is to find the optimum geometry of a radiant enclosure from the knowledge of temperature and heat flux over some parts of boundary surface, namely the design surface. The solution of radiative heat transfer is based on the net r...
متن کاملOn High-order Model Regularization for Constrained Optimization
In two recent papers regularization methods based on Taylor polynomial models for minimization were proposed that only rely on Hölder conditions on the higher order employed derivatives. Grapiglia and Nesterov considered cubic regularization with a sufficient descent condition that uses the current gradient and resembles the classical Armijo’s criterion. Cartis, Gould, and Toint used Taylor mod...
متن کامل