A Large-Scale Gaussian Belief Propagation Solver for Kernel Ridge Regression
نویسندگان
چکیده
We introduce an efficient parallel implementation of a Kernel Ridge Regression solver, based on the Gaussian Belief Propagation algorithm (GaBP). Our approach can be easily used in Peer-to-Peer and grid environments, where there is no central authority that allocates work. Empirically, our solver has high accuracy in solving classification problems. We have tested our distributed implementation on large scale datasets using up to 1,024 CPUs of an IBM Blue Gene supercomputer. 1. KERNEL RIDGE REGRESSION Kernel Ridge Regression (KRR) implements a regularized form of the least squares method useful for both regression and classification. The non-linear version of KRR is similar to the SupportVector Machine (SVM) problem. However, in the latter, special emphasis is given to points close to the decision boundary, which is not provided by the cost function used by KRR.
منابع مشابه
A Gaussian Belief Propagation Solver for Large Scale Support Vector Machines
Support vector machines (SVMs) are an extremely successful type of classification and regression algorithms. Building an SVM entails solving a constrained convex quadratic programming problem, which is quadratic in the number of training samples. We introduce an efficient parallel implementation of an support vector regression solver, based on the Gaussian Belief Propagation algorithm (GaBP). I...
متن کاملRandom walk kernels and learning curves for Gaussian process regression on random graphs
We consider learning on graphs, guided by kernels that encode similarity between vertices. Our focus is on random walk kernels, the analogues of squared exponential kernels in Euclidean spaces. We show that on large, locally treelike, graphs these have some counter-intuitive properties, specifically in the limit of large kernel lengthscales. We consider using these kernels as covariance matrice...
متن کاملN ov 2 00 8 Gaussian Belief Propagation : Theory and Application
The canonical problem of solving a system of linear equations arises in numerous contexts in information theory, communication theory, and related fields. In this contribution, we develop a solution based upon Gaussian belief propagation (GaBP) that does not involve direct matrix inversion. The iterative nature of our approach allows for a distributed message-passing implementation of the solut...
متن کاملGaussian Belief Propagation: Theory and Application
The canonical problem of solving a system of linear equations arises in numerous contexts in information theory, communication theory, and related fields. In this contribution, we develop a solution based upon Gaussian belief propagation (GaBP) that does not involve direct matrix inversion. The iterative nature of our approach allows for a distributed message-passing implementation of the solut...
متن کاملEfficient Approximation for Large-Scale Kernel Clustering Analysis
Kernel k-means is useful for performing clustering on nonlinearly separable data. The kernel k-means is hard to scale to large data due to the quadratic complexity. In this paper, we propose an approach which utilizes the low-dimensional feature approximation of the Gaussian kernel function to capitalize a fast linear k-means solver to perform the nonlinear kernel k-means. This approach takes a...
متن کامل