Compactly Supported Radial Basis Function Kernels

The use of kernels is a key factor in the success of many classification algorithms by allowing nonlinear decision surfaces. Radial basis function (RBF) kernels are commonly used but often associated with dense Gram matrices. We consider a mathematical operator to sparsify any RBF kernel systematically, yielding a kernel with a compact support and sparse Gram matrix. Having many zero elements in Gram matrices can greatly reduce computer storage requirements and the number of floating point operations needed in computation. This paper develops a unified framework to study the efficiency gain and information loss due to the sparsifying operation. In particular, we propose two quantitative measures: similarity and sparsity and study their tradeoff, which is used to adpatively tune the thresholding parameter in the sparsifying operator. We then implement compactly supported RBF kernels to support vector machines (SVM), least squares SVM, and kernel principal component analysis. Simulations demonstrate that properly-tuned compactly supported kernels give favorable performances while enjoying efficient algorithms for computation.