Mean Field Theory for Density Estimation Using Support Vector Machines

This paper presents a novel algorithm for density estimation which is based on the support vector machines (SVM) approach and it uses the Mean Field (MF) theory for developing an easy and efficient learning procedure for the SVM. The traditional formulation of the SVM density estimation decomposes the parameters of the problem into a quadratic optimization which can be solved using standard optimization techniques. The proposed algorithm approximates the distribution of the SVM parameters as a Gaussian Process and uses the Mean Field theory to easily estimate these parameters. The new algorithm selects the weights of the mixture of kernels used in the SVM estimate more accurately and faster than traditional quadratic programming-based algorithms. The performance of the proposed algorithm is illustrated using a number of simulated densities. The evaluation shows that the method provides satisfactory results while keeping a reasonable convergence speed.