New smoothing SVM algorithm with tight error bound and efficient reduced techniques

The quadratically convergent algorithms for training SVM with smoothing methods are discussed in this paper. By smoothing the objective function of an SVM formulation, Lee and Mangasarian [Comput Optim Appl 20(1):5-22, 2001] presented one such algorithm called SSVM and proved that the error bound between the new smooth problem and the original one was O(1/p) for large positive smoothing parameter p. We derive a new method by smoothing the optimal optimality conditions of the SVM formulation, and we prove that the error bound is O(1/p), which is better than Lee and Mangasarian’s result. Based on SWM identity and updating Hessian iteratively, an “optimal” reduced technique is also proposed to solve Newton equation with lower computational complexity. Experimental results show that the proposed smoothing method has the same accuracy as SSVM, whose error bound is also tightened to O(1/p) in this paper, and the proposed optimal scheme works faster than RSVM for large-scale problems. The proposed smoothing method could work on any differentiable smoothing function, while SSVM only works on the twice differentiable smoothing functions.