Speeding-up Model Selection for Support Vector Machines

One big difficulty in the practical use of support vector machines is the selection of a suitable kernel function and its appropriate parameter setting for a given application. There is no rule for the selection and people have to estimate the machine’s performance based on a costly multi-trial iteration of training and testing phases. In this paper, we describe a method to reduce the model selection training time for support vector machines. The main idea is, in the process of trying a series of models, the support vectors in previously trained machines are used to initialize the working set in training a new machine. This initialization helps to reduce the number of required optimization loops, thus reducing the training time of the model selection process. Experimental results on real-life datasets show that the training time for each subsequent machine can be reduced effectively in a variety of situations in the model selection process. The method is applicable to different model search strategies and does not affect the model selection result.