On L1-norm multi-class support vector machines: methodology and theory∗

Binary Support Vector Machines have proven to deliver high performance. In multi-class classification, however, issues remain with respect to variable selection. One challenging issue is classification and variable selection in presence of a large number of variables in the magnitude of thousands, which greatly exceeds the size of training sample. This often occurs in genomics classification. To meet the challenge, this article proposes a novel multi-class support vector machine, which performs classification and variable selection simultaneously through an L1-norm penalized sparse representation. The proposed methodology, together with the developed regularization solution path, permits variable selection in such a situation. For the proposed methodology, a statistical learning theory is developed to quantify the generalization error to understand the basic structure of sparse learning, permitting the number of variables greatly exceeding the sample size. The operating characteristics of the methodology are examined via both simulated and benchmark data, and are compared against some competitors in terms of accuracy of prediction. The numerical results suggest that the proposed methodology is highly competitive. ∗ This research was supported in part by National Science Foundation Grant IIS0328802. The authors would like to thank the editor, the associate editor and three anonymous referees for helpful comments and suggestions.