Online Learning for Structural Kernels

While online learning techniques have existed since Rosenblatt’s introduction of the Perceptron in 1957, there has been a renewed interest lately due to the need for efficient classification algorithms. Additionally, kernel techniques have allowed online learning to be extended to problems whose classes are not linearly separable in their native space. Online algorithms typically result in poorer performance than support vector machines, but they have the advantage of vastly reduced computational resources and in some cases do offer performance comparable to the state of the art. This work provides an implementation of several state of the art online learning algorithms as part of the SVM-Light-TK software package, and provides several evaluations of the same. Novelties include the evaluation of such algorithms in the face of high-dimensional structural kernels, and their application to Question Classification and Semantic Role Labelling Boundary Classification tasks from the NLP domain. Additionally, we present a comparison of these online algorithms with batch mode Support Vector Machines.