A Weighted Generalized Least–squares Support Vector Machine

Among Neural Network methods, the Support Vector Machine (SVM) solutions are attracting increasing attention, mostly because they automatically derive the “optimal” network structure, in respect to generalization error for a given problem. In practice it means, that a lot of decisions that had to be made during the design of a traditional NN (e.g. the number of neurons, the length and type of the learning cycle etc.) are eliminated, but the price to pay is SVM high computational complexity. The Least–Squares Support Vector Machine (LS–SVM) is especially important, because it is computationally more effective than the standard SVM methods. In the LS–SVM case training means solving a set of linear equations instead of the long and computationally hard quadratic programming problem that standard SVM involves [1]. While the least–squares version incorporates all training data in the network to produce the result, the traditional SVM selects some of them (the support vectors) that are important in the regression. This sparseness of traditional SVM can also be reached with LS-SVM by applying a pruning method [2], but this requires the entire large problem to be solved at least once. The LS–SVM method should also be able to handle outliers, resulting, e.g. from non–Gaussian noise, therefore another modification of the method, called weighted LS–SVM, is introduced to reduce the effects of this type of noise. The proposed Generalized LS–SVM [5] modifies the formulation of the LS–SVM, which can provide a pruned result. This approach is extended with weighting, which enables us to accomplish both goals.