Multiplicative Updates for Large Margin Classifiers
نویسندگان
چکیده
Various problems in nonnegative quadratic programming arise in the training of large margin classifiers. We derive multiplicative updates for these problems that converge monotonically to the desired solutions for hard and soft margin classifiers. The updates differ strikingly in form from other multiplicative updates used in machine learning. In this paper, we provide complete proofs of convergence for these updates and extend previous work to incorporate sum and box constraints in addition to nonnegativity.
منابع مشابه
Statistical signal processing with nonnegativity constraints
Nonnegativity constraints arise frequently in statistical learning and pattern recognition. Multiplicative updates provide natural solutions to optimizations involving these constraints. One well known set of multiplicative updates is given by the ExpectationMaximization algorithm for hidden Markov models, as used in automatic speech recognition. Recently, we have derived similar algorithms for...
متن کاملMultiplicative updates For Non-Negative Kernel SVM
We present multiplicative updates for solving hard and soft margin support vector machines (SVM) with non-negative kernels. They follow as a natural extension of the updates for non-negative matrix factorization. No additional parameter setting, such as choosing learning, rate is required. Experiments demonstrate rapid convergence to good classifiers. We analyze the rates of asymptotic converge...
متن کاملEnsembles of Partially Trained SVMs with Multiplicative Updates
The training of support vector machines (SVM) involves a quadratic programming problem, which is often optimized by a complicated numerical solver. In this paper, we propose a much simpler approach based on multiplicative updates. This idea was first explored in [Cristianini et al., 1999], but its convergence is sensitive to a learning rate that has to be fixed manually. Moreover, the update ru...
متن کاملEfficient Multiplicative Updates for Support Vector Machines
The dual formulation of the support vector machine (SVM) objective function is an instance of a nonnegative quadratic programming problem. We reformulate the SVM objective function as a matrix factorization problem which establishes a connection with the regularized nonnegative matrix factorization (NMF) problem. This allows us to derive a novel multiplicative algorithm for solving hard and sof...
متن کاملNonnegative Matrix Factorization for Semi-supervised Dimensionality Reduction
We show how to incorporate information from labeled examples into nonnegative matrix factorization (NMF), a popular unsupervised learning algorithm for dimensionality reduction. In addition to mapping the data into a space of lower dimensionality, our approach aims to preserve the nonnegative components of the data that are important for classification. We identify these components from the sup...
متن کامل