The Role of Weight Shrinking in Large Margin Perceptron Learning
نویسندگان
چکیده
We introduce into the classical perceptron algorithm with margin a mechanism that shrinks the current weight vector as a first step of the update. If the shrinking factor is constant the resulting algorithm may be regarded as a margin-error-driven version of NORMA with constant learning rate. In this case we show that the allowed strength of shrinking depends on the value of the maximum margin. We also consider variable shrinking factors for which there is no such dependence. In both cases we obtain new generalizations of the perceptron with margin able to provably attain in a finite number of steps any desirable approximation of the maximal margin hyperplane. The new approximate maximum margin classifiers appear experimentally to be very competitive in 2-norm soft margin tasks involving linear kernels.
منابع مشابه
Flexible Margin Selection for Reranking with Full Pairwise Samples
Perceptron like large margin algorithms are introduced for the experiments with various margin selections. Compared to the previous perceptron reranking algorithms, the new algorithms use full pairwise samples and allow us to search for margins in a larger space. Our experimental results on the data set of (Collins, 2000) show that a perceptron like ordinal regression algorithm with uneven marg...
متن کاملLarge Margin Classifier via Semiparametric Inference
In this paper, we construct a learning method of stochastic perceptron based on semiparametric inference, and show that this method produces large margin solutions. In semiparametric inference, the parameters are divided into structural parameters which are to be estimated and nuisance parameters in which we do not have any interest. Here, the weight vector of perceptron is de ned as structural...
متن کاملConstant Rate Approximate Maximum Margin Algorithms
We present a new class of perceptron-like algorithms with margin in which the “effective” learning rate, defined as the ratio of the learning rate to the length of the weight vector, remains constant. We prove that the new algorithms converge in a finite number of steps and show that there exists a limit of the parameters involved in which convergence leads to classification with maximum margin.
متن کاملPerceptron-like Algorithms and Generalization Bounds for Learning to Rank
Learning to rank is a supervised learning problem where the output space is the space of rankings but the supervision space is the space of relevance scores. We make theoretical contributions to the learning to rank problem both in the online and batch settings. First, we propose a perceptron-like algorithm for learning a ranking function in an online setting. Our algorithm is an extension of t...
متن کاملStatistical Mechanics of On-line Ensemble Teacher Learning through a Novel Perceptron Learning Rule
In ensemble teacher learning, ensemble teachers have only uncertain information about the true teacher, and this information is given by an ensemble consisting of an infinite number of ensemble teachers whose variety is sufficiently rich. In this learning, a student learns from an ensemble teacher that is iteratively selected randomly from a pool of many ensemble teachers. An interesting point ...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
برای دانلود متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید
ثبت ناماگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید
ورودعنوان ژورنال:
- CoRR
دوره abs/1205.4698 شماره
صفحات -
تاریخ انتشار 2012