Agnostic Learning and Structural Risk Minimization
نویسنده
چکیده
In this lecture we address some limitations of the analysis of Occam algorithms that limit their applicability. We first discuss the case where the target concept c is not in H which is known as the non-realizable case and as agnostic PAC learning. We then turn to the case where the number of examples m is fixed (i.e., we cannot ask for more examples as in the standard PAC model) and consider how we can handle an infinite size hypothesis class. This is the classical model selection problem that can be solved by structural risk minimization. Although the results are presented for (unions of) finite classes the same arguments translate directly to the infinite case by replacing ln |H| with the VC dimension.
منابع مشابه
Principles of Risk Minimization for Learning Theory
Learning is posed as a problem of function estimation, for which two principles of solution are considered: empirical risk minimization and structural risk minimization. These two principles are applied to two different statements of the function estimation problem: global and local. Systematic improvements in prediction power are illustrated in application to zip-code recognition.
متن کاملRademacher penalties and structural risk minimization
We suggest a penalty function to be used in various problems of structural risk minimization. This penalty is data dependent and is based on the sup-norm of the so called Rademacher process indexed by the underlying class of functions (sets). The standard complexity penalties, used in learning problems and based on the VCdimensions of the classes, are conservative upper bounds (in a probabilist...
متن کاملBoosted ARTMAP: Modifications to fuzzy ARTMAP motivated by boosting theory
In this paper, several modifications to the Fuzzy ARTMAP neural network architecture are proposed for conducting classification in complex, possibly noisy, environments. The goal of these modifications is to improve upon the generalization performance of Fuzzy ART-based neural networks, such as Fuzzy ARTMAP, in these situations. One of the major difficulties of employing Fuzzy ARTMAP on such le...
متن کاملBayesian Learning / Structural
We study links and diierences between structural risk minimization and bayesian learning. We try to nd which a priori is made in each of the following algorithms, showed equivalent to a Bayesian approach: SVMs, backpropagation, RBF, 1-NN. Finally, we propose new versions of Bayes Point Machines .
متن کاملRisk Minimization and Minimum Description for Linear Discriminant Functions
Statistical learning theory provides a formal criterion for learning a concept from examples. This theory addresses directly the tradeoff in empirical fit and generalization. In practice, this leads to the structural risk minimization principle where one minimizes a bound on the overall risk functional. For learning linear discriminant functions, this bound is impacted by the minimum of two ter...
متن کامل