Hoeffding's Inequality
نویسنده
چکیده
Notice that nR̂n(h) ∼ binom(n,R(h)) and so E [ R̂n(h) ] = R(h). We would like to understand how accurate R̂n(h) is as an estimate of R(h). Thankfully there are many well known concentration inequalities that provide us with quantitative answers to this question. The goal of this lecture is to establish one such bound: Hoeffding’s inequality [2]. This inequality was originally proved in the 1960’s and will imply that
منابع مشابه
Cs229 Supplemental Lecture Notes Hoeffding's Inequality
A basic question in probability, statistics, and machine learning is the following: given a random variable Z with expectation E[Z], how likely is Z to be close to its expectation? And more precisely, how close is it likely to be? With that in mind, these notes give a few tools for computing bounds of the form P(Z ≥ E[Z] + t) and P(Z ≤ E[Z]− t) (1) for t ≥ 0. Our first bound is perhaps the most...
متن کاملNew Splitting Criteria for Decision Trees in Stationary Data Streams.
The most popular tools for stream data mining are based on decision trees. In previous 15 years, all designed methods, headed by the very fast decision tree algorithm, relayed on Hoeffding's inequality and hundreds of researchers followed this scheme. Recently, we have demonstrated that although the Hoeffding decision trees are an effective tool for dealing with stream data, they are a purely h...
متن کاملA Novel Test for Independence Derived from an Exact Distribution of ith Nearest Neighbours
Dependence measures and tests for independence have recently attracted a lot of attention, because they are the cornerstone of algorithms for network inference in probabilistic graphical models. Pearson's product moment correlation coefficient is still by far the most widely used statistic yet it is largely constrained to detecting linear relationships. In this work we provide an exact formula ...
متن کامل. e ON UNBIASED ESTIMATION FOR RANDOMIZED RESPONSE MODELS
For quantitative randomized response models, optimal unbiased estimation of regular functionals of distribution functions is considered. In this context, distribution theory of Hoeffding's U-statistics and von Mises' differentiable statistical functions is extended to randomized response models. Estimation of the basic distributions is also considered.
متن کاملA generalization of Hoeffding’s lemma, and a new class of covariance inequalities
MSC: primary 62H05 60E15 secondary 62M10 a b s t r a c t We generalize Hoeffding's lemma to apply to covariances between functions of several random variables. Our generalization leads to a new class of covariance inequalities involving the Vitali or Hardy–Krause variation. These inequalities are relevant to the study of weakly dependent processes. Published by Elsevier B.V.
متن کامل