The first term is the estimation error and measures the performance of the discrimination rule with respect to the best hypothesis in H. In previous lectures, we studied performance guarantees for this quantity when ĥn is ERM. Now, we also consider the approximation error, which captures how well a class of hypotheses {Hk} approximates the Bayes decision boundary. For example, consider the hist...

In this paper, we present a proof in ACL2(r) of Taylor’s formula with remainder. This important theorem allows a function f with n + 1 derivatives on the interval [a, b] to be approximated with a Taylor series of n terms centered at a. Moreover, the formula allows the error in the approximation to be bounded by a term involving the (n + 1)st derivative of f on (a, b). The results in this paper ...

This paper is concerned with the performance of Orthogonal Matching Pursuit (OMP) algorithms applied to a dictionary D in a Hilbert space H. Given an element f ∈ H, OMP generates a sequence of approximations fn, n = 1, 2, . . ., each of which is a linear combination of n dictionary elements chosen by a greedy criterion. It is studied whether the approximations fn are in some sense comparable to...