Ideal Denoising in an orthonormal basis chosen from a library of bases

Suppose we have observations yi = si+zi, i = 1; :::; n, where (si) is signal and (zi) is i.i.d. Gaussian white noise. Suppose we have available a library L of orthogonal bases, such as the Wavelet Packet bases or the Cosine Packet bases of Coifman and Meyer. We wish to select, adaptively based on the noisy data (yi), a basis in which best to recover the signal (\de-noising"). Let Mn be the total number of distinct vectors occcuring among all bases in the library and let tn = p 2 log(Mn). (For wavelet packets, Mn = n log2(n).) Let y[B] denote the original data y transformed into the Basis B. Choose > 8 and set n = ( (1 + tn)). De ne the entropy functional E (y;B) = X i min(y i [B]; n): Let B̂ be the best orthogonal basis according to this entropy: B̂ = arg minB2LE (y;B): De ne the hard-threshold nonlinearity t(y) = y1fjyj>tg. In the empirical best basis, apply hard-thresholding with threshold t = p n: ŝ i [B̂] = p n(yi[B̂]): Theorem: With probability exceeding n = 1 e=Mn, kŝ sk2 (1 8= ) 1 n min B2LEkŝB sk 2 2: Here the minimum is over all ideal procedures working in all bases of the library, i.e. in basis B, ŝB is just yi[B]1fjsi[B]j>1g. In short, the basis-adaptive estimator achieves a loss within a logarithmic factor of the ideal risk which would be achievable if one had available an oracle which would supply perfect information about the ideal basis in which to de-noise, and also about which coordinates were large or small. The result extends in obvious ways to more general orthogonal basis libraries, basically to any libraries constructed from an at-most polynomially-growing number of coe cient functionals. Parallel results can be developed for closely related entropies.