Kernel Mean Estimation via Spectral Filtering: Supplementary Material

This note contains supplementary materials to Kernel Mean Estimation via Spectral Filtering. 1 Proof of Theorem 1 (i) Since μ̌λ = μ̂ λ λ+1 = μ̂P λ+1 , we have ‖μ̌λ − μP‖ = ∥∥∥∥ μ̂P λ+ 1 − μP ∥∥∥∥ ≤ ∥∥∥∥ μ̂P λ+ 1 − μP λ+ 1 ∥∥∥∥+ ∥∥∥∥ μP λ+ 1 − μP ∥∥∥∥ ≤ ‖μ̂P − μP‖+ λ‖μP‖. From [1], we have that ‖μ̂P − μP‖ = OP(n) and therefore the result follows. (ii) Define ∆ := EP‖μ̂P − μP‖ = ∫ k(x,x) dP(x)−‖μP‖ 2 n . Consider EP‖μ̌λ − μP‖ −∆ = EP ∥∥∥∥ n nβ + c (μ̂P − μP)− μP ∥∥∥∥ 2 −∆ = ( n nβ + c )2 ∆+ c (nβ + c)2 ‖μP‖ −∆ = c‖μP‖ − (c + 2cn)∆ (nβ + c)2 . Substituting for ∆ in the r.h.s. of the above equation, we have EP‖μ̌λ − μP‖ −∆ = (nc + c + 2cn)‖μP‖ − (c + 2cn) ∫ k(x, x) dP(x) n(nβ + c)2 . It is easy to verify that EP‖μ̌λ − μP‖ −∆ < 0 if ‖μP‖ ∫ k(x, x) dP(x) < inf n c + 2cn nc2 + c2 + 2cnβ = 2β 21/ββ + c1/β(β − 1)(β−1)/β . Remark. If k(x, y) = 〈x, y〉, then it is easy to check that Pc,β = {P ∈ M +(R) : ‖θ‖ 2 2 trace(Σ) < A 1−A} where θ and Σ represent the mean vector and covariance matrix. Note that this choice of kernel yields a setting similar to classical James-Stein estimation, wherein for all n and all P ∈ Pc,β := {P ∈ Nθ,σ : ‖θ‖ < σ √ dA/(1−A)}, μ̌λ is admissible for any d, where Nθ,σ := {P ∈ M +(R) : dP(x) = (2πσ2)−d/2e ‖x−θ‖2 2σ2 dx, θ ∈ R, σ > 0}. On the other hand, the James-Stein estimator is admissible for only d ≥ 3 but for any P ∈ Nθ,σ.