Weak convergence of the regularization path in penalized M-estimation

We consider an estimator b βn(t) defined as the element φ ∈ Φ minimizing a contrast process Λn(φ, t) for each t. We give some general results for deriving the weak convergence of √ n(b βn − β) in the space of bounded functions, where, for each t, β(t) is the φ ∈ Φ minimizing the limit of Λn(φ, t) as n → ∞. These results are applied in the context of penalized M-estimation, that is, when Λn(φ, t) = Mn(φ)+ tJn(φ), where Mn is a usual contrast process and Jn a penalty such as the l norm or the squared l norm. The function b βn is then called a regularization path. For instance we show that the central limit theorem established for the lasso estimator in Knight and Fu [2000] continues to hold in a functional sense for the regularization path. Other examples include various possible contrast processes for Mn such as those considered in Pollard [1985].