The Numerical Delta Method and Bootstrap ∗

This paper studies inference on nondifferentiable functions using methods based on numerical differentiation. First we show that for an appropriately chosen sequence of step sizes, numerical derivative based delta methods provide consistent inference for functions of parameters that are directionally differentiable. Examples of directionally differentiable functions arise in a variety of economic applications such as moment inequalities models and threshold regression models. Second, we propose a numerical bootstrap method that provides asymptotically valid inference even for parameters that are not known to be directionally differentiable. The numerical bootstrap can consistently estimate the limiting distribution in many cases where the conventional bootstrap is known to fail and where subsampling has been the most commonly used inference approach. Applications include constrained and unconstrained M-estimators converging at both regular and nonstandard rates, misspecified simulated GMM models with nondifferentiable moments, LASSO, and 1-norm Support Vector Machine regression.