Comparing Predictive Accuracy, Twenty Years Later: A Personal Perspective on the Use and Abuse of Diebold-Mariano Tests∗
نویسندگان
چکیده
The Diebold-Mariano (DM) test was intended for comparing forecasts; it has been, and remains, useful in that regard. The DM test was not intended for comparing models. Unfortunately, however, much of the large subsequent literature uses DM -type tests for comparing models, in (pseudo-) out-of-sample environments. In that case, much simpler yet more compelling full-sample model comparison procedures exist; they have been, and should continue to be, widely used. The hunch that (pseudo-) out-of-sample analysis is somehow the “only,” or “best,” or even a “good” way to provide insurance against insample over-fitting in model comparisons proves largely false. On the other hand, (pseudo-) out-of-sample analysis may be useful for learning about comparative historical predictive performance.
منابع مشابه
Discussion of Comparing Predictive Accuracy, Twenty Years Later: A Personal Perspective on the Use and Abuse of Diebold-Mariano Tests
This interesting article provides an insiders view of the early days of the modernforecast comparison literature, which might be dated as beginning with Diebold and Mariano (1995) and West (1996), papers which spawned numerous extensions and re nements over the subsequent years.1 The article also reviews how forecast evaluation methods have been employed in the literature, o¤ering some criti...
متن کاملComparing Predictive Accuracy
We propose and evaluate explicit tests of the null hypothesis of no difference in the accuracy of two competing forecasts. In contrast to previously developed tests, a wide variety of accuracy measures can be used (in particular, the loss function need not be quadratic, and need not even be symmetric), and forecast errors can be non-Gaussian, nonzero mean, serially correlated, and contemporaneo...
متن کاملWorking Paper No. 09-29 Predictive Density Construction and Accuracy Testing with Multiple Possibly Misspecified Diffusion Models
This paper develops tests for comparing the accuracy of predictive densities derived from (possibly misspecified) diffusion models. In particular, we first outline a simple simulation-based framework for constructing predictive densities for one-factor and stochastic volatility models. Then, we construct accuracy assessment tests that are in the spirit of Diebold and Mariano (1995) and White (2...
متن کاملPredictive Density Construction and Accuracy Testing with Multiple Possibly Misspecified Diffusion Models∗
This paper develops tests for comparing the accuracy of predictive densities derived from (possibly misspecified) diffusion models. In particular, we first outline a simple simulation-based framework for constructing predictive densities for one-factor and stochastic volatility models. Then, we construct accuracy assessment tests that are in the spirit of Diebold and Mariano (1995) and White (2...
متن کاملPredictive Density Accuracy Tests for Di usion Processes
This paper makes three contributions. First, we develop a simple simulation based framework for constructing predictive densities for one-factor and stochastic volatility di usion processes (at arbitrary prediction horizons), that is suitable for the case in which the functional form of the conditional distribution is not known. Second, we outline a simulation and bootstrap based methodology th...
متن کامل