How to Advance Theory with Structural VARs: Use the Sims-Cogley-Nason Approach

The common approach to evaluating a model in the structural VAR literature is to compare the impulse responses from structural VARs run on the data to the theoretical impulse responses from the model. The Sims-Cogley-Nason approach instead compares the structural VARs run on the data to identical structural VARs run on data from the model of the same length as the actual data. Chari, Kehoe, and McGrattan (2006) argue that the inappropriate comparison made by the common approach is the root of the problems in the SVAR literature. In practice, the problems can be solved simply. Switching from the common approach to the Sims-Cogley-Nason approach basically involves changing a few lines of computer code and a few lines of text. This switch will vastly increase the value of the structural VAR literature for economic theory. ∗Forthcoming in the NBER Macroeconomics Annual 2006. This work is a response to the comments of Lawrence Christiano, Martin Eichenbaum, and Robert Vigfusson (forthcoming in the NBER Macroeconomics Annual 2006 ) on the critique of structural VARs with long-run restrictions by V.V. Chari, Patrick Kehoe, and Ellen McGrattan. The author thanks numerous economists including his coauthors, Tim Cogley, Jesus Fernandez-Villaverde, Bob Hall, Chris House, Narayana Kocherlakota, Ricardo Lagos, Monika Piazzesi, Juan Rubio-Ramirez, Tom Sargent, Martin Schneider, and Jim Stock for very helpful comments. The author also thanks the NSF for support and Kathy Rolfe and Joan Gieseke for excellent editorial assistance. Any views expressed here are those of the author and not necessarily those of the Federal Reserve Bank of Minneapolis or the Federal Reserve System. Most of the existing structural VAR literature argues that a useful way of advancing theory is to directly compare impulse responses from structural VARs run on the data to theoretical impulse responses from models. The crux of the Chari, Kehoe, and McGrattan (2006) (henceforth, CKM) critique of this common approach is that it compares the empirical impulse responses from the data to inappropriate objects in the model. We argue that instead of being compared to the theoretical impulse responses, the empirical impulse responses should be compared to impulse responses from identical structural VARs run on data from the model of the same length as the actual data. We refer to this latter approach as the Sims-Cogley-Nason approach since it has been advocated by Sims (1989) and successfully applied by Cogley and Nason (1995). CKM argue that in making the inappropriate comparison, the common approach makes an error mistake relative to the Sims-Cogley-Nason approach. That error makes the common approach prone to various pitfalls, including small-sample bias and lag-truncation bias. For example, the data length may be so short that the researcher is forced to use a short lag length, and the estimated VAR may be a poor approximation to the model’s infinite-order VAR. In contrast, since the Sims-Cogley-Nason approach treats the data from the U.S. economy and the model economy symmetrically, it avoids the potential problems of the common approach. On purely logical grounds, then, the Sims-Cogley-Nason approach seems to dominate the common approach. How well does the common approach do in practice using SVARs based on long-run restrictions on data from a real business cycle model? CKM show that for data of the relevant length, SVARs do miserably: the bias is large and SVARs are unable to distinguish between models of interest–unless technology shocks account for virtually all the fluctuations in output. Christiano, Eichenbaum, and Vigfusson (2006) (henceforth, CEV), perhaps the most prominent defenders of the common approach, seem to agree with CKM on the most important matters of substance. Indeed, since there seems to be no dispute that the Sims-CogleyNason approach dominates the common approach, there should be little disagreement over how future research in this area should be conducted. Likewise, there seems to be no dispute that when shocks other than technology play a sizable role in output fluctuations, SVARs do miserably. The primary point of disagreement between CEV and CKM is thus a relatively minor one about the likely size of the errors in the past literature that uses the common approach. CEV argue that the errors are small because the evidence is overwhelming that in U.S. data, technology shocks account for virtually all the fluctuations in output. CKM point to both 20 years of business cycle research and simple statistics in the data that all lead to the opposite conclusion about technology shocks and, hence, to the opposite conclusion as to the size of the errors of the common approach. CEV also venture beyond the confines of the CKM critique and analyze SVARs with short-run restrictions. They focus on SVARs applied to monetary models which satisfy the same recursive identifying assumptions as their SVARs. CEV argue that the error in this application of the common approach is small, and thus the technique can be used broadly to distinguish promising models from the rest. Here the primary problem with their analysis is that it is subject to the Lucas and Stokey critique (Lucas and Stokey 1987): only a tiny subset of existing monetary models in the literature actually satisfies the recursive identifying assumptions. That subset does not include even, for example, the best-known monetary models of Lucas (1972 and 1990). Yet the technique has been used to reject these and other such models. Clearly, comparing impulse responses from SVARs with a set of identifying assumptions to those from models which do not satisfy those assumptions is problematic. Notice that the Sims-Cogley-Nason approach is immune to the Lucas and Stokey critique. Under this approach, it is entirely coherent to compare impulse responses with a set of identifying assumptions to those from models which do not satisfy these assumptions. Under this approach, the impulse responses are simply statistics with possibly little economic interpretation. Now, those statistics may not be interpretable as being close to the model’s theoretical response, but so what? When Kydland and Prescott (1982) compare variances, covariances, and cross-correlations in the model and the data, it does not matter whether these statistics have some deep economic interpretation. Of course, it is not true that all statistics are equally desirable. What properties lead certain statistics to be more desirable than others? One important property is that the statistics vary across alternative models in such a way that, with samples of the lengths we have, they can be used to point with confidence toward one class of models and away from another. (If no such statistics exist, then the data have little to say about the theories of interest.) A second desirable property is that the statistics depend on key features of theory and not on inessential auxiliary assumptions. An important question for a serious assessment of the SVAR literature is, in what sense are the SVAR statistics more or less desirable than a host of other non—SVAR—related statistics? Regrettably, there seems to be little or no work in the SVAR literature directed to this critical question. To reiterate: The CKM critique does not apply to all SVAR analyses, only those that use the common approach rather than the Sims-Cogley-Nason approach. Switching to that dominant approach would cost little–changing only a few lines of computer code and a few lines of text. By making such a switch, researchers using the SVAR approach can vastly enhance their role in guiding theory. In these comments, I begin by carefully describing the difference between the common