How Robust Standard Errors Expose Methodological Problems They Do Not Fix

“Robust standard errors” are used in a vast array of scholarship to correct standard errors for model misspecification. However, when misspecification is bad enough to make classical and robust standard errors diverge, assuming that it is nevertheless not so bad as to bias everything else requires considerable optimism. And even if the optimism is warranted, we show that settling for a misspecified model will still bias estimators of all but a few quantities of interest. We suggest instead that robust and classical standard error differences be treated like canaries in the coal mine, providing clues about model misspecification and likely biases. At that point, we can use standard model checking diagnostics to find the problem and modern approaches to choosing a better model. With several simulations and real examples, we demonstrate that following these procedures can drastically reduce biases, improve statistical inferences, and change substantive conclusions. ∗Our thanks to Neal Beck, Tim Büthe, Helen Milner, Eric Neumayer, Rich Nielsen, and Brandon Stewart for many helpful comments, and David Zhang for expert research assistance. †Institute for Quantitative Social Science, 1737 Cambridge Street, Harvard University, Cambridge MA 02138; http://GKing.harvard.edu, [email protected], (617) 500-7570. ‡Department of Government, 1737 Cambridge Street, Harvard University, Cambridge MA 02138; http://scholar.harvard.edu/mroberts/home