Robust Nonparametric Testing for Causal Inference in Observational Studies

We consider the decision problem of making causal conclusions from observational data. Typically, using standard matched pairs techniques, there is a source of uncertainty that is not usually quantified, namely the uncertainty due to the choice of the experimenter: two different reasonable experimenters can easily have opposite results. In this work we present an alternative to the standard nonparametric hypothesis tests, where our tests are robust to the choice of experimenter. In particular, these tests provide the maximum and minimum P -value associated with the set of reasonable assignments of matched pairs. We create robust versions of the sign test, the Wilcoxon signed rank test, the Kolmogorov-Smirnov test, and the Wilcoxon rank sum test (also called the Mann-Whitney U test). keywords: causal inference, observational studies, nonparametric hypothesis test, matched pairs design, discrete optimization, integer programming.