Logical Satisfiability of Counterfactuals for Faithful Explanations in NLI

Evaluating an explanation's faithfulness is desired for many reasons such as trust, interpretability and diagnosing the sources of model's errors. In this work, which focuses on NLI task, we introduce methodology Faithfulness-through-Counterfactuals, first generates a counterfactual hypothesis based logical predicates expressed in explanation, then evaluates if prediction consistent with that logic (i.e. new formula \textit{logically satisfiable}). contrast to existing approaches, does not require any explanations training separate verification model. We validate efficacy automatic generation, leveraging few-shot priming paradigm. Next, show our proposed metric distinguishes between human-model agreement disagreement input. addition, conduct sensitivity analysis sensitive unfaithful explanations.