Understanding Catastrophic Overfitting in Single-step Adversarial Training

Although fast adversarial training has demonstrated both robustness and efficiency, the problem of "catastrophic overfitting" been observed. This is a phenomenon in which, during single-step training, robust accuracy against projected gradient descent (PGD) suddenly decreases to 0% after few epochs, whereas sign method (FGSM) increases 100%. In this paper, we demonstrate that catastrophic overfitting very closely related characteristic which uses only examples with maximum perturbation, not all direction, leads decision boundary distortion highly curved loss surface. Based on observation, propose simple prevents overfitting, but also overrides belief it difficult prevent multi-step attacks training.