Deep learning-based numerical schemes such as Physically Informed Neural Networks (PINNs) have recently emerged an alternative to classical for solving Partial Differential Equations (PDEs). They are very appealing at first sight because implementing vanilla versions of PINNs based on strong residual forms is easy, and neural networks offer high approximation capabilities. However, when the PDE...