Re-Introducing BN Into Transformers for Vision Tasks

In recent years, Transformer-based models have exhibited significant advancements over previous in natural language processing and vision tasks. This powerful methodology has also been extended to the 3D point cloud domain, where it can mitigate inherent difficulties posed by irregular disorderly nature of clouds. However, attention mechanism within Transformer presents challenges for utilizing Batch Normalization (BN), as statistical information cannot be extracted efficiently from data set. Thus, this study proposes a novel residual structure, ResBN, which effectively handle data. Additionally, replace BN transformer 2D image processing, we introduce Patch (PN) technique. ResBN PN are evaluated on datasets respectively through experiments, demonstrating their efficacy enhancing classification performance.