Hateful Memes Detection Based on Multi-Task Learning

With the popularity of posting memes on social platforms, severe negative impact hateful is growing. As existing detection models have lower accuracy than humans, still a challenge to statistical learning and artificial intelligence. This paper proposed multi-task method consisting primary multimodal task two unimodal auxiliary tasks address this issue. We introduced self-supervised generation strategy in generate labels automatically. Meanwhile, we used BERT RESNET as backbone for text image classification, respectively, then fusion them with late method. In training phase, backward guidance technique adaptive weight adjustment were capture consistency variability between different modalities, numerically improving generalization robustness model. The experiment conducted Facebook AI dataset shows that prediction our model outperformed comparing models.