Hierarchical ConViT with Attention-Based Relational Reasoner for Visual Analogical Reasoning

Raven’s Progressive Matrices (RPMs) have been widely used to evaluate the visual reasoning ability of humans. To tackle challenges perception and logic on RPMs, we propose a Hierarchical ConViT with Attention-based Relational Reasoner (HCV-ARR). Traditional solution methods often apply relatively shallow convolution networks visually perceive shape patterns in RPM images, which may not fully model long-range dependencies complex pattern combinations RPMs. The proposed consists convolutional block capture low-level attributes patterns, transformer high-level image semantics such as formations. Furthermore, hierarchical captures features from multiple receptive fields, where layers focus fine details while deeper semantics. better underlying rules embedded an (ARR) is establish relations among images. ARR well exploits hidden question images through developed element-wise attentive reasoner. Experimental results three datasets demonstrate that HCV-ARR achieves significant performance gain compared state-of-the-art models. source code available at: https://github.com/wentaoheunnc/HCV-ARR.