Image captioning aims to generate a corresponding description of an image. In recent years, neural encoder-decoder models have been the dominant approaches, in which Convolutional Neural Network (CNN) and Long Short Term Memory (LSTM) are used translate image into natural language description. Among these visual attention mechanisms widely enable deeper understanding through fine-grained analys...

Descriptive region features extracted by object detection networks have played an important role in the recent advancements of image captioning. However, they are still criticized for lack contextual information and fine-grained details, which contrast merits traditional grid features. In this paper, we introduce a novel Dual-Level Collaborative Transformer (DLCT) network to realize complementa...

Grid-based features have been proven to be as effective region-based in multi-modal tasks such visual question answering. However, its application image captioning encounters two main issues, namely, noisy and fragmented semantics. In this paper, we propose a novel feature selection scheme, with Relation-Aware Selection (RAS) Fine-grained Semantic Guidance (FSG) learning strategy. Based on the ...

In this paper, we study the local visual modeling with grid features for image captioning, which is critical generating accurate and detailed captions. To achieve target, propose a Locality-Sensitive Transformer Network (LSTNet) two novel designs, namely Attention (LSA) Fusion (LSF). LSA deployed intra-layer interaction in via relationship between each its neighbors. It reduces difficulty of ob...

Video captioning, i.e. , the task of generating captions from video sequences creates a bridge between Natural Language Processing and Computer Vision domains computer science. The semantically accurate description is quite complex. Considering complexity, problem, results obtained in recent research works are praiseworthy. However, there plenty scope for further investigation. This paper addre...