TextMatcher: Cross-Attentional Neural Network to Compare Image and Text

We study a multimodal-learning problem where, given an image containing single-line (printed or handwritten) text and candidate transcription, the goal is to assess whether represented in corresponds text. This problem, which we dub matching, primarily motivated by real industrial application scenario of automated cheque processing, whose automatically information bank (e.g., issue date) match data that have been entered customer while depositing teller machine (ATM). The finds more general several other scenarios too, e.g., personal-identity-document processing user-registration procedures. devise machine-learning model specifically designed for text-matching problem. proposed model, termed TextMatcher, compares two inputs applying novel cross-attention mechanism over embedding representations text, it trained end-to-end fashion on desired distribution errors be detected. demonstrate effectiveness TextMatcher automated-cheque-processing use case, where shown generalize well future unseen dates, unlike existing models related problems. further performance different distributions public IAM dataset. Results attest that, compared naïve problems, achieves higher variety configurations.