Robust Learning for Text Classification with Multi-source Noise Simulation and Hard Example Mining

Many real-world applications involve the use of Optical Character Recognition (OCR) engines to transform handwritten images into transcripts on which downstream Natural Language Processing (NLP) models are applied. In this process, OCR may introduce errors and inputs NLP become noisy. Despite that pre-trained achieve state-of-the-art performance in many benchmarks, we prove they not robust noisy texts generated by real engines. This greatly limits application scenarios. order improve model transcripts, it is natural train labelled texts. However, most cases there only clean Since no pictures corresponding text, impossible directly recognition obtain data. Human resources can be employed copy take pictures, but extremely expensive considering size data for training. Consequently, interested making intrinsically a low resource manner. We propose novel training framework 1) employs simple effective methods simulate noises from 2) iteratively mines hard examples large number simulated samples optimal performance. 3) To make our learn noise-invariant representations, stability loss employed. Experiments three datasets show proposed boosts robustness margin. believe work promote actual scenarios, although algorithm straightforward. codes publicly available (https://github.com/tal-ai/Robust-learning-MSSHEM).