Partial and synchronized captioning: A new tool for second language listening development

This study investigates a novel method of captioning, partial and synchronized, as a listening tool for second language (L2) learners. In this method, the term partial and synchronized caption (PSC) pertains to the presence of a selected set of words in a caption where words are synced to their corresponding speech signal, using a state-of-the-art automatic speech recognition (ASR) technology. The system automatically selects words/phrases which are likely to hinder the learner’s listening comprehension and discards the rest. To evaluate the system, the performance of 58 Kyoto University students was assessed by a listening comprehension test on TED talks, under three conditions: no caption, full caption and PSC. Analysis of results revealed that while reducing the textual density of captions to less than 30%, PSC realizes comprehension performance as well as the full caption condition. Moreover, it gains higher scores compared to other conditions for a new segment of the same video without any captions. The findings suggest that PSC can be incorporated into CALL systems as an alternative method to enhance L2 listening comprehension.