Toshiba BRIDJE at NTCIR-5 CLIR: Evaluation using Geometric Means

Toshiba participated in the Monolingual and Bilingual IR tasks at NTCIR-5 CLIR using the BRIDJE system. We submitted 24 runs covering three topic languages (Japanese, English and Chinese) and two document languages (Japanese and English), and achieved the highest performances in the E-J-T, EJ-D, C-J-T, C-J-D, J-E-T and J-E-D subtasks. This paper (re-)examines Partial Disambiguation and the Pivot Language approach for Bilingual IR, Selective Sampling with memory Resetting for Mono/Bilingual IR and a new Monolingual IR strategy called Bounceand-Throw, using the Geometric Mean versions of Average Precision and Q-measure in addition to the standard Arithmetic Mean ones. Our analyses show that the Geometric Mean, which focusses on the “harder” topics, provides new insight into retrieval effectiveness evaluation.