Evaluating an Intelligent Q&A System for Mobile Cultural Learning

The paper presents a user study designed to evaluate the impact of an intelligent Question and Answering (Q&A) system for mobile cultural learning on the user’s subjective impressions and retention performance. We have chosen a recent pedagogical framework for mobile learning as a theoretical foundation for this work. The framework postulates four types of mobile learning from which we have chosen to implement the second one (i.e., High Transactional Distance and Individualised Mobile Learning (HI))[13]. In particular, our Q&A system enables individual learners to interact with unknown cultural content under simulated mobile conditions in a well-organised and structured manner using natural language. To investigate the impact of the Q&A system on the participants’ retention of cultural content, we compared two variations of the Q&A system in the lab under simulated mobile conditions. The systems differed both in terms of the approach to processing natural language (i.e., scripts vs. linguistic parsing) and style of a Q&A session (free vs. constrained). Below, we present the results of the study and a series of design recommendations that should aid in the development of more robust Q&A models for mobile cultural systems.