Ed-media 2006

This paper reports the method and results of a comparative usability study conducted on four Learning Management Systems (LMS), two commercial and two Open Source. The results suggest that there is no single most usable LMS, and that there is no significant difference in the usability of commercial vs. Open Source applications for eLearning. Finally, the paper proposes some simple workarounds that teachers and instructional developers can exploit in order to fix some LMS usability problems in student interface. Introduction Learning Management Systems (LMS) are complex web based applications that support online or blended learning activities by providing tools for content delivery, learning assessment, communication services (e.g., dis cussions forums or chat lines), and course management (e.g., editing, back-up, enrollment, etc.) (Lepori, Cantoni & Rezzonico, 2005). The selection and adoption of a LMS by a teaching institution or a corporate training system follows the analysis of some basic parameters, usually including technical features (e.g., programming language used or required hardware infrastructure, etc.), available functions (e.g., discussion forums, integrated streaming services, etc.), supported formats (e.g., HTML, PDF, different video encoding, etc.) and learning technology standards compliance (e.g., SCORM). Such analyses (CVS, n.d. ; FNL, n.d.; EDUTOOLS, n.d.) are mostly system-oriented, i.e., measure a definite set of features independent from the users, and only a very limited number of comparative studies on LMS actually consider other parameters (e.g., Botturi, 2004, which includes usability and Open Source community concerns). As members of the family of web applications, LMSs producers should care after de facto web design standards; in particular, they have make their applications usable, as a distinctive feature of their quality for the user experience (Bolchini, Cantoni & Di Blas, 2003). According to ISO 9241 the degree of usability is based on “the effectiveness, efficiency, and satisfaction with which specified users achieve specified goals in particular environments” (ISO, 1998). Providing web users with a usable environment can lead to significant savings and improved performances (Donahue, et Al., 1999; Nielsen, 2003). In terms of teaching and learning, having a usable LMS means potentially reducing teacher time invested in setting up and managing the course and improving the students’ learning experience – teachers and learners do not need to struggle with difficult technologies but can focus on content. This paper reports the method and results of a comparative usability study conducted in 2004-2005 on four different LMS. The next section proposes some reasons for conducting a structured usability evaluation on LMS. After that we explain the methodology adopted for the study, and then we present and discuss its main results. About LMS usability Why is it useful? Recent developments (Triacca et Al., 2004; Frick et Al., 2005) indicate that web usability is becoming an important issue for eLearning and for LMS development. A recent survey (Pulichino, 2004) shows that eLearning practitioners perceive usability a key factor in eLearning applications planning and use. The results of that survey indicate three aspects: (a) usability is an essential consideration when designing eLearning; (b) eLearning components should always be tested for usability; and (c) eLearning components effectiveness can be greatly enhanced through user-centered design methodologies. From the perspective of LMS selection, adoption and maintenance, the investigating the usability of LMS can be very interesting also for at least three reasons: 1. It may reveal usability breakdowns and provide indications for enhancing the application itself, by creating workarounds or by fixing the code – a possible alternative with Open Source. 2. It allows LMS manager to create guidelines for course authors and instructors that are actually supportive for their practice and focus on their problems instead of being (only) general introductions to the tool. 3. It allows user-oriented instead of system-oriented comparison and assessment of LMSs.