Learning as immersive experiences: Using the four-dimensional framework for designing and evaluating immersive learning experiences in a virtual world_1024

Traditional approaches to learning have often focused upon knowledge transfer strategies that have centred on textually-based engagements with learners, and dialogic methods of interaction with tutors. The use of virtual worlds, with text-based, voice-based and a feeling of ‘presence’ naturally is allowing for more complex social interactions and designed learning experiences and role plays, as well as encouraging learner empowerment through increased interactivity. To unpick these complex social interactions and more interactive designed experiences, this paper considers the use of virtual worlds in relation to structured learning activities for college and lifelong learners. This consideration necessarily has implications upon learning theories adopted and practices taken up, with real implications for tutors and learners alike. Alongside this is the notion of learning as an ongoing set of processes mediated via social interactions and experiential learning circumstances within designed virtual and hybrid spaces. This implies the need for new methodologies for evaluating the efficacy, benefits and challenges of learning in these new ways. Towards this aim, this paper proposes an evaluation methodology for supporting the development of specified learning activities in virtual worlds, based upon inductive methods and augmented by the four-dimensional framework reported in a previous study. The study undertaken aimed to test the efficacy of the proposed evaluation methodology and framework, and to evaluate the broader uses of a virtual world for supporting lifelong learners specifically in their educational choices and career decisions. The paper presents the findings of the study and considers that virtual worlds are reorganising significantly how we relate to the British Journal of Educational Technology Vol 41 No 1 2010 69–85 doi:10.1111/j.1467-8535.2009.01024.x © 2009 The Authors. Journal compilation © 2009 Becta. Published by Blackwell Publishing, 9600 Garsington Road, Oxford OX4 2DQ, UK and 350 Main Street, Malden, MA 02148, USA. design and delivery of learning. This is opening up a transition in learning predicated upon the notion of learning design through the lens of ‘immersive learning experiences’ rather than sets of knowledge to be transferred between tutor and learner. The challenges that remain for tutors rest with the design and delivery of these activities and experiences. The approach advocated here builds upon an incremental testing and evaluation of virtual world learning experiences. Background The widespread reporting of Second Life (SL)—a social virtual world—has helped to highlight the more general use of immersive worlds for supporting a variety of human activities and interactions, presenting a wealth of new opportunities and challenges for enriching how we learn (eg, Boulos, Hetherington & Wheeler, 2007; PrasolovaFørland, Sourin & Sourina, 2006), as well as how we work and play. In this way, SL, in common with other virtual world applications, has opened up the potential for users and learners, teachers and trainers, policy makers and decision makers to easily collaborate together in immersive three-dimensional (3D) environments regardless of distance in real time. At the heart of the immersive experiences is the presence of the learner or user as an ‘avatar’ in the virtual space. This avatar represents the embodiment of the user in the virtual space and facilitates a greater sense of control within the immersive environments, allowing users to more readily engage with the experiences as they unfold in real time (Gazzard, 2009). The more general use of virtual environments over the last few years has been facilitated greatly through Web-based technologies and applications, as well as increasing broadband connectivity and computer graphics capabilities. Together, these allow a range of options in the context of education and training, not least sharing documents and files, holding meetings and events, networking and hosting virtual seminars, lectures and conferences, running research experiments, providing forums for sharing research findings and meeting international colleagues (eg, de Freitas, 2008). Such applications also have an even greater potential for integrating different technologies by supporting social software applications (eg, Facebook, Flickr and Wikipedia), presenting e-learning materials and content, and offering learners’ games and rich social interactions. In addition, custom online virtual platforms originating mainly from Universities and research institutes have also been developed mainly for educational and learning purposes (eg, Liarokapis, Petridis, Lister & White, 2002; Liarokapis et al, 2004). These are more experimental prototypes and usually use dedicated hardware devices such as advanced visualisation (head-mounted displays, stereoscopic displays), interaction (3D mouse, orientation and position sensors) as well as haptics (gloves). However, usually the costs involved in these types of configurations are still very high, compared to the alternatives presented above. 70 British Journal of Educational Technology Vol 41 No 1 2010 © 2009 The Authors. Journal compilation © 2009 Becta. This flexibility of usage alongside potential global reach for users has led to a sudden and wide growth in the emergence of virtual world applications: in work preparing this paper, 80 virtual world applications were identified with another 100 planned by the end of 2009 (de Freitas, 2008). While not all of these virtual worlds have applicability for learning, and many are aimed at young children (eg, Club Penguin), the extent of the field, not just in terms of potential use for education and training, but actual usage and uptake by users is extensive. For example, SL, a social open world, currently has 13 million registered accounts (as of March 2008). This paper however is focused upon how virtual worlds can be better understood and used specifically in the context of education and training, and here the use of SL for supporting seminar activities and lectures and other educational purposes has been documented in a number of recent studies and reports (eg, Dickey, 2005; Hut, 2007; Jennings and Collins 2008; for a list of examples of SL use by UK universities, see Kirriemuir 2008). Both the broad emergence and the applicability of immersive spaces for undertaking learning have led to wide interest from learning practitioners in finding out more about how they may be best deployed in the class and seminar room. However, the breadth of applications of virtual worlds, and their relatively swift emergence, has made this a challenging area for researchers and tutors (Hendaoui, Limayem & Thompson, 2008). The area is fragmented due to the nature of its cross-disciplinary appeal and the literature is dispersed around a range of disciplines. Suitably then, this study, undertaken as part of the Joint Information Systems Committee (JISC)-funded MyPlan project (see http://www.lkl.ac.uk/research/myplan), led by the London Knowledge Lab, University of London, set out to explore in a cross-disciplinary way how virtual worlds might be most effectively evaluated in relation to designed learning activities, and whether this evaluation methodology could be used as part of the design process and feedback into an iterative design of activities that could then be replicated by other researchers and learning practitioners. Underpinning this cross-disciplinary approach to the emerging field of serious games and virtual worlds, the authors in previous work have been attempting to reconceptualise ideas around learning, in particular away from more traditional approaches and towards a notion of learning as more centred upon experience and exploration. To understand this we are considering the role of multimodal interfaces (eg, 3D interfaces) and perceptual modelling (cognitive-based approaches), in that our interactions with the environment and our social interactions with others are adopting an approach towards constructing learning experiences as a process of ‘choreography’ rather than based around data recall strategies (de Freitas & Neumann, 2009). This approach reorganises how we produce and develop learning activities, with a greater emphasis upon learner control, greater engagement, learner-generated content and peersupported communities, which jointly may increase learning gains. Work outlining an ‘exploratory learning model’ to support this experience-based and open-ended approach to learning in training contexts is outlined elsewhere (Jarvis & de Freitas, 2009a), and this paper aims to present the outcomes from a study undertaken to Learning as immersive experiences 71 © 2009 The Authors. Journal compilation © 2009 Becta. evaluate the efficacy of using SL as a platform for supporting lifelong learners. In particular, the study was testing the ‘four dimensional framework’ developed in previous studies (de Freitas & Oliver, 2006). Methodology Literature searches have found few other evaluative frameworks for exploring the uses and designs of learning activities in virtual worlds, and these are generally trainingcentred (eg, Fu, Jensen & Hinkelman, 2008). Therefore, this evaluation study adopted an inductive methodology, which requires researchers to construct theories and explanations based upon observations conducted using educational research approaches, including the use of survey data and observations (Gill & Johnson, 1997). A similar approach has been adopted in the Serious Games—Engaging Training Solutions project co-funded by the UK Technology Strategy Board, Selex Systems and TruSim (a division of Blitz Games), but this focused upon measuring the efficacy of game-based learning rather than virtual world learning activities (Jarvis & de Freitas, 2009a). The methodology was selected to address some of the wider issues of efficacy as well as highlighting some of the main issues and challenges arising from this approach to learning and support. In addition to the inductive method, the study combined the use of the ‘four dimensional framework’ to provide a more structured approach to the synthesis and analysis of the research findings. The four-dimensional framework has been proposed in previous studies and papers, (eg, de Freitas & Oliver, 2006). The framework emerged from user studies with tutors and learners around the selection and use of game-based learning. But it has since been used to support the game design and development process (Jarvis & de Freitas, 2009a). In this study, we applied its use for supporting other immersive experiences—in virtual worlds. The framework proposes four dimensions: the learner, the pedagogic models used, the representation used and the context within which learning takes place (see Figure 1). Figure 1: The four-dimensional framework Source: Sara de Freitas, 2008 72 British Journal of Educational Technology Vol 41 No 1 2010 © 2009 The Authors. Journal compilation © 2009 Becta. The first dimension involves a process of profiling and modelling the learner and their requirements. This profile ensures a close match between the learning activities and the required outcomes. The emphasis upon the learner highlights the importance of the interaction between the learner and their environment. For example, more naturalistic interactions may provide less of a gap in learning transfer. Information and communication technology (ICT) capabilities may affect the way that the learner interacts with the experience, and their abilities to become immersed in the activities in the first place. Feedback to the learner is an important aspect of reflection upon learning and may be central to the most effective learning experience—or individual perception of effectiveness (eg, Jarvis & de Freitas, 2009b). The second dimension analyses the pedagogic perspective of the learning activities, and includes a consideration of the kinds of learning and teaching models adopted alongside the methods for supporting the learning processes. This may include the use of associative models based upon task-centred approaches of learning and consistent with training methodology (eg, Gagné, 1965), and constructivist models of learning that involve building upon existing knowledge on the part of the learner (eg, Vygotsky, 1978). ‘Situative’ models of learning involve more socially constructed approaches to learning (eg, Wenger’s model of communities of practice, 1998). Particular selection of learning theories may anticipate the types of learning outcomes that result. For example, it has been observed that immersive experiences based upon task-centred analysis and learning task construction result in task-centred outputs, and although effective may be limited to more training-based contexts for learning. Also, certain forms may reinforce particular approaches more readily. The third dimension outlines the representation itself, how interactive the learning experience needs to be, what levels of fidelity are required and how immersive the experience needs to be. The link between fidelity and learning has been well explored in the work around simulations, but what constitutes interactivity and immersion are relatively under-researched areas and so present challenges for researchers designing experiments. The representational dimension includes the ‘diegesis’ or world of the experience, and may affect levels of engagement and motivation. The final dimension of context may impact upon the place where learning is undertaken, for example, in school or informal contexts; it may also affect the disciplinary context, for example, which subject area is being studied, and whether the learning is conceptual or applied. Context may also include the supporting resources used for learning. The interactions between the learner and their context are particularly important as the learner may be present in a physical and a virtual space at the same time. These hybrid spaces are relatively unexplored in research terms, but may allow for different approaches to learning beyond those outlined here. Each dimension has dependencies upon the others; however, jointly, the four dimensions provide a conceptual framework for exploring immersive learning and, we argue, have implications upon learning design as a whole, particularly when applied to immerLearning as immersive experiences 73 © 2009 The Authors. Journal compilation © 2009 Becta. sive learning environments. In part to test the efficacy of the framework and the methodology outlined, the study aims to explore this framework. For ease of use, the findings of the study are synthesised in relation to these four dimensions. Using SL to support planning lifelong learning The JISC MyPlan project as a whole aimed to develop a personalised system for planning lifelong learning. The component of the study outlined in this paper aimed to explore the possibilities of using a virtual world for supporting lifelong learners in their career decisions and educational choices. In particular, we were interested to find out whether this method could support mentoring and social interactions for learners in a blended virtual context supplemented with face-to-face tutoring. The study was therefore designed as user studies with two defined groups of learners: learners studying at Birkbeck College on the IT Applications programme and learners from Hackney Community College studying on BTEC courses. The data collection methods for the study included preand post-activity surveys, video observations of real world and the in-world sessions, recordings and chat logs. The study was undertaken with ethical considerations and active consent from the participants. The sessions were held in two computer labs at Birkbeck College, University of London (BBK) and Hackney Community College, London (HCC), and in SL. Learner groups from both institutions were selected for the study. The learners from Birkbeck’s IT Applications programme were mature part-time learners all over 18 years of age, and were self-motivated learners. The learners from Hackney Community College were aged between 18 and 24 years of age and were studying for BTEC courses. The two groups offered significant contrast, allowing the researchers to test a range of different responses to the learning activities under exploration. The Learning Day sessions were constructed in order to allow for some degree of structured activities, and some degree of exploration on the part of the learner. The activities functioned as a method for highlighting the main issues arising from this mode of learning, and to aid with producing guidelines for tutors using the tools. Although the intention was that each learner had access to the Internet, some learners’ sessions at HCC were shared since not enough computers were available. User groups consisted of 7 learners at BBK and 14 at HCC. A tutor with experience of SL guided the sessions, which lasted between 2 and 3 hours. At the beginning and end of both sessions individual learners were asked to answer an online survey. Although factors outside our control altered the sessions (see below), they aimed to take the following structure: an introduction to the session, where the tutor introduces the session, explaining the timetable and answering any questions from the learners. This is followed by an introduction to SL, where the tutor takes learners through an induction into SL, including the creation of an avatar, movement around the virtual world, and text chat functions. This is followed by sessions using a blended approach with face-to-face and virtual components. This includes the tutor and learners visiting the Universities & Colleges Admissions Service (UCAS) SL island, a session with a UCAS 74 British Journal of Educational Technology Vol 41 No 1 2010 © 2009 The Authors. Journal compilation © 2009 Becta. advisor in SL, a visit to the Serious Games Institute in SL and a short session with David Burden, an expert in SL who discusses the merits of using SL. See Figure 2, where David Burden takes the participants on a virtual tour. The group then visits the IBM island where they walk around and converse with an expert. To complete the session, the tutor holds a debrief meeting with the group, including a discussion about their experience and completion of survey. Modelling the learner and their learning experiences As outlined above, the learner dimension provides a modelling of the needs and requirements of the learner and learner group, including their ICT capabilities. These cohorts were therefore surveyed. A total of 18 learners answered the pre-activity survey, 7 (38.89%) were BBK learners and 11 (61.11%) were from HCC. The average for selfrated ICT skills (using a scale from 1–5, where 1 = not very good and 5 = excellent) was 3.94, where BBK learners’ skill was rated as 3.57 and HCC learners’ skill was 4.18. The high self-rating for ICT skills, and in particular HCC learners self-rated their ICT skills considerably higher than BBK learners, may be attributed to the difference in age groups or the greater familiarity of younger learners with new technologies. Notably, though our previous user studies had also found a higher estimation of technological capabilities from mature learners (de Freitas, Harrison, Magoulas, Mee, Mohamad & Oliver, 2006). Also the capabilities of the learners in using related games technologies were surveyed. It was found that in the user groups polled, 66.67% of learners do play video games (28.57% from BBK and 90.91% from HCC), of which 70% from HCC play every day. Video games are played once a week by 50% of BBK learners and 10% of HCC learners; two to five times a week by 20% of HCC learners; and once a month by 50% of BBK learners who play video games. The learners (when asked to select one or more options from the survey) who play video games answered that online games were the most Figure 2: Meeting in-world in Second Life for virtual tour Source: David Burden Learning as immersive experiences 75 © 2009 The Authors. Journal compilation © 2009 Becta. popular (40% from HCC and 100% from BBK), followed by PC (30% from HCC) and console (30% from HCC and 50% from BBK). Other forms of video games that are played are mobile games by 20% of HCC learners and virtual games by 10% of learners. HCC learners are heavy gamers, which may explain the fact that only a few (18.18%) had seen or experienced a virtual world. This is at least partly attributable to the comparatively higher numbers of users using multiplayer online games when compared with virtual worlds. Surprisingly perhaps, only 22.22% of the sample had used virtual worlds before. Broken down by institution, 28.57% of BBK learners and 18.18% of HCC learners had used this type of application before. All of the learners who had used virtual worlds previously had chosen SL, and none had used a different platform, such as Olive. All of the learners had used SL only once. A total of 16 learners answered the post-activity survey (two learners from HCC left after the session without having completed the post-activity survey). All seven learners from BBK and nine from HCC completed this survey. When asked how much they had enjoyed the SL session (using a scale from 1–5, where 1 = didn’t enjoy the session and 5 = really enjoyed the session), BBK learners averaged 3.14 while HCC learners averaged 3.22. The survey asked learners about how much they had enjoyed the different aspects of sessions. The findings of the survey, including Likert numbers, are included in Table 1 below. More generally, the survey synthesis found that 43.75% of the sample (42.88% from BBK and 44.44% from HCC) would recommend the use of SL to their friends. However, when asked whether SL sessions helped them to reflect upon their educational choices and career decisions, only 12.5% of the sample answered positively (14.29% from BBK and 11.11% from HCC). Nevertheless, when asked whether they would like to use SL or another virtual world as part of an educational environment for international collaboration with learners globally, the majority of the sample (81.25%) answered affirmatively (100% from BBK and 66.67% from HCC). This indicates that there were problems Table 1: A comparison of how well liked each aspect of the session was by each user group Aspect of session BBK learners HCC learners The face-to-face sessions 3 2.5 Using the SL application 3.14 2.66 Creating avatars 2.2 3.14 Moving in the virtual space 2.42 2.75 The visit to the UCAS island 2.83 3.125 The SGI presentations 3 3.125 The visit to IBM’s island 2.85 3 Meeting the experts 3.14 2.87 Reacting with your fellow learners in-world 3.5 3.14 BBK, Birkbeck College, University of London; HCC, Hackney Community College, London; SL, Second Life; SGI, Serious Games Institute; UCAS, Universities & Colleges Admissions Service. 76 British Journal of Educational Technology Vol 41 No 1 2010 © 2009 The Authors. Journal compilation © 2009 Becta. with the method we used for structuring the learning activities, for example providing more time for feedback and reflection may have been advantageous. Learning models and theories The pedagogic dimension of the study design rested largely upon a posited constructivist model where knowledge construction on the part of the learner was inferred. It was expected that the learners’ experiences would build upon previous experiences, in particular using previous experience of similar formats of learning and previous knowledge of career decisions and educational choices. However, this area of the study design perhaps presupposed too much prior knowledge on the part of the learner, and some learners found it difficult to engage with the virtual world. A more structured pedagogic model and more structured activities in-world may have been more effective and warrants further testing. Existing constructivist theories of learning are being supplemented by new ones, currently being piloted (eg, the exploratory learning model of de Freitas and Neumann 2009) and the use of social virtual worlds such as SL favours social interactions. Therefore, although a more constructivist approach was favoured for the study, the findings seemed to point to greater strengths for supporting social learning. One college learner noted that: ‘[it] brings all people from every aspect of the world together and learn about each other [sic]’. A greater focus upon social interactions and pedagogic models designed to support more socially focused activities may be a better approach for future design. The strengths of the social virtual world need to be better reflected in learning design strategies. The emphasis and strength of the system for supporting social interactions was supported by the comment from one learner around the use of voice capability: ‘we couldn’t use voice on this trial, but I’m sure this would help quite a bit.’ However, in some studies tutors have expressed a preference for using text interactions due to the ease of turn taking when managing groups of learners in-world. Other studies with SL have demonstrated similar findings to this study. In particular, the study undertaken by Dr Diane Carr observing the use of SL with Masters learners at the Institute of Education, UK as outlined on the Learning in Social Worlds project blog (Carr, 2008), demonstrated some similarities, such as problems with using text chat, disorientation and ambiguity, the need to spend time getting used to the interface and the complexity around structuring experiences that are useful for supporting learning. Carr summarises this: A great deal of ‘structuring’ was going on during the sessions—the tutors’ frantically [sic] use of Instant Messenger, for instance, that was not visible to the learners. Also, there were 2 or 3 tutors at each session, taking on different roles in relation to content and class management (Carr, 2008). The study also pointed to strengths of using SL in terms of enhancing social interactions, which is useful for distance and online learners, adding a greater sense of ‘presence’ than traditional virtual learning environments such as Blackboard, with which the use of SL was compared (rather than face-to-face learning). Carr’s study also found Learning as immersive experiences 77 © 2009 The Authors. Journal compilation © 2009 Becta. that some individual learners were unable to adapt to the use of virtual worlds. Our observation was that those unfamiliar with text chat had a particular disconnect from the application: as both the 3D interface and text chat were unfamiliar to them, they felt excluded from the session. Learners’ capabilities with using the interfaces therefore do need to be considered in advance of using the technologies, and additional induction training may be needed in these cases, or alternative learning strategies (eg, Web-based) may be offered. Usability, interactivity and accessibility In the course of this study, the main area of consideration with relevance to the representational dimension focused on the usability of SL. There were clearly issues with the technology, not least because there were significant problems with connectivity and local development work being undertaken at Linden Lab that day that affected the access to the system, and had a negative impact upon the study findings. These technical issues had a clear impact upon the transfer of the learning experience. The comments from learners underlined these technical issues. In the area of usability of the system, learners commented that ‘movement was a bit sluggish, but I suppose that’s more to do with the Internet connection I think.’ One of the college learners noted: ‘make it so it dont [sic] glitch as much and add a few more features to the island’. The connectivity problems were significant and led to some comments from the learners, such as ‘a better Internet connection would have allowed us to have a “fuller” experience. I think that would have made it better’. The issues are significant and tutors aiming to use SL would have to find coping mechanisms for these kinds of problems that occur with limited broadband, accessibility issues and regular maintenance work at Linden Lab. The newness of the technologies and the architectural issues with SL has led a group of open source developers to develop OpenSim (http:// www.opensimulator.com), with the aim of developing a more scalable architecture and allowing the application to be hosted behind institutional firewalls, reducing considerably the technical issues experienced on the day of testing. However, despite these difficulties at least one of the mature learners could see real benefits for those using the application with disabilities: I work with drama/theatre and people with a disability—acquired brain injury—who are on a programme getting them back to work. I think there are some really interesting possibilities in helping to develop confidence among such clients interacting virtually before or as an adjunct to ‘real’ life social interaction and skills development. The representation of the virtual world itself therefore can have a negative impact upon learning, not least because of the level of expectation on the part of the learner. There is evidence that regular gamers find the graphics of virtual worlds too low level, and can experience negative transfer as a result. Learner expectation is a factor for tutors to deal with when using immersive worlds. However, if the activities are well structured and feedback is given by tutors to the learners then there are possibilities for using the tools, 78 British Journal of Educational Technology Vol 41 No 1 2010 © 2009 The Authors. Journal compilation © 2009 Becta. in particular, where social interactions and support may be required. The representation of the virtual world then creates an additional design tool for the tutor: once usability and accessibility issues are addressed, the tutor may explore learning through the interchange between the real and virtual representations—or hybrid spaces (and experiences). In this context, virtual worlds may be used as metaphors of learning or life experiences that can be reflected upon and interacted with in social groups. Real and virtual contexts There were wider contextual issues that affected the efficacy of the learning experiences, and these centred upon a lack of engagement with the virtual worlds due in part to specific learners’ background and age. For example, one or two learners did have problems relating to the format. One mature learner commented, ‘I am afraid that I cannot relate to the virtual world’. Another learner commented that ‘I think anyone new to SL would need someone to show them how to use it, as it is not intuitive to non computer games players.’ The first learner commented that ‘my worry is that it would exclude people who weren’t technologically sophisticated.’ The first learner felt that: ‘I can’t relate to a virtual world and imaginary people; it makes me restless and want to be with real people.’ Interestingly, this learner found it difficult to relate to the fact that avatars were all human-driven, and felt distanced from the real people due to the interface and use of avatars. This was compounded by the fact that the learner was not familiar with the process of text chat and found it alienating for communicating with others. In addition, the study raised particular issues around accessibility and usability, including the quality of broadband connectivity and the user interface design. It is undeniable that using SL behind the institutional firewalls is a difficult and imprecise undertaking, and negative first impressions can be off-putting to the extent that some will not return. As an indication of this, Linden Lab estimate that half of all users never return after their first hour in SL (Lorica, Magoulas & the O’Reilly Radar Team, 2008). However for those that do there are interesting applications that can be investigated (de Freitas, 2008). It is worth considering that while the learners were participating in a study situated at college and university, and as such the context of learning was strictly formal, it would be interesting to gauge the reactions if the study were undertaken in informal learning settings, at home, or in work based settings. Discussion While multiplayer games may have educational potential in the future, virtual worlds are generally regarded as having greater educational potential (de Freitas, 2008). Currently this is broadly because of the focus of activities. However, the method for comparing the benefits of structured activities in games over open-ended explorations of virtual worlds is an area in need of further research. Of interest here may be how to bring together the structured activities of games with the exploration and social power Learning as immersive experiences 79 © 2009 The Authors. Journal compilation © 2009 Becta. of virtual worlds. The motivational capacities of game-play when brought together with the social interactions of virtual worlds may be a powerful teaching combination in the future. The wider trends of technical convergence between games technologies and educational uses is occurring in the shape of serious games and simulations, but while simulations and games for learning are more established approaches, and have more literature to accompany them, the uses of virtual worlds for learning is still a relatively new field, and as this preliminary study has shown there is a significant learning curve when using virtual world applications to support learning, both for tutors and learners. The main impediment lies in the context and familiarity of the form. Indeed, factors such as where the virtual world is used and the past experience of users with the system are significant aspects ensuring or preventing effective use. Additionally, prior experience of gameplay may not be a positive factor, and previous game experience may in fact have a negative impact upon learning with virtual world applications, as gameplayers are used to much higher levels of fidelity and interactivity than are presently available in virtual worlds. With convergence, this is in the process of changing, but as the testing session revealed issues, such as firewalls and graphics, capabilities of hardware can significantly reduce the immersion of the experience and so reduce the effectiveness of the experience. The technical issues did significantly impede the users’ seamless experience and, in contrast with other studies, the least liked aspects of the interaction in SL were creating avatars and moving in-world. This was certainly due to extremely slow connections as a result of maintenance work that day at HCC and due to multiple users on the network at BBK, both of which caused slow download times. In general, the research indicates that control over avatars can be a critical aspect of allowing users to become engaged and motivated through empowerment of controlling their own representation in-world although as Carr, for example, has indicated for some learners this can be off-putting and produce a ‘pain barrier’ to be overcome. From our study, it was clear that the college learners felt more familiar with the process of avatar creation and that this did hold their attention: Figure 3 shows a college learner who had personalised his avatar within a few minutes of using SL, although he had no prior knowledge of SL. Younger learners are adapting to new approaches more readily and concepts such as avatars and customisation of one’s avatar are integrated into their prior knowledge of online gaming. The research team experienced significant challenges with assessing and validating the efficacy of SL for supporting educational choices and career decisions, in terms of the methods of structuring of exercises, providing the best support for the learners and also in terms of technical issues experienced by the users. While some learners were clearly visibly engaged, more work is needed to find out ways of engaging more learners with how to structure the activities, and greater support in advance of trialling is required. More rigorous frameworks and metrics would also be useful for supporting future efficacy studies. The research team would like to undertake further larger and more longitudinal studies towards that end. 80 British Journal of Educational Technology Vol 41 No 1 2010 © 2009 The Authors. Journal compilation © 2009 Becta. Reflecting on these difficulties, only a handful of learners tested (12.5% of learners) expressed that SL helped them to reflect upon their educational choices and career decisions. This indicates that the platform is one in which the format used with users would not be appropriate for mentoring learners. In particular the technical issues such as accessibility and usability were too jarring for the learners, and got in the way of them appreciating the value of the form. Problems with SL such as connection speed, difficulty to move around, orientation, lack of signposts, and not using voice as used in a classroom setting, impeded the study. The HCC did visit the UCAS island, but needed more support with their interactions with the information there. They also thought more signposting on the island would be helpful. They enjoyed visiting the IBM island but also needed more support and guidance in-world. Due to technical issues it was not possible to provide this. However, if the activities were better structured, and the technical issues could be overcome then the format may have potential for mentoring and other socially-driven interactions and learning modes. On the other hand, 81.25% of learners saw positive links between using SL as part of an educational environment for international collaboration with learners globally. This indicates that there are other aspects of SL that may be used in the future for supporting socially-based learning activities designed for lifelong learners. The social dimension of SL is clearly a powerful component of the format, and when the technology becomes more stable, and broadband and sufficient graphics capabilities can be guaranteed within institutions, then it could be used for role play, mentoring and for social skills acquisition. The main lessons arising from this study demonstrated a need to evaluate the platform with a larger sample of learners. While this study is useful for defining some of the Figure 3: Photo of one of the students participating in the study in Second Life Source: Sara de Freitas, 2008 Learning as immersive experiences 81 © 2009 The Authors. Journal compilation © 2009 Becta. evaluation issues, larger numbers of learners would yield a richer dataset and more scope for analysis. In addition, there is a need to consider the design criteria for more structured activities, find ways to better orientate the learners and tutors in advance of the study, and a need to utilise more concerted and experienced technical support and resources. While it was found that the inductive methodology of data collection was effective for providing information about the use of SL (in particular, the combination of chat logs, video footage and surveys was useful for providing a more multidimensional impression of the usage of SL), the use of in-depth semistructured interviews with some of the participants would have been useful for providing a more qualitative dimension for study findings analysis. A follow-up study examining the design, development and use of virtual worlds for tertiary education with lifelong learners would be helpful for validating this evaluation methodology. Moreover, a study using greater numbers of users exploring the patterns of use of modules being taught in SL, in particular with a comparison between face-to-face learner groups, pure distance or online learners and hybrid groups of both would be desirable. The use of immersive learning centrally implies a shift from considering and designing learning tasks to choreographing learning experiences as a whole, mediated by structured and semistructured social interactions. This has implications upon elements of how the learning day as a whole is structured in terms of the different requirements such as duration of sessions, breaks, and necessary facilities and technical support. But it also has implications upon pedagogic considerations, such as learning theories and models applied, the role of the tutor and the context of learning. This shift merits consideration of learning experiences as involving social interactions between members of the learning group, supporting exploratory individual pathways and identification of methods of tutoring that focus more upon mentoring and guiding development. Towards this end, tutors may analyse the learner group and consider their ICT skills levels, game experience and learning approaches. Also, they may consider the pedagogic approaches needed for the subject area taught, learner group and context of learning. Use of the four-dimensional framework can support this process, in terms of the selection of media used and the questions that the tutor needs to ask themselves when structuring and considering the most appropriate ways of integrating immersive learning into their plans. Orientation is important for new users of virtual worlds to induct them into using the platform, and for maximising their engagement with virtual worlds as a whole. As this study has demonstrated, those who are familiar with gaming and who use multiplayer games regularly often find the unstructured and open-ended aspect of virtual worlds difficult to adapt to, as they are used to more structured and purposeful activities, and it can take a long while for them to adapt to these more open and exploratory social worlds. In order to support learners who are novices or regular gameplayers, it would be useful to hold start-up sessions with learners in advance of learning sessions to allow learners to become orientated with the user interface. For example, sessions may be 82 British Journal of Educational Technology Vol 41 No 1 2010 © 2009 The Authors. Journal compilation © 2009 Becta. held where learners log in remotely from home, allowing sufficient time for them to become used to the interface, and minimising the technical issues. In addition to that, orientation sensors (ie, Wiimote) may be used to allow for more tangible orientation in the virtual social worlds. Conclusions This study set out with the intention of testing a virtual world using a predeveloped evaluation methodology and approach. The approach was based upon an assumption that learning experiences need to be designed, used and tested in a multidimensional way due to the multimodal nature of the interface. To support this, the fourdimensional framework was used with the inductive method to gather data and to synthesis and analyse the findings. As a whole, the approach has worked well in this first iteration, its main strength being that the use of the evaluation methodology allowed the research team to evaluate the learning experience according to specific criteria. The presented evaluation methodology may be used as a design tool for designing learning activities in-world as well as for evaluating the efficacy of experiences, due to its set of consistent criteria. The approach does augment the existing methods for evaluation, but needs to be tested with a larger sample and in wider contexts of use to verify its efficacy across different platforms. While the study itself was affected by technical issues that in general were off-putting for those unfamiliar with virtual worlds, still some benefits of using SL for supporting under-served learners, for engaging learners and for supporting distributed groups of learners were highlighted, due to the engaging nature of the form and to its international reach. While it is generally considered that improvements of the SL platform, and the advent of OpenSim and other new-generation virtual worlds will significantly reduce many of the technical issues experienced by the learners, it is also recognised that such tools are still relatively immature and that more work needs to be undertaken to establish their most effective uses, to produce clear guidelines and to exploit their capabilities to the highest degree. Particular strengths of the medium were highlighted; for example, the learners were positive about using the tools for supporting international collaboration, indicating the power of the tool for supporting distributed learning communities based upon shared interests. While the study has not proved conclusively the power of the tool for mentoring, the sessions with the mentor were very effective in practice, and in the future one-to-one sessions with mentors based abroad or not co-located could be further explored. However, more context and advance study is needed to situate the activities in-world and greater time for reflection needs to be provided. Virtual worlds may also support peer collaboration and may be used, for example, for collaborative assignments in-world with practical outputs, for example, designing a marketing campaign in-world, and work centring upon social interactions would be well served in this virtual world. Also, there is real potential for supporting online learning methods by extending the benefits of audio-graphic conferencing to provide a greater sense of presence, thereby potentially reducing non-completion rates. Learning as immersive experiences 83 © 2009 The Authors. Journal compilation © 2009 Becta. The potential for using a social virtual world such as SL for supporting life decisions and educational choices has been established with this study, but thorough testing of sessions, appropriate technical support, use of established and tested pedagogical principles and well-structured sessions are essential for providing enriched experiences that are properly contextualised for the learner. In particular, this immersive learning approach could work well with distance and online learners, distributed user groups or as an additional support for face-to-face learners.The use of virtual worlds may also need to be considered with respect to using a ‘blend’ of other media support mechanisms, such as videoconferencing and virtual learning environments, which may help to support the community-based and social collaborative strengths of immersive environments. ReferencesBoulos, M., Hetherington, L. & Wheeler, S. (2007). Second Life: an overview of the potential of3-D virtual worlds in medical and Health education. Health Information & Libraries Journal, 24,4, 233–245.Carr, D. (2008). Learning to Teach in Second Life. Report for Learning from Online Worlds; Teaching inSecond Life. Institute of Education/Eduserv Foundation, April 2008. Retrieved October 13, 2008,from http://learningfromsocialworlds.wordpress.com/learning-to-teach-in-second-life/Dickey, M. D. (2005). Three-dimensional virtual worlds and distance learning: two case studies ofActive Worlds as a medium for distance education. British Journal of Educational Technology, 36,3, 439–451.de Freitas, S. (2008). Serious virtual worlds: a scoping study. Bristol: Joint Information SystemsCommittee. Retrieved April 27, 2009, from http://www.jisc.ac.uk/publications/publications/seriousvirtualworldsreport.aspxde Freitas, S. & Neumann, T. (2009). The use of ‘exploratory learning’ for supporting immersivelearning in virtual environments. Computers and Education, 52, 2, 343–352.de Freitas, S. & Oliver, M. (2006). How can exploratory learning with games and simulationswithin the curriculum be most effectively evaluated? Computers and Education, 46, 249–264.de Freitas, S., Harrison, I., Magoulas, G., Mee, A., Mohamad, F., Oliver, M. et al (2006). Thedevelopment of a system for supporting the lifelong learner. British Journal of EducationalTechnology. Collaborative e-support for lifelong learning, 37, 6, 867–880.Fu, D., Jensen, R. & Hinkelman, E. (2008). Evaluating game technologies for training. IEEE aerospaceconference. San Mateo, CA: Stottler Henke Assoc., Inc.Gagné, R. M. (1965). The conditions of learning. New York: Holt, Rinehart & Winston.Gazzard, A. (2009). The avatar and the player: understanding the relationship beyond the screen.In G. Rebolledo-Mendez, F. Liarokapis & S. de Freitas (Eds), Proceedings of the IEEE Games andVirtual Worlds for Serious Applications conference, Coventry, UK (pp. 190–193). Proceedings of the1st IEEE International Conference in Games and Virtual Worlds for Serious Applications, IEEEComputer Society, Coventry, UK, 23–24 March.Gill, J. & Johnson, P. (1997). Research methods for managers (2nd ed.). London: Paul ChapmanPublishing.Hendaoui, A., Limayem, M. & Thompson, C. W. (2008). 3D social virtual worlds: research issuesand challenges. IEEE Internet Computing, 12, 1, 88–92.Hut, P. (2007). Virtual laboratories. Progress of Theoretical Physics, 164, 38–53.Jarvis, S. & de Freitas, S. (2009a). Towards a development approach for serious games. In T. M.Connolly, M. Stansfield & E. Boyle (Eds), Games-based learning advancements for multi-sensoryhuman-computer interfaces: techniques and effective practices. Hershey, PA: IGI Global.Jarvis, S. & de Freitas, S. (2009b). Evaluation of an Immersive Learning Programme to supportTriage Training. Proceedings of the 1st IEEE International Conference in Games and Virtual Worldsfor Serious Applications, IEEE Computer Society, Coventry, UK, 23–24 March (pp. 117–122)ISBN: 978-0-7695-3588-3.84 British Journal of Educational TechnologyVol 41 No 1 2010 © 2009 The Authors. Journal compilation © 2009 Becta. Jennings, N. & Collins, C. (2008). Virtual or Virtually U: educational institutions in Second Life.International Journal of Social Sciences, 2, 3, 180–186.Kirriemuir, J. (2008). Measuring the impact of Second Life for educational purposes. Retrieved August4, 2008, from http://www.eduserv.org.uk/foundation/sl/uksnapshot052008Liarokapis, F., Mourkoussis, N., White, M., Darcy, J., Sifniotis, M., Petridis, P. et al (2004). Web3Dand augmented reality to support engineering education. World Transactions on Engineering andTechnology Education, 3, 1, 11–14.Liarokapis, F., Petridis, P., Lister, P. F. & White, M. (2002). Multimedia Augmented Reality Inter-face for E-Learning (MARIE). World Transactions on Engineering and Technology Education, 1, 2,173–176.Lorica, B., Magoulas, R. & the O’Reilly Radar Team (2008). Virtual worlds: a business guide:2008. O’Reilly Radar Report. http://radar.oreilly.com/research/virtual-world-report.html.Publisher name: O’Reilly, Web publication, date accessed 29th October 2009.Prasolova-Førland, E., Sourin, A. & Sourina, O. (2006). Cybercampuses: design issues and futuredirections. Visual Computing, 22, 12, 1015–1028.Vygotsky, L. S. (1978). M. Cole, V. John-Steiner, S. Scribner & E. Souberman (Eds), Mind in society,Cambridge, Massachusetts and London England: Harvard University Press.Wenger, E. (1998). Communities of practice. Cambridge: Cambridge University Press.Learning as immersive experiences 85 © 2009 The Authors. Journal compilation © 2009 Becta. Copyright of British Journal of Educational Technology is the property of Blackwell Publishing Limited and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use.