Defining and evaluating quality in e-learning: lessons drawn from case studies in the field of computer-assisted language learning (CALL)

1] Book abtract:The aim of this publication is to discuss the broad question of quality when integrating technology into teaching and learning contexts. The book draws on the experiences of researchers and tutors working in different subject disciplines in order to focus on the commonalities identified when exploiting new technologies within a distinct pedagogical environment. This resource therefore hopes to offer students and teachers an insight into the various applications of technology in teaching and learning.This book can be dissected into a number of areas, including innovative research currently being undertaken at the fore of this technological revolution in order to support integration; the employment of technologies with a link to facilitating communities of membership; the use of specific Reusable Learning Objects designed for both secondary and tertiary education respectively; the use of e-portfolios for students, teachers and information workers; and the critical evaluation of technology.This resource therefore proposes to offer students and teachers an insight into the different applications of technology in teaching and learning. It is hoped that this can be drawn on by undergraduate and postgraduate students; instructional designers; educational managers; teachers; teacher trainers; academics; media technology students with the express intention of illuminating some of the quality issues surrounding the exploitation of technology for teaching and learning purposes.2] Chapter abstract:This chapter draws largely on the thesis completed in 2008, which focused on effectiveness research and the evaluation of computer-assisted language learning (CALL). In this chapter I explore some commonalities between CALL and other disciplines to emphasise the point that a definition of quality for CALL and an evaluative framework for CALL may have relevance for other disciplines. A brief introduction to CALL evaluation and a definition of quality for CALL is given, drawing on the contribution of a number of CALL researchers who have drawn up criteria for a common agenda for evaluating CALL. My thesis pulled these together into a sequenced model for evaluation that addresses issues of internal and external validity, criteria for judging task appropriateness, and principles for determining appropriate quality control criteria. The thesis has appeared in book form (Evaluating Computer-assisted Language Learning: An Integrated Approach to Effectiveness Research for CALL, © Peter Lang AG, 2011) in which the full evaluation model is set out with its theoretical foundations and all the contributing case studies. The starting point for my framework for quality control is Chapelle’s six criteria for evaluating CALL (2001). I looked both within and beyond CALL to widen the evaluative scope to make sure the model addressed not just pedagogy but also software and digital platforms, and their interrelationship, or synergy. To this end I cross-mapped Chapelle with Mehanna (2004) on e-learning pedagogies, with Pederson (1988) and Dunkel (1991) on effectiveness research for CALL, with Ingraham and Emery (1991) and Hubbard (1988) on evaluating software for CALL, and with Clarke (2005) and Barr, Carvalho Martins, Duffner, Gillespie, and Wright (2007) on evaluating digital platforms for CALL. This model was road-tested and developed during four different language teaching case studies carried out at the University of Ulster between 2003 and 2006, culminating in an improved version which proposes a framework for integrated evaluation of CALL platforms, programs and pedagogies and a set of checklists for good practice that are ready-to-use in any CALL context. Some of the checklists are probably usable in other, non-linguistic, contexts with no changes required; the remainder should, with some adaptation, be usable beyond CALL. This chapter concludes with a brief introduction to the improved model for evaluation and some of the quality control checklists.