Evaluating the Quality of Online-Courses based on the Relational Quality Criteria Catalogue (RQCC)

Author(s):

Stefan IskeNorbert MederStefan Iske

Conference:

ECER 2009

Network:

11. Educational Effectiveness and Quality Assurance

Format:

Paper

Session Information

11 SES 03 A, E-learning and informal learning

Paper Session

Time:

2009-09-28

14:00-15:30

Room:

HG, HS 46

Chair:

José Cajide

Contribution

This paper focuses the general question of how to evaluate the quality of on-line courses. It is based on the research findings of the project “Electronic Learning and Assistance Network” (ELAN) which is funded by the European Commission (eTen-programme). In order to answer the initial question, a relational approach to quality assurance of on-line courses will be presented. Following a rigorous learner-oriented approach, quality is considered as a relation of course AND learner and not as an substantial attribute of a course: So, the learner and his characteristics plays a decisive role in establishing quality and for this reason becomes a co-producer. In this regard, the evaluation of on-line courses alone in the sense of „product quality” – e.g. based on technical checklists – is insufficient. It has to be supplemented by the evaluation of usage and the overall evaluation of the appropriateness of the on-line course in relation to the learner’s demands and requirements. Therefore, on-line course are analyzed as specific environments to facilitate learning and more general as a space of possibilities, which contains four aspects to facilitate learning: - Incentives of learning: Does the on-line course offer an incentive for starting and continuing learning (for example: incentive setting, scenario, stimulation)? - Learning process: Does the on-line course offer a specific process of learning (for example: learning activities or sequence of content)? - Learning support: Does the on-line course offer help in case the learner is not able to master a learning related problem or difficulty by himself (for example: FAQ, Email). - Feedback on learning results: Does the on-line course inform the learner about his learning outcome (for example: quiz, test, certificate)? This paper will present major results of the application of the relational approach to quality assurance of on-line courses and will discuss the pros and cons.

Method

Concerning methodological aspects, the relational approach to quality assurance of on-line courses is implemented as follows: on the one hand, the specific characteristics of the on-line course are analysed from a theory-driven point of view, based on a Relational Quality Criteria Catalogue. This Catalogue falls into two parts: Criteria concerning the prototypical learner of course („typical target audience“) and the description of learning arrangements as a “Space of Possibilities”. on the other hand, the usage of the on-line course by learners is analysed from an empirical point of view (e.g. on-line questionnaires, quantitative and qualitative analysis of forum-postings and emails, log data). This theory-driven and this empirical perspective allows to analyse the reference of on-line course and specific learner and therefore allows to evaluate the quality of on-line courses in a relational way.

Expected Outcomes

As the main outcome the Relational Quality Criteria Catalogue will be presented which was developed within the ELAN project. It differs from “product-oriented” check-lists as it realises a strictly learner-oriented approach in accordance with the concept of e-learning as self-regulated learning. The presented relational approach to Quality Assurance aims at establishing fundamental categories for both the theory-based and empirical-based (learner based) evaluation of e-learning quality and presents an innovative and so far neglected approach in the field of e-learning. The outcomes of the relational approach to quality assurance will be discussed.

References

Ehlers, U. & Pawlowski, J. M. (2006). Handbook on quality and standardisation in elearning. Berlin [u.a.]: Springer. Ehlers, U. (2004). E-Learning services in the crossfire: pedagogy, economy and technology. Bielefeld: Bertelsmann. Ehlers, U. (2004). Qualität im E-Learning aus Lernersicht: Grundlagen, Empirie und Modellkonzeption subjektiver Qualität. Wiesbaden: Verlag für Sozialwissenschaften. Ehlers, U. & Pawlowski, J. M. (2006). Handbook on quality and standardisation in elearning. Berlin [u.a.]: Springer. Harvey, L. & Diana Green, (2000). Qualität definieren. Fünf unterschiedliche Ansätze. In A. Helmke, W. Hornstein & E. Terhard (Ed.), Qualität und Qualitätssicherung im Bildungsbereich: Schule, Sozialpädagogik, Hochschule (Vol.41, pp. 17-39). Weinheim. Klebl, M. (2006). Nachhaltiges Design digitaler Lernmedien: netzgestützte Bildungsprozesse mit IMS Learning Design. Bozen: Studien-Verl. Meder, Norbert, 2004: Der Sprachspieler. Der postmoderne Mensch oder das Bildungsideal im Zeitalter der neuen Technologien. 2. wesentlich erweiterte Auflage. Würzburg. Meder, Norbert/ Johannes Fromme, 2001: Computerspiele und Bildung. Zur theoretischen Einführung. In: Johannes Fromme/ Norbert Meder(Hrsg.): Bildung und Computerspiele. Zum kreativen Umgang mit elektronischen Bildschirmen. Opladen, S. 11-28. Pawlowski, J. M. (2001). Das Essener-Lern-Modell (ELM): Ein Vorgehensmodell zur Entwicklung computerunterstützter Lernumgebungen. Witsch, Monika, 2008: Kultur und Bildung. Ein kulturwissenschaftliche Grundlegung von Bildung im Anschluss an Georg Simmel, Ernst Cassirer und Richard Hönigswald. Würzburg. On-line resources Dublin Core Metadata Initiative (DCMI), IMS Global Learning Consortium / Learning Design specification, WebKollegNRW, European Quality Observatory, Evaluation Criteria for Peer Reviews of MERLOT Learning Ressources, Canadian Recommended E-Learning Guidelines (CanREGs), MECA-ODL (Methodology for the analysis of quality in open distance learning), Leitfaden für die Begutachtung von Fernlehrgängen (BIBB), Staatliche Zentralstelle für Fernunterricht ZFU , Q.E.D. - Qualitätsinitiative E-Learning in Deutschland, DELZert Deutsche E-Learning Zertifizierungsgesellschaft GbR , Qualitätssiegel eLearning - TU Darmstadt , Qualität im E-Learning - eTeaching.org , E-Learning Courseware Certification - E-Learning Courseware Certfication – ASTD, QuiL - Qualitätskriterien für IT-basierte Lernsysteme,

Author Information

Stefan Iske

Universität Duisburg-Essen

Norbert Meder

Universität Duisburg-Essen

Faculty of Bildungswissenschaften

Bielefeld

Stefan Iske

siske

Bildungswissenschaften / Universität Duisburg-Essen