Evaluating the Quality of Online-Courses based on the Relational Quality Criteria Catalogue (RQCC)

Author(s):

Stefan IskeNorbert MederStefan Iske

Conference:

ECER 2009

Network:

11. Educational Effectiveness and Quality Assurance

Format:

Paper

Session Information

11 SES 03 A, E-learning and informal learning

Paper Session

Time:

2009-09-28

14:00-15:30

Room:

HG, HS 46

Chair:

José Cajide

Contribution

This paper focuses the general question of how to evaluate the quality of on-line courses. It is based on the research findings of the project “Electronic Learning and Assistance Network” (ELAN) which is funded by the European Commission (eTen-programme). In order to answer the initial question, a relational approach to quality assurance of on-line courses will be presented. Following a rigorous learner-oriented approach, quality is considered as a relation of course AND learner and not as an substantial attribute of a course: So, the learner and his characteristics plays a decisive role in establishing quality and for this reason becomes a co-producer. In this regard, the evaluation of on-line courses alone in the sense of „product quality” – e.g. based on technical checklists – is insufficient. It has to be supplemented by the evaluation of usage and the overall evaluation of the appropriateness of the on-line course in relation to the learner’s demands and requirements. Therefore, on-line course are analyzed as specific environments to facilitate learning and more general as a space of possibilities, which contains four aspects to facilitate learning: - Incentives of learning: Does the on-line course offer an incentive for starting and continuing learning (for example: incentive setting, scenario, stimulation)? - Learning process: Does the on-line course offer a specific process of learning (for example: learning activities or sequence of content)? - Learning support: Does the on-line course offer help in case the learner is not able to master a learning related problem or difficulty by himself (for example: FAQ, Email). - Feedback on learning results: Does the on-line course inform the learner about his learning outcome (for example: quiz, test, certificate)? This paper will present major results of the application of the relational approach to quality assurance of on-line courses and will discuss the pros and cons.

Method

Concerning methodological aspects, the relational approach to quality assurance of on-line courses is implemented as follows: on the one hand, the specific characteristics of the on-line course are analysed from a theory-driven point of view, based on a Relational Quality Criteria Catalogue. This Catalogue falls into two parts: Criteria concerning the prototypical learner of course („typical target audience“) and the description of learning arrangements as a “Space of Possibilities”. on the other hand, the usage of the on-line course by learners is analysed from an empirical point of view (e.g. on-line questionnaires, quantitative and qualitative analysis of forum-postings and emails, log data). This theory-driven and this empirical perspective allows to analyse the reference of on-line course and specific learner and therefore allows to evaluate the quality of on-line courses in a relational way.

Expected Outcomes

As the main outcome the Relational Quality Criteria Catalogue will be presented which was developed within the ELAN project. It differs from “product-oriented” check-lists as it realises a strictly learner-oriented approach in accordance with the concept of e-learning as self-regulated learning. The presented relational approach to Quality Assurance aims at establishing fundamental categories for both the theory-based and empirical-based (learner based) evaluation of e-learning quality and presents an innovative and so far neglected approach in the field of e-learning. The outcomes of the relational approach to quality assurance will be discussed.

References

Ehlers, U. & Pawlowski, J. M. (2006). Handbook on quality and standardisation in elearning. Berlin [u.a.]: Springer. Ehlers, U. (2004). E-Learning services in the crossfire: pedagogy, economy and technology. Bielefeld: Bertelsmann. Ehlers, U. (2004). Qualität im E-Learning aus Lernersicht: Grundlagen, Empirie und Modellkonzeption subjektiver Qualität. Wiesbaden: Verlag für Sozialwissenschaften. Ehlers, U. & Pawlowski, J. M. (2006). Handbook on quality and standardisation in elearning. Berlin [u.a.]: Springer. Harvey, L. & Diana Green, (2000). Qualität definieren. Fünf unterschiedliche Ansätze. In A. Helmke, W. Hornstein & E. Terhard (Ed.), Qualität und Qualitätssicherung im Bildungsbereich: Schule, Sozialpädagogik, Hochschule (Vol.41, pp. 17-39). Weinheim. Klebl, M. (2006). Nachhaltiges Design digitaler Lernmedien: netzgestützte Bildungsprozesse mit IMS Learning Design. Bozen: Studien-Verl. Meder, Norbert, 2004: Der Sprachspieler. Der postmoderne Mensch oder das Bildungsideal im Zeitalter der neuen Technologien. 2. wesentlich erweiterte Auflage. Würzburg. Meder, Norbert/ Johannes Fromme, 2001: Computerspiele und Bildung. Zur theoretischen Einführung. In: Johannes Fromme/ Norbert Meder(Hrsg.): Bildung und Computerspiele. Zum kreativen Umgang mit elektronischen Bildschirmen. Opladen, S. 11-28. Pawlowski, J. M. (2001). Das Essener-Lern-Modell (ELM): Ein Vorgehensmodell zur Entwicklung computerunterstützter Lernumgebungen. Witsch, Monika, 2008: Kultur und Bildung. Ein kulturwissenschaftliche Grundlegung von Bildung im Anschluss an Georg Simmel, Ernst Cassirer und Richard Hönigswald. Würzburg. On-line resources Dublin Core Metadata Initiative (DCMI), IMS Global Learning Consortium / Learning Design specification, WebKollegNRW, European Quality Observatory, Evaluation Criteria for Peer Reviews of MERLOT Learning Ressources, Canadian Recommended E-Learning Guidelines (CanREGs), MECA-ODL (Methodology for the analysis of quality in open distance learning), Leitfaden für die Begutachtung von Fernlehrgängen (BIBB), Staatliche Zentralstelle für Fernunterricht ZFU , Q.E.D. - Qualitätsinitiative E-Learning in Deutschland, DELZert Deutsche E-Learning Zertifizierungsgesellschaft GbR , Qualitätssiegel eLearning - TU Darmstadt , Qualität im E-Learning - eTeaching.org , E-Learning Courseware Certification - E-Learning Courseware Certfication – ASTD, QuiL - Qualitätskriterien für IT-basierte Lernsysteme,

Author Information

Stefan Iske

Universität Duisburg-Essen

Norbert Meder

Universität Duisburg-Essen

Faculty of Bildungswissenschaften

Bielefeld

Stefan Iske

siske

Bildungswissenschaften / Universität Duisburg-Essen

Bielefeld

Update Modus of this Database

The current conference programme can be browsed in the conference management system (conftool) and, closer to the conference, in the conference app.
This database will be updated with the conference data after ECER.

Search the ECER Programme

Search for keywords and phrases in "Text Search"
Restrict in which part of the abstracts to search in "Where to search"
Search for authors and in the respective field.
For planning your conference attendance, please use the conference app, which will be issued some weeks before the conference and the conference agenda provided in conftool.
If you are a session chair, best look up your chairing duties in the conference system (Conftool) or the app.