Research questions, objectives and theoretical framework|

Research questions
This paper is based on my - project, HELVYRD (Competence based assessment for learning within the framework of vocational didactics) is part of the Norwegian research project  MECVET (Measuring Competence in Vocational Education and Training) that will  develop a model-based instrument for measuring vocational competence where the original prototype is the German KOMET (see Rauner et al, 2009). The focus of my study is to compare the judgments of raters that assess the test tasks of this instrument with raters that assess the trade examination exams of apprentices in industrial mechanics and electricians What are the criteria they use, and to what extent are their judgment holistic and have vocational validity? In my study I will also include the construction of criteria in the two assessment systems and their institutional position within the Norwegian VET-system compared with the German VET-system.

Theoretical framework

The HELVYRD-project brings together different theoretical frameworks. It is addressing issues that are at the core of assessment theory (Black et al, 2000) for example the referencing of performance against norms, criteria or holistic judgement (Wolf, 2001), methods for constructing open task-assignments with high degree of vocational validity (Haasler & Erpenbeck, 2008; Johnson, 2008; Becker et al., 2010), the institutionalization of model-based assessment systems compared with curriculum-based assessment systems (Baartman et al, 2007). The German KOMET assessement instrument is based on a model from which are derived 8 dimensions or criteria (functionality, presentation, effectiveness, sustainability, business and work process, environmental, creativity, social acceptability) and for each task assignment a solution space is developed by experts in the trade. Thus the raters are trained to adhere to these guidelines whereas raters in the Norwegian trade examinations are assumed to have a focus on official competence goals, trade-specific criteria and the work contexts of the apprentices. Both the construction of these two assessment systems (tasks, rating criteria etc) and the judgmental sources of the raters will be compared since these factors are crucial to the institutionalization of model-based assessment like the KOMET. As a test for large scale assessment it has been trialed in Germany, China, South Africa (Rauner, 2012) and is presently examined in Norway (MECVET-project). In parts of Germany it may be integrated in the VET-system since it accords with recent changes rooted in the Lernfeld-reform (Fischer, 2011). The HELVYRD-project is also relevant for our understanding of competence-based examinations in VET. Although some studies have been done in this field, the research on the judgmental basis of raters is almost non-existant in the international VET-literature (MacLean & Wilson, 2009).

Baartman, L.K.J. et al (2007) Evaluating assessment quality in competence-based education: A qualitative comparison of two frameworks. Educational research review, 2, 114-129. Becker, M et al.. (eds..) (2010): Von der Arbeitsanalyse zur Diagnose beruflicher Kompetenzen:Methoden und methodologische Beiträge aus der Berufsbildungsforschung, Frankfurt am Main: Peter Lang Black, P. (2000). Research and the Development of Educational Assessment. Oxford Review of Education. Vol.26, Nos. 3&4, pp407-419. Fischer, A. (2011) Das Lernfeldkonzept als Forschungsanlass und Diskursthema in der Berufs- und Wirtschaftspädagogik – Leuphana Notizen. bwp@ Spezial 5, 1-16 Haasler, B. & Erpenbeck, J (2008): Assessing vocational competences. In Rauner, F. & MacLean, R. (2008) Handbook of technical and vocational education and training research. Springer Media Johnson, M. (2008) Assessing at the borderline: Judging a vocationally related portfolio holistically. Issues in educational research, 18, 1, 26-44 Maclean, R. & D. Wilson (Eds.) International handbook of education for the changing world of work. Springer Science+Business Media B.V. Rauner, F. (2009a): Messen beruflicherKompetenzen, Band 1: Grundlagen und Konzeption des KOMETProjektes,Berlin LIT Verlag Rauner, Felix et.al.(2012): Project Report Comet-Pilot Test South Africa, i:BB Universität Bremen Wolf, A. (2001) Competence-based assessment. In Raven, J. & J. Stephenson (eds) Competence in the learning society. NY: Peter Lang