Inspired by large scale assessment like PISA, TIMMS and PIAAC, German scholars made a feasibility study of a “PISA-VET” or VET-LSA (large scale assessment, Baethge, 2006) piloted in several European countries covering four different trades: electricians, car-mechatronics, social and health care, business and administration. One of the objectives of this initiative was to strengthen the content-specific descriptions of the European Qualification Frameworks’ occupational profiles as common measurement dimensions comparing profiles and learning outcomes at the end of VET (Baethge, 2010).  It came to an end in 2009, and different evaluation reports have highlighted the challenges of constructing a large scale assessment instrument for international comparisons of vocational competence at the end of different VET-structures (Olsen, 2009; Baethge & Arends, 2009). 

The authors of this proposal participated both in the Norwegian piloting of VET-LSA and are engaged in subsequent activities by conducting the MECVET-project (Measuring Competence Development in Vocational Education and Training). In a Norwegian context, MECVET replicates and validates another instrument for large scale assessment within the framework of the German KOMET-project (coordinated by IBB, University of Bremen; Rauner et al, 2009).  This initiative toned down the (ex ante) comparative ambitions of VET-LSA and gave higher priority to the construction of a diagnostic tool that would support competence development at different educational levels (Rauner et al, 2009). Given the later objective, one may distinguish between competence assessment models that underlie large scale comparisons and those that have a local validity (Leigh et al., 2007).  These two alternatives vary on a number of parameters – from designs that typically have focus on structural aspects at a national level supporting decisions about input factors that contribute to intended outcomes guided by principles of comparability, independence of tasks, large samples and psychometric approaches.  This contrasts with the local assessment models in schools or companies where process improvement and learning are fore-fronted, widening the scope for self-assessment and judgmental variation in the factors to be included and the composition of tasks.  As a tentative “middle ground” we have outlined a sectoral assessment approach (Blings & Spöttl, 2008) where “sectoral” is less defined in terms of geographical criteria than in terms of vocational/trade specificities.  It relies on substantial models of competence development in a cluster of trades (for example health and social services) that may be characterized by specific work process structures (Fischer & Rauner, 2002). Decisions about contextual factors need to take into account comparability and important contingencies. It also incorporates the notion of a “Beruf” that expands the domain to be assessed in relation to the performance of specific tasks (Fischer et al., 2015; Shavelson, 2010). The model is a mid-stage summary of our validation of the KOMET-model in Norwegian VET and serves as a conceptual template for further explorations. Methodologically it also represents a “break down” (a moment for meta-reflection, Winograd & Flores, 1986) of our large scale assessment design that encountered unanticipated challenges necessitating a multi-case qualitative strategy to research (Stake, 2005).  The research question was about the feasibility of a large scale assessment design for Norwegian VET.

Baethge, M. (2010): Ein europäisches Berufsbildungs-PISA als politisches und methodisches Projekt. Im Münk, D. & Schelten, A. (Eds) Kompetenzermittlung für die Berufsbildung. Verfahren, Probleme und Perspektiven im nationalen, europäischen und internationalen Raum. Bielefeld : Bertelsmann BIBB. S. 19-36 Baethge, M. et al (2006): PISA-VET. A feasibility study. Stuttgart: Franz Steiner. Baethge, M. & Arends, L. (2009): Feasability Study VET-LSA. A comparative analysis of occupational profiles and VET programmes in 8 European countries. Vocational Training Research volume 8, Bielefeld: Bertelsmann Verlag Blings, J. & Spöttl, G. (2008): "Ways toward a European vocational education and training space: a “bottom-up” approach", Journal of European Industrial Training, Vol. 32 No 2/3 pp. 157 - 170 Braun, V., & Clarke, V. (2006): Using thematic analysis in psychology. Qualitative research in psychology, 3(2), 77-101. Fischer, M.; Rauner, F. (2002): The implications of work process knowledge for vocational education and training. In Boreham, N.C.; Samurçay, R. & Fischer, M. Work process knowledge. London: Routledge, p. 160-170. Fischer, M., Rauner, F. & Zhao, Z. (Eds.) (2015): Kompetenzdiagnostik in der beruflichen Bildung. Methoden zum Erfassen und Entwickeln beruflicher Kompetenz: COMET auf dem Prüfstand. Berlin: Lit Verlag. Leigh, I.W. et al (2007): Assessment Models. Professional Psychology: Research and Practice, Vol 38. No 5, pp 463-473 Olsen, O.J. (2009): Feasability Study VET-LSA. National report from Norway. Department of Sociology, University of Bergen, Bergen May 13th 2009 Rauner, F.; Haasler, B.; Heinemann, L. & Grollmann, P. (2009): Messen beruflicher Kompetenzen, Band 1: Grundlagen und Konzeption des KOMET-Projektes, Berlin LIT Verlag. Robson, C. & McCartan, K. (2015): Real World Research. 4th Edition. NY: Wiley. Shavelson, R. J. (2010): On the measurement of competency Empirical research in vocational education and training 2, 1, pp. 41-63 Stake, R.E. (2005): Multiple Case Study Analysis. NY: The Guilford Press. Winograd, T., & Flores, F. (1986): Understanding computers and cognition: A new foundation for design. Norwood, NJ: Ablex.