What Do Skills Demonstrations reveal? The Reliability and confidence in the assessment process of Skills Demonstrations in VET

Student assessments for upper secondary vocational qualifications have been reformed such that vocational competence is assessed based on skills demonstrations. Skills demonstrations enable students to demonstrate their competence either at the workplace or in other surroundings that provide an accurate representation of the functional modules relevant to their occupational proficiency. Skills demonstrations in upper secondary vocational qualifications are known as vocational skills demonstrations. In addition to student assessments, national evaluation of learning outcomes in vocational education and training were also reformed to become skills demonstrations-based. As a result, assessment information for national evaluations will be obtained directly from skills demonstrations and separate national tests are no longer needed. However, coordination of student assessments and national evaluations involves tensions between the different aspects of assessment.

The purpose of this study was to provide information on use of assessments to develop the assessment and evaluation system further based on skills demonstrations, to increase theoretical understanding of assessing vocational competence based on vocational skills demonstrations and to observe tensions present in assessments, reliability of assessments and confidence in the assessment process. In addition, this study strives to provide relevant information to facilitate drawing practical conclusions in the process of developing the assessment system based on skills demonstrations. The conceptual framework is based on theories on assessment and assessment concepts involving, in particular, different assessment and evaluation traditions, different functions of assessments and evaluation and criteria or quality requirements set for assessment information. In addition, the relationship between assessment and learning is observed. The research questions posed were as follows: In what way are control and confidence present in the process of assessing occupational proficiency on the basis of vocational skills demonstrations? What kind of assessment information do assessments of vocational competence provide in different contexts, in particular, student assessments, the assessment practises of educational institutions and system-level evaluation? How is reliability of assessment information ensured and how is confidence in the new system of assessing vocational competence on the basis of skills demonstrations promoted?

Skills demonstrations have many functions and purposes in political decision-making and national guidance, but in the finalised system, the function of skills demonstrations is yet to be specified in detail and the objectives of skills demonstrations tend to include both formative and summative assessment targets in parallel. Regulations concerning skills demonstrations involve balancing between tensions: there is ambivalence between control and trust in all contexts of quality assurance of assessments based on skills demonstrations – quality assurance is both rigid and flexible and there are dissonances between different quality requirements. Assessments based on skills demonstrations are considered to be an accurate and usable method of student assessment, because skills demonstrations are authentic situations and assessment is based on set criteria Vocational skills demonstrations increase the accuracy and validity of assessment results, but reliability is an issue in national evaluation of learning outcomes.

Crooks, T.J. 2002. Educational Assessment in New Zealand Schools. Profiles of Educational Assessment Systems World-Wide. Assessment in Education 9 (2), 237–253. Fetterman, D. 2001. Foundations of Empowerment Evaluation. London: Sage Publications. Guba, G.E. & Lincoln, Y.S. 1989. Fourth Generation Evaluation. Newbury Park, CA: Sage. O´Neil O. 2002. A Question of Trust. The BBC Reith Lectures 2002. Cambridge: Cambridge University Press. Patton, M.Q. 1997. Utilization – Focused Evaluation: The new Century Text. 3.painos. Thousand Oaks, CA: Sage Publications. Pawson, R. & Tilley, T. 1997a. Realistic Evaluation. London: Sage Publication. Räkköläinen, M. 2011. What do skills demonstrations reveal? The realibility and confidence in assessment process of skills demonstrations in upper secondary vocational gualifications. Acta Universitatis Tamperensis 1636. Tampere University Press. Saunders, M. 2006a. Do we Hear the Voices? The “Presence“ of Evaluation Theory and Practice in Social Development. Evaluation. The Internation Journal of Theory, Research and Practice 12 (2), 251–264. Stobart, G. 2008. Testing Times. The uses and abuses of assessment. Lontoo: Routledge. Stame, N. 2004a. Theory-based Evaluation and Types of Complexity. Evaluation. The International Journal of Theory, Research and Practice 10 (1), 58–76.