Constructing Legitimate Meritocracies: Comparing policies for employing external instruments ensuring comparability of high-stakes teacher judgments in Scandinavian secondary education
Author(s):
Sverre Tveit (presenting / submitting)
Conference:
ECER 2014
Format:
Paper

Session Information

23 SES 02 A, Forms of Assessment in Global Settings

Paper Session

Time:
2014-09-02
15:15-16:45
Room:
B331 Sala de Aulas
Chair:
Anna Tsatsaroni

Contribution

John Carson’s (2006) historical analysis of the evolution of the meritocratic systems in France and the United States is a helpful lens for understanding how two distinct different traditions for determining student achievement emerged in Europe and the United States as the education systems expanded over the course of the nineteenth an twentieth centuries. These two traditions are manifested in different ways in contemporary national assessment policies on the Scandinavian Peninsula. The evolution of the Norwegian and Swedish assessment traditions form the point of departure for this papers’ examination of two main strategies for legitimizing external instruments underpinning meritocratic systems.

Carson describes how the French and American republics responded in different ways to the problem of balancing equality and difference as their education systems evolved. The French adopted a national, universal and comprehensive approach to education with rigorous examinations relying on expert judgments that determined what students should move up in the system. The Americans put more weight on personal attributes than on formal education and embraced the intelligence tests as a means of social advancement or distinction. By the 1920s and 1930s distinct different ways of understanding differences in mental abilities had emerged. The World Wars prompted an increased need to measuring populations’ talent and skills and new theories and empirical approaches to psychometric testing flourished in the United States, which had substantial influence on allies in Europe, particularly in the United Kingdom.

On the Scandinavian Peninsula the European and the American traditions of employing external instruments was manifested in two different strategies for complementing and supporting teachers’ judgments to ensure validity, comparability and thus legitimacy of students’ school-leaving certificates. A commonality of Scandinavian education policies post World War II was the strong trust in teachers. The Norwegian secondary level teaching profession managed to gain control of the external instruments that controlled admission to higher education already in 1884. Since then the teacher profession has been highly involved in producing and grading secondary education examinations, which thus gained high legitimacy among educators (Lysne, 1999). Psychometric tests, on the other hand, became controversial among Norwegian educators due to political and ideological disputes in the 1970s which resulted in a significantly weaker understanding of psychometric testing among educators and policy makers (Tveit, forthcoming).

Swedish educators were highly involved in the American led development of new psychometric instruments post World War II (Lundahl & Pettersson, 2010). The higher education admission tests (SWESAT) and extensive use of national tests in primary and secondary education are evidence of the American influence in contemporary Swedish assessment policy. Whereas educational testing in various ways became a distinct academic discipline and an industry in the United States and the United Kingdom; in Sweden national tests were used to standardize teachers’ grading practices. By ensuring better comparability of teachers’ judgments, these could replace admission examinations (terminated in 1965) as the basis for selection to subsequent education levels. For decades the national tests gained high legitimacy among educators. More professionalization of the testing development and increased emphasis on using the tests to control teachers judgments have however led to decreased legitimacy of the tests in recent years (Lundahl, 2009).

In contemporarily Norwegian policy, teachers’ determination of student achievement is complemented with independent teachers’ assessment of students’ achievements on external examinations. Swedish students’ achievements, on the other hand, are determined by their respective teachers only, however for major subjects a national test produced by trained professionals form part of the assessed evidence. The paper investigates how two distinctively different traditions and policies for using external instruments are applied to ensure legitimacy of meritocratic systems.

Method

A literature review identifying the assessment traditions in Norway and Sweden form the backdrop for a comparative analysis of contemporary policies for educational assessment in both countries. The analysis of policy documents underpinning the last educational reform and contemporary system for educational assessment includes some 100 policy documents from the Parliamentary, Ministry and government agency levels in the period from year 2003 to 2013. Among these are white and green papers, legislation and decrees, commissioning letters and government commissioned research. 10 expert interviews conducted in 2013 are analysed. These include representatives from the political and administrative levels of the Ministry of Education and Research, and the director and selected leaders of relevant units of the Norwegian Directorate for Education and Training and the Swedish National Agency for Education. The expert data help identifying important tacit knowledge, silences and challenging inconsistencies in official written policies. The data analysis draws on the methodological approach of Thematic Analysis – TA (Boyatis, 1998) and Qualitative Content Analysis – QCA ( Schreier, 2012) facilitated with the qualitative data analysis software tool NVivo 10 (Bazeley, 2007). Drawing on theory and previous research on policy borrowing (Lundahl & Waldow, 2009; Halpin and Troyna, 1995; Phillips and Ochs, 2004, 2010; Waldow, 2009) the paper sheds light over how the various purposes of educational assessment that can be identified in international research are emphasized when legitimizing external instruments for educational assessment.

Expected Outcomes

Drawing on literature on educational assessment (Black & Wiliam, 1998; Broadfoot, 1996; Sadler, 1989; Stobart, 2009) a conceptual analysis identified three main aspects of educational assessment that have dominated when Norwegian and Swedish governments aim to legitimize policies for educational assessment: Certifying, governing and supporting student learning. Certifying learning involves teachers’ judgments and external instruments role in reporting student learning. Governing learning involves controlling content and steering the education system based on aggregated assessment data. Supporting learning involves using assessment data to give feedback to students as to how they can improve their learning. While the study shows that the main purpose of the external instruments is to ensure valid certification of student achievement, analyses of the policy documents and expert interview data identify how the purposes of governing and supporting learning also are emphasized to legitimize these policies. The paper shows that the purpose of certifying student achievement was more emphasized in Sweden than in Norway, which put more emphasis on the purpose of supporting student learning. While addressing the comparative differences of emphasis on purposes of assessment, the paper also give attention to paradoxes of policy imports, particularly the notions and concepts of criteria, standards and formative assessment that are imported from the international research community. The study reveals considerable differences in how influence from international actors such as the OECD is reported in policy documents and interpreted by policy makers. The paper demonstrates the complexity of importing and applying policies and practices in new contexts and concludes by addressing some fundamental challenges and pitfalls for policy borrowing in the field of educational assessment.

References

Bazeley, Patricia (2007). Qualitative Data Analysis with NVivo. London: SAGE Publications. Black, Paul & Wiliam, Dylan (1998). Assessment and Classroom Learning. Assessment in Education: Principles, Policy & Practice, 5:1. Boyatis, Richard E. (1998). Transforming Qualitative Information: Thematic Analysis and Code Development. London: SAGE Publications. Broadfoot, Patricia (1996). Education, assessment, and society: a sociological analysis. Bristol: Open University. Carson, John (2006). The measure of merit. Talents Intelligence, and Inequality in the French and American Republics, 1750-1940. Princeton University Press. Halpin, David & Troya, Barry (1995). The politics of Education Policy Borrowing. Comparative Education, 31: 3, pp. 303-310. Lysne, Anders (1999). Karakterer og kompetanse. Stridstema i norsk skolehistorie. AVA forlag. Lundahl, Christian & Waldow, Florian (2009). Standardisation and ‘quick languages’: the shape‐shifting of standardized measurement of pupil achievement in Sweden and Germany. Comparative Education, 45:3, pp. 365-385. Lundahl, Christian & Petterson, Daniel (2010). Den svenska skolens resultat. Från standardprov til PISA. In: Eyvind Elstad & Kirsen Sivesind. Oslo: Universitetsforlaget. Sadler, Royce (1989). Formative assessment and the design of instructional systems. Instructional Science 18: 119-144. Schreier, Margrit (2012). Qualitative Content Analysis in Practice. London: SAGE publications Stobart, Gordon (2008). Testing Times: The Uses and Abuses of Assessment. London: Routledge. Phillips, David & Ochs, Kimberly (2004). Some Methodological Challenges in Comparative Education. British Educational Research Journal, 30:6, pp. 773-784. Phillips, David & Ochs, Kimberly (2010). Processes of Policy Borrowing in Education: some explanatory and analytical devices. Comparative Education, 39:4, pp. 451-461. Tveit, Sverre (forthcoming). Educational Assessment in Norway. Assessment in Education: Principles, Policy & Practice. Waldow, Florian (2009). Undeclared imports: silent borrowing in educational policy-making and research in Sweden. Comparative Education, 45:4, pp. 477-494.

Author Information

Sverre Tveit (presenting / submitting)
University of Oslo
Department of education
OSLO

Update Modus of this Database

The current conference programme can be browsed in the conference management system (conftool) and, closer to the conference, in the conference app.
This database will be updated with the conference data after ECER. 

Search the ECER Programme

  • Search for keywords and phrases in "Text Search"
  • Restrict in which part of the abstracts to search in "Where to search"
  • Search for authors and in the respective field.
  • For planning your conference attendance, please use the conference app, which will be issued some weeks before the conference and the conference agenda provided in conftool.
  • If you are a session chair, best look up your chairing duties in the conference system (Conftool) or the app.