Across education systems, pressures to standardise assessment collide with traditions of teacher autonomy. This tension raises a crucial question: What counts as valid assessment?

The fundamental nature of assessment is often ignored in policy debates and public discourse. To address this gap, Jönsson (2020) has proposed a distinction between two approaches: assessments based on qualitative (or evaluative) judgments made by assessors and those grounded in quantitative measurement principles informed by psychometrics. In this study, we refer to these paradigms as judgment and measurement.

Psychometrics traces its origins to the latter half of the 19th century, with major advances occurring during the 20th century in relation to the measurement of human intelligence. From this foundation, numerous branches have emerged, including tests of scholastic aptitude, personality, and academic achievement (e.g., Black, 1998; Urbina, 2004). Qualitative judgments (Sadler, 1989), also called evaluative judgments (Boud et al., 2018), are used to appreciate the quality of student performance. According to Sadler (1989), a qualitative judgment is made by a knowledgeable expert and is not reducible to a formula that nonexperts can apply.

The most notable difference between evaluative judgment and a psychometric perspective lies in their focus. In psychometrics, assessment is treated as an inferential process, targeting an invisible theoretical construct such as knowledge. Consequently, psychometric approaches depend on indirect indicators, aggregating data from multiple items. In contrast, evaluative judgment centers on the quality of performance itself. Assessing an essay or report is a direct act, requiring no inference about hidden traits.

Building on this conceptual distinction, this study examines how these paradigms framed summative final assessment and positioned teachers in the context of a nationwide Finnish assessment reform. The reform introduced detailed national assessment criteria for final grades at the end of lower-secondary education, aiming to make teachers’ grading more consistent while preserving classroom-based assessment practices.

A clear understanding of final assessment is essential because of its profound impact on students’ futures and the functioning of society. In this context, teachers are users of public power and may need to justify their grading decisions. Clarity in grading principles is critical: Should professional judgment count as valid evidence, or should grades rely solely on external measures? Do we want teachers to act asassessment machines? Or do we want to externalise assessment to tests, viewing teachers as technicians without power over assessment?

We analyzed two complementary datasets: 56 official and media documents saping public discourse during the reform, and 66 semi-structured interviews with lower-secondary subject teachers across six disciplines.

Our analysis of the public and official documents showed that the measurement and judgment paradigms were intertwined in discourse, and so tightly that it was difficult to disentangle them. Moreover, teachers were positioned in somewhat confusing ways both 1) as parts of the assessment machine (conductors of measurement) and 2) as assessment machines (professionals using evaluative judgment).

In the interview data, we identified three distinct ways in which teachers referred to assessment and their positionings within it: 1) as designers of the assessment machine, who constructed instruments to ensure reliability (referring to measurement), 2) as assessment machines, who applied holistic, criteria-based judgments that resisted reduction to mere metrics (referring to judgment); and 3) as calibrators who adjusted quantitative indicators intuitively with the criteria (referring to intuitive judgment).

Our findings indicate that these paradigms of assessment are intertwined in societal discourse, simultaneously positioning teachers as agentic professionals and sources of error – an inherently contradictory stance. This underscores the need for clearer conceptualisations of assessment to inform both policy decisions and practical implementation, clarifying whether teachers are expected to act as objective technicians or professional interpreters.

We conducted a qualitative, theory-informed analysis combining (1) documentary and media sources and (2) teacher interviews. The document data (n = 56) included official guidance and reports from the Finnish National Agency for Education (FNAE) and the Ministry of Education and Culture (MEC), alongside prominent news coverage from Helsingin Sanomat (the most widely published newspaper in Finland) and Yleisradio (Finland’s national public service broadcaster) during 2017–2024. To identify relevant documents, we used the search term “päättöarv*” (final assess*) across both outlets for the years 2017–2024. To understand how teachers themselves related to the measurement and judgment paradigms, we analysed teacher interviews that were collected as part of the nationwide research project funded by the MEC. The interview sample comprised 66 lower-secondary subject teachers from six disciplines (history, mathematics, physics, mother tongue and literature, English as a foreign language, Finnish/Swedish as a second language). Interviews were semi-structured, discipline-informed, and averaged 53 minutes (range: 23–87 minutes), covering experiences of the reform, criteria use, grading practices, and perceived impacts. Participation was voluntary with informed consent. We operationalized two paradigms of assessment—psychometric measurement and evaluative judgment—drawing on Jönsson’s (2020) conceptual paper. Indicators of a psychometric paradigm included quantitative scoring, summarizing scores from several items, inferences about latent traits, primary focus on reliability, and low teacher autonomy. Indicators of and evaluative judgment paradigm included assessment of performance quality, holistic use of several criteria simultaneously, direct (non-inferential) assessment, primary focus on validity, and high teacher autonomy. First, given the large volume of data, we began by extracting parts that addressed assessment as a psychometric practice or a process of evaluative judgment. Next, based on these excerpts, we analyzed how and by whom these two paradigms were produced and how they were potentially contested across the datasets. Using the abovementioned indicators, we systematically mapped how final assessment was conceptualised throughout the reform. Finally, we examine how teachers were positioned in relation to assessment (Harré et al., 2009), focusing on how ideal, normal and avoidable teacher positionings were articulated by different actors and documents, as well as by teachers themselves.

Our study shows that the discourse around assessment reform entwined the two distinct conceptualizations of assessment: psychometric measurement and evaluative judgment. Public and official communications simultaneously affirmed teacher professionalism and questioned teacher reliability. This double discourse framed teachers as both expert interpreters and potential sources of error, creating an inherently paradoxical positioning. Similarily, teachers’ accounts reflected various conceptualizations of assessment and assessors. These illustrate adaptive professionalism but also reveal conceptual tensions: when policy discourse merges psychometric measurement with evaluative judgment, teachers receive ambiguous signals about what constitutes valid evidence and how final grades should be determined. We argue that conceptual clarity is essential for both policy and practice. Assessment systems should articulate when a psychometric orientation is warranted and when evaluative judgment is central. Professional development practices should be designed accordingly. If the differences between the paradigms are not understood, and if measurement is framed as a superior assessment paradigm or used as a benchmark for teachers’ judgments, this can contribute to mistrust in teacher professionalism (see Novak & Carlbaum, 2017). The conflicting expectations placed on teachers – between acting as objective technicians and professional interpreters – echoes long-standing global tensions in educational assessment (e.g., Broadfoot & Black, 2004; Klenowski, 2011), particularly in how assessment is understood publicly (Gardner, 2017). Finland offers a compelling case due to its strong tradition of teacher autonomy. That teachers feel both empowered and scrutinized in this context suggests that external pressures to standardize and quantify assessment (often introduced by global actors such as the OECD) can reshape professional identities even within high-autonomy systems. Ultimately, assessment does not merely measure learning; it organizes professional work and identities. The Finnish case suggests that sustaining trust in teacher-determined final grades requires naming the paradigms at play and designing infrastructures that help teachers judge well.

Black, P. (1998). Testing: Friend or foe? Theory and practice of assessment and testing. Routledge. Boud, D., Ajjawi, R., Dawson, P., & Tai, J. (2018). Developing evaluative judgement in higher education. Routledge. Broadfoot, P., & Black, P. (2004). Redefining assessment? The first 10 years of assessment in education. Assessment in Education: Principles, Policy & Practice, 11(1), 7–26. https://doi.org/10.1080/0969594042000208976 Gardner, J. (Ed.). (2017). The public understanding of assessment. Routledge. Harré, R., Moghaddam, F. M., Cairnie, T. P., Rothbart, D., & Sabat, S. R. (2009). Recent advances in positioning theory. Theory & Psychology, 19(1), 5–31. https://doi.org/10.1177/095935430810 Jönsson, A. (2020). Definitions of formative assessment need to make a distinction between a psychometric understanding of assessment and “evaluative judgment.” Frontiers in Education, 5. https://doi.org/10.3389/feduc.2020.00002 Klenowski, V. (2011). Assessment for learning in the accountability era: Queensland, Australia. Studies in Educational Evaluation, 37(1), 78–83. https://doi.org/10.1016/j.stueduc.2011.03.003 Novak, J., & Carlbaum, S. (2017). Juridification of examination systems: Extending state level authority over teacher assessments through regrading of national tests. Journal of Education Policy, 32(5), 673–693. https://doi.org/10.1080/02680939.2017.1318454 Sadler, D. R. (1989). Formative assessment and the design of instructional systems. Urbina, S. (2004). Essentials of psychological testing. John Wiley and Sons.