This paper presents an ongoing study investigating the role of standard-setting in high-stakes examinations for international secondary school exit qualifications. Rather than treating standard-setting as a discrete technical procedure conducted after test construction, the paper advances the position that standard-setting should be conceptualised as an integral component of the overall validity argument for an assessment. Drawing on an argument-based approach to validity (Kane, 2006; Kane, 2013), the study frames standard-setting decisions as evaluative inferences that require explicit warrants, backing, and rebuttals, rather than as neutral psychometric outcomes. Within this framework, cut scores are understood as consequential judgments that link observed performance to claims about student achievement. These judgments must be supported by coherent evidence relating to construct representation, the quality of expert judgment processes, and the intended and unintended consequences of decisions (Messick, 1989; Mislevy, Almond, & Lukas, 2003). In high-stakes contexts, the defensibility of pass/fail or grade boundary decisions depends not solely on statistical precision, but on the transparency and plausibility of the interpretive and evaluative claims that underpin them (Cizek & Bunch, 2007). The study is situated within the context of a new national school-leaving qualification which is very high stakes in its context. The resulting of the qualification process, employs a hybrid model of criterion-referenced and norm-referenced standard-setting for its secondary school exit-level examinations. The assessment materials analysed include both fictitious and live examination papers composed of closed-type items, primarily multiple-choice questions and other selected-response formats. These items are designed to sample a defined construct domain aligned to curriculum and qualification specifications. Methodologically, the study adopts a qualitative-dominant mixed-methods design (Creswell, 2022). Documentary analysis is used to examine the qualification’s standard-setting policies, technical reports, and grade-award documentation. This is complemented by the analysis of standard-setting meeting artefacts, including judges’ performance descriptors, item-level judgments, and rationale statements. Particular attention is paid to how criterion-based expectations are articulated and subsequently moderated through norm-referenced statistical evidence. The analysis maps these processes onto an explicit validity argument, identifying the claims, warrants, and evidence used to justify final cut scores. The study is currently underway; it is anticipated that hybrid standard-setting models introduce distinctive tensions between construct-based expectations and cohort-relative outcomes. Full findings, including implications for the design and communication of standard-setting decisions in international high-stakes assessment, will be presented at the conference.

Cizek, G. J., & Bunch, M. B. (2007). Standard setting: A guide to establishing and evaluating performance standards on tests. Sage. Creswell JW. (2022). A concise introduction to mixed methods research (2nd ed.) Sage. Kane, M. T. (2006). Validation. In R. L. Brennan (Ed.), Educational measurement (4th ed., pp. 17–64). Praeger. Kane, M. T. (2013). Validating the interpretations and uses of test scores. Journal of Educational Measurement, 50(1), 1–73. Messick, S. (1989). Validity. In R. L. Linn (Ed.), Educational measurement (3rd ed., pp. 13–103). Macmillan. Mislevy, R. J., Almond, R. G., & Lukas, J. F. (2003). A brief introduction to evidence-centered design. ETS Research Report.