Gathering evidence for or against the validity argument of large-scale tests is a crucial aspect of maintaining their integrity. One important source of such evidence is the test’s internal structure, which can be investigated using Differential Item Functioning (DIF) analyses. The present study is a comprehensive investigation of DIF in the Swedish Scholastic Aptitude Test (SweSAT), a large-scale test for selection to higher education, related to test takers’ socio-economic status (SES) and language background. In this study, the test takers’ highest parental education level was used as the indicator of SES. Language background was operationalized as whether a test taker took the course “Swedish as a second language” in elementary schoolyear 9 or not. Data on SES and language background were collected when an individual entered the Evaluation Through Follow-up (UGU) dataset at age 16. SweSAT scores were collected later. Using the Mantel-Haenszel procedure on archival data from approximately 3,360 items and nearly one million test takers, the study examined whether any test items systematically favored any group. We hypothesized that DIF would be minimal in the quantitative section and more pronounced in the verbal section, mirroring patterns and their proposed explanations observed previously with gender as the grouping variable. In the verbal domain, vocabulary items were expected to contain the most DIF, followed by sentence completion items and English reading comprehension items, with Swedish reading comprehension items containing the least DIF. Vocabulary items potentially favored groups more familiar with the word’s contextual domain (for example, words drawn from everyday life might have favored lower-SES test takers and those with Swedish as a second language, while words requiring specialized cultural knowledge might have disfavored those groups). Other than that, no specific patterns were expected. The analysis is still in progress, and the results will be presented at the conference. Overall, the findings are expected to contribute to the SweSAT and the public by adding validity evidence based on SES and language background for or against the SweSAT’s validity argument. Any identified DIF patterns attributable to bias will be used to formulate recommendations for enhancing test fairness, such as refining item content or format, thus further strengthening the test’s validity argument and transparency.

Allalouf, A., Hambleton, R. K., & Sireci, S. G. (1999). Identifying the causes of DIF in translated verbal items. Journal of Educational Measurement, 36(3), 185–198. Carlton, S. T., & Harris, A. M. (1992). Characteristics associated with differential item functioning on the Scholastic Aptitude Test: Gender and majority/minority group comparisons. (Report no. ETS-RR-92-64). Princeton, NJ: Educational Testing Service. Holland, P. W., & Thayer, D. T. (1988). Differential item performance and the Mantel- Haenszel procedure. In H. Wainer & H. I. Braun (Eds.), Test validity (pp. 129– 145). Hillsdale NJ: Erlbaum. Schmitt, A. P., Holland, P. W., & Dorans, N. J. (1992). Evaluating hypotheses about differential item functioning. (Report No. ETS-RR-92-8). Princeton, NJ: Educational Testing Service. Stage, C. (2005). Socialgruppsskillnader i resultat på högskoleprovet [Social group differences in scores on the Swedish Scholastic Assessment Test] (Report No. BVM 11:2005). Umeå, Sweden: Umeå University. Wedman, J. (2017). Reasons for gender-related differential item functioning in a college admissions test, Scandinavian Journal of Educational Measurement, 62(6), 959–970. DOI: 10.1080/00313831.2017.1402365 Wester, A. (1997). Differential item functioning (DIF) in relation to item content: A study of three subtests in the SweSAT with focus on gender. (Report No. EM No 27). Umeå, Sweden: Umeå University.