International Large Scale Assessments (ILSA), such as PISA and ICCS, provide internationally comparative data on students' knowledge and abilities in various subjects. The results across assessments permit countries to make comparisons of their educational systems over time and in a global context.  To make this possible, the implementation and the methodology on which the studies are based need to be rigorously standardized and of high quality. But even in a well-designed study, missing data almost always occurs. Missing data can reduce the statistical power of a study and can produce biased estimates, leading to invalid conclusions. The mechanisms by which missing data occurs are many. Such a mechanism emerge, for example, from studies based on low stake tests (which ILSA should be considered as). In low stake tests the students nor their teachers receive any feedback based on the students' results. Besides risking reduced validity of results from comparisons, both over time and between countries, low stake tests run the risk of giving rise to a greater proportion of missing data.

Sweden has a long tradition of high quality population administrative register data and this tradition has led us into having a great deal of data linked to the individuals via so-called social security numbers. It is relatively common for researchers and authorities to employ these high quality data in their analysis entailing more reliable results. The Swedish National Agency for Education regularly use register data when producing the official statistics and to a certain extent also when carrying out evaluation studies.

The ILSA:s, used to evaluate the condition of the Swedish schooling system, both by the Swedish National Educational Agency as well as by decision-makers and other stakeholders. To further the possibilities of secondary analyses and to increase relevance t to the national context, it is therefore pertinent to collate data from registers with data from the ILSA:s.

Historically, the Swedish National Agency for Education has only been able to link register data to ILSA data for participating students. This is because the participating students are considered as having given their consent for such linkages. However, before conducting PISA 2022 and ICCS 2022, the legal requirements (?) changed so that it became possible for the Swedish National Agency for Education to link register data also to nonresponding students, i.e. not only to the participating students.

The Swedish samples in PISA 2022 and ICCS 2022 consist of 7 732 15-year-olds and 3 900 students in grade 8, respectively. After the students who are to be excluded due to cognitive or physical impairment or alternatively due to not having good enough skills in the Swedish language, 7 203 in PISA 2022 and 3 632 in ICCS 2022 remain. Of these, the weighted student nonresponse is 15 percent and 13 percent in PISA and ICCS respectively. By employing register data, such as for example the students' final grades in primary school, migration background and the parents' level of education, on the full sample we have studied covariation of student nonresponses and student background characteristics (Swedish National Agency for Education, 2023a; Swedish National Agency for Education 2023b). Furthermore, we have carried out post-stratification type analyses (Little & Rubin, 2020)) to estimate the effect of nonresponses on students’ achievement. Finally, we compared students’ achievements, computed with PISA:s and ICCS rather non-informative nonparticipation adjusted weights, and students’ achievements computed with nonparticipation weights adjusted with register data. (OECD, 2023; IEA, 2023).

The results indicate that student nonresponses lead to a bias that contributes to a certain overestimation of the students' average results Hence, Sweden's results seem to be too high given which students that participated as well as which did not participate. But, the overestimation differs between the two studies. In PISA the bias seems to be larger than the bias in ICCS and where the bias seems to lead to a significant overestimation of the students’ results in PISA the bias in ICCS seems to have a non-significant effect on the students’ results. Furthermore, we find that regardless of whether we study PISA or ICCS, two studies that differ methodologically in several aspects but are similar in the way of compensating for any person nonresponse bias, the effect of the missing-compensating elements on student’s achievements is negligible. The results of this study in terms of how the missingness lead to overestimation of students' average results in ILSA:s are consistent with previously published studies, both in relation to ILSA:s (Micklewright et al., 2012; Meinck et.al., 2023) and more generally to sample studies in general (Groves & Peytcheva, 2008; Brick & Tourangeau, 2017). However, more would need to be done as we do not know the relationship between the proportion of missingness and the size of its’ bias. And we do not know if this relationship changes over time or how this relationship might differ in an international comparison. Furthermore, when compensating for missing data our results lead to the questioning of how reasonable it is to make the assumption of missing completely at random (MCAR). Something that is commonly done in ILSA:s given a sampled school or class.

Groves & Peytcheva. (2008). The Impact of Nonresponse Rates on Nonresponse Bias: A Meta-Analysis. IEA. (2023). ICCS 2022 Technical Report. Meinck et.al. (2023). Bias risks in ILSA related to non‑participation: evidence from a longitudinal large‑scale survey in Germany (PISA Plus) Micklewright et al. (2012). Non-response biases in surveys of schoolchildren: the case of the English Programme for International Student Assessment (PISA) samples. OECD. (2023). PISA 2022 Technical Report. Roderick J. A. Little, Donald B. Rubin. (2020). Statistical Analysis with Missing Data, 3rd Ed. Swedish National Agency for Education (2023a). ICCS 2022 metodbilaga. Swedish National Agency for Education (2023b). PISA 2022 metodbilaga.