Cross-national comparisons of educational achievement rely upon each participating country collecting nationally representative data. When it comes to missing data, researchers would usually think of omitted answers, not reached questions, and perhaps more general non- response due to non-participation. However, in ILSAs, in most countries specific parts of the international defined target population are - due to various reasons - not considered as of interest or inaccessible, which results that right from the start a part of the population is disregarded. While obtaining high response rates are a key part of reaching this goal, other potentially important factors may also be at play. As noted by Anders et al (2021) and Jerrim (2021), response rates are only part of the story. Other factors – such as the precise definition of the target population and decision about how many schools/students to exclude – also have an impact as well. When taken together, this can result in the data collected having sub-optimal levels of population coverage, jeopardizing a key assumption underpinning cross-country comparisons - that the data for each nation is nationally representative. The paper focuses upon one such issue – exclusion rates – which has received relatively little attention in the academic literature. We elaborate on the causes of missing out a part of the target population, how these are calculated and reported, and how they changed over time. The data we analyze about such exclusion rates are drawn from all IEA studies conducted between 1999 and 2019. This incorporates six rounds of TIMSS, four rounds of PIRLS, two rounds of ICCS and two rounds of ICILS. All countries that took part in any of these studies/cycles are included. Using descriptive analyses (e.g. benchmarks, correlations, scatterplots) and OLS (ordinary least squares regression model) methods, we find there to be modest variation in exclusion rates across countries, and that there has been a relatively small increase in exclusion rates in some over time. We also demonstrate how exclusion rates tend to be higher in studies of primary school students than for studies of secondary school students. Finally, while there seems to be little relationship between exclusion rates and response rates, there is a modest association between the level of exclusions and test performance. Given the relatively high and rising level of exclusions in some countries, it is important that exclusion rates do not increase any further and – ideally – start to decline.

Anders, J., Has, S., Jerrim, J., Shure, N., & Zieger, L. (2020). Is Canada really an education superpower? The impact of non-participation on results from PISA 2015. Educational Assessment, Evaluation and Accountability, 33, 1, 229-249. https://doi.org/10.1007/s11092-020- 09329-5 Jerrim, J. (2021). PISA 2018 in England, Northern Ireland, Scotland and Wales: Is the data really representative of all four corners of the UK? Review of Education, 9(3). doi:10.1002/rev3.3270