The purpose of large-scale international assessments is to compare the educational achievement of young people across countries. In order for such cross-national comparisons to be robust, it is vital that the participating students are representative of the target population (Micklewright, Schnepf and Skinner 2012). In this paper we consider whether this is the case for one high-performing country (Canada) that has recently been described as an “education super-power” (Coughlan 2017). Our analysis illustrates how the PISA 2015 sample for Canada only covers around half of the 15-year-old population, compared to over 90 percent in countries like Japan and South Korea (OECD 2017). We discuss how this situation emerges from differences in how children with Special Educational Needs are defined and treated, variation in school participation rates and (perhaps most importantly) the comparatively high-rates of pupil absence in Canada during the PISA study. The paper describes how the OECD attempts to limit such issues within the design of PISA, but also the limitations with some of approaches currently used. Particular attention is paid to the “non-response bias analysis” that some countries routinely conduct where responses rates are not sufficiently high. The paper concludes by investigating how Canada’s performance in the 2015 PISA rankings would change under different assumptions of how the set of non-participating students would have performed were they to have sat the PISA test.

Coughlan, S. 2017. How Canada became an education superpower. Accessed 23/01/2019 from www.bbc.co.uk/news/business-40708421 Micklewright, J. , Schnepf, S. V. and Skinner, C. J. (2012) Non-response biases in surveys of school children: the case of the English PISA samples. J. R. Statist. Soc. A, 175 (4), 915-938. OECD. 2017. PISA 2015 technical report. OECD: Paris.