Session Information
09 SES 17 A, Exclusions and Non-response: Contemporary Missing Data Issues in International Large-scale Studies
Symposium
Contribution
The OECD claims that the PISA study allows for representative statements about student performance. In some countries, however, entire schools and individual students are excluded from taking the tests. In addition, an increasing number of students do not sit the test for various reasons and are therefore missing in the data. In PISA 2018 more than 10 percent of the Swedish students in the PISA sample were excluded from the test or did not participate for other reasons. While this is exclusions and non-response is considered a problem (Anders et al., 2020; Micklewright et al., 2012), it is difficult to quantify the bias because the performance levels of the excluded/non-responding schools or their students are generally unknown. To address this problem, we constructed a unique database combining Swedish PISA data with Swedish register data. This database includes national tests results, subject grades as well as information on parental education and migration background for all Swedish students tested in PISA 2018. Moreover, our database also comprises corresponding information for the full cohort of students that was eligible to sit the PISA test (100 000 students born in 2002). We compare the performances and background data of the PISA sample with the entire population of 15-year- olds to shed light on any bias in PISA. The results of the analyses reveal certain degree of bias and cast doubt on the representativeness of the 2018 PISA results in Sweden. Based on the results of the national tests available for all students from Sweden, we find that students who participated in PISA perform on average more than one standard deviation better than students who were excluded from PISA or did not participate for other reasons. The findings are discussed in relation to the general problem of missingness in survey data as well as in relation to the comparability of results over time in PISA.
References
Anders, J., Has, S., Jerrim, J., Shure, N., & Zieger, L. (2020). Is Canada really an education superpower? The impact of non-participation on results from PISA 2015. Educational Assessment, Evaluation and Accountability, 33, 1, 229-249. https://doi.org/10.1007/s11092-020- 09329-5 Micklewright, J., Schnepf, S. V., & Skinner, C. J. (2012). Non-response biases in surveys of school children: the case of the English PISA samples. Journal of the Royal Statistical Society. Series A (General), 175, 915–938.
Search the ECER Programme
- Search for keywords and phrases in "Text Search"
- Restrict in which part of the abstracts to search in "Where to search"
- Search for authors and in the respective field.
- For planning your conference attendance you may want to use the conference app, which will be issued some weeks before the conference
- If you are a session chair, best look up your chairing duties in the conference system (Conftool) or the app.