09 SES 05 A, Issues in Measurement and Sampling in Large Scale Assessments
Of the OECD countries that participate in the Programme for International Student Assessment (PISA), Turkey has one of the lowest levels of performance and the highest rates of improvement in PISA scores between 2003 and 2012. New evidence presented in this paper suggests that existing accounts have underestimated both progress and inequity in Turkey because they did not take into account the large proportion of 15-16 year olds that are ineligible for the PISA sampling frame, either because they are no longer in school or because they are severely delayed. Using Turkey’s Demographic and Health Survey (DHS) data for 2003, 2008 and 2013 we show that the proportion of 15-16 year olds that were actually eligible for the PISA sample in Turkey has nearly doubled from 45% in PISA 2003, to 80% in PISA 2012. By combining household survey data on access/attainment and PISA survey data on learning outcomes we show that: (1) the improvement in the percentage of 15-16 year olds reaching Level 2 in PISA (functional literacy and functional numeracy) is up to twice as large as that reflected in official PISA reports, (2) the gap in functional literacy rates between rich and poor youth in 2012 is 2.3 times as large as was previously thought, and (3) contrary to earlier research the gap between rich and poor has not declined between 2003 and 2012. The paper emphasises the importance of accounting for sample eligibility and representivity when making inter-country and inter-temporal comparisons using PISA (or any other international assessment) data, particularly for developing countries with expanding education systems.
Bloem, S. (2013). PISA for low- and middle-income countries. OECD Education Working Papers, (93), 40. Dülger, I. (2004). Case Study on Turkey: Rapid Coverage for Compulsory Education Program. In Scaling Up Poverty Reduction (pp. 1–46). Shanghai, China. Filmer, D., Hasan, A., & Pritchett, L. (2006). A Millennium Learning Goal: Measuring Real Progress in Education. SSRN Electronic Journal. http://doi.org/10.2139/ssrn.982968 Gur, B., Celik, Z., & Ozoglu, M. (2012). Policy options for Turkey: a critique of the interpretation and utilization of PISA results in Turkey. Journal of Education Policy, 27(1), 1–21. Hanushek, E. A., & Woessmann, L. (2011a). Sample Selectivity and the Validity of International Student Achievement Tests in Economic Research. Economics Letters. Retrieved from http://www.sciencedirect.com/science/article/pii/S0165176510003605 Hanushek, E., & Woessmann, L. (2008). The role of cognitive skills in economic development. Journal of Economic Literature, 46(3), 607–668. Retrieved from http://www.jstor.org/stable/27647039 Lewin, K. M., & Little, A. W. (2011). Access to education revisited: Equity, drop out and transitions to secondary school in South Asia and Sub-Saharan Africa. International Journal of Educational Development, 31(4), 333–337. OECD. (2004). PISA Report 2003: Learning for Tomorrow’s World. Paris: Organization for Economic Cooperation and Development. OECD. (2012b). PISA 2012 Results: Excellence through Equity: Giving Every Student the Chance to Succeed (Vol.2). Paris: Organization for Economic Cooperation and Development. Pritchett, L. (2013). The Rebirth of Education: Schooling Ain’t Learning. 2013. Washington, DC.: Center for Global Development. Rotberg, I. (1995). Myths About Test Score Comparisons. Science, 270(5241), 1446–1448. http://doi.org/10.1126/science.270.5241.1446 Spaull, N., & Taylor, S. (2015). Access to What? Creating a Composite Measure of Educational Quantity and Educational Quality for 11 African Countries. Comparative Education Review, 59(1), 133–165. Taylor, S., & Spaull, N. (2015). Measuring access to learning over a period of increased access to schooling: The case of Southern and Eastern Africa since 2000. International Journal of Educational Development, 41, 47–59. UNESCO Institute for Statistics. (2010). Measuring educational participation: Analysis of data quality and methodology based on ten studies (Technical Paper 4).
00. Central Events (Keynotes, EERA-Panel, EERJ Round Table, Invited Sessions)
Network 1. Continuing Professional Development: Learning for Individuals, Leaders, and Organisations
Network 2. Vocational Education and Training (VETNET)
Network 3. Curriculum Innovation
Network 4. Inclusive Education
Network 5. Children and Youth at Risk and Urban Education
Network 6. Open Learning: Media, Environments and Cultures
Network 7. Social Justice and Intercultural Education
Network 8. Research on Health Education
Network 9. Assessment, Evaluation, Testing and Measurement
Network 10. Teacher Education Research
Network 11. Educational Effectiveness and Quality Assurance
Network 12. LISnet - Library and Information Science Network
Network 13. Philosophy of Education
Network 14. Communities, Families and Schooling in Educational Research
Network 15. Research Partnerships in Education
Network 16. ICT in Education and Training
Network 17. Histories of Education
Network 18. Research in Sport Pedagogy
Network 19. Ethnography
Network 20. Research in Innovative Intercultural Learning Environments
Network 22. Research in Higher Education
Network 23. Policy Studies and Politics of Education
Network 24. Mathematics Education Research
Network 25. Research on Children's Rights in Education
Network 26. Educational Leadership
Network 27. Didactics – Learning and Teaching
The programme is updated regularly (each day in the morning)
- Search for keywords and phrases in "Text Search"
- Restrict in which part of the abstracts to search in "Where to search"
- Search for authors and in the respective field.
- For planning your conference attendance you may want to use the conference app, which will be issued some weeks before the conference
- If you are a session chair, best look up your chairing duties in the conference system (Conftool) or the app.