Education quality and equity is at the heart of the 2030 Agenda for Sustainable Development and the Education 2030 Framework for Action. With the SDG goal 4 the world’s countries committed to “Ensure inclusive and equitable quality education and promote lifelong learning opportunities for all”. The SDGs stress the need for quality education and effective learning outcomes (target 4.1), which are strongly linked to aspects of inclusion and equity and conclude: “No education target should be considered met unless met by all” (UNESCO, 2015, p. 7). This strive for equity is also reflected in target 4.5. The current monitoring and reporting on SDG goal 4 target 4.5 has focus on gender, disability, language of children and Migration and forced displacement (UNESCO, 2016, p. 256). The current reporting states that children taught and tested in languages they do not speak at home are hindered in their early acquisition of reading and writing skills. To promote education in the first or home language of children, a thematic indicator on SDGs is developed showing the share of pupils in primary education whose first or home language is the language of instruction (UNESCO Institute for Statistics, 2017). With the implicit call for instruction in the children’s first or home language, expectations are raised that language policies are a key for the improvement of learning outcomes.

Yet, caution is needed when language policies are portrait as single key to improve learning outcomes for students with other languages. As UNESCO, 2015, p. 44 states, “attention to poverty must remain a priority, as poverty is still the single greatest barrier to inclusion at all levels and in all regions of the world.” Language minorities and families of children having home languages other than the language of instruction tend also to face a higher risk of poverty. Therefore, reporting on education and language should include a perspective on poverty and social background of students. For that reason, also pressure in some countries on migrant families to abandon their home languages cannot be the key to the improvement of outcomes (Agirdag & Vanlaar, 2016). Analysis of PISA data show, that controlling for students’ socio-economic status reduces performance gaps between non-immigrant and immigrant students in most developed countries (OECD, 2015). National reporting also shows the impact of social disadvantages of learners with different languages on their achievement, for example for primary education in Austria (Breit, Bruneforth, & Schreiner, 2016, p. 102).

Although this relation is not new to research, it seems the relation of poverty and language needs more attention in the monitoring of educational goals. This presentation aims to address this gap by an analysis of language of instruction as an instructional factor behind achievement gaps in mathematics and reading for countries participating in TIMSS/PIRLS 2011. For the analysis having the same home language and language of instruction as is interpreted as an educational treatment to isolate the impact of language from other characteristics of student populations.

The analysis is based on data from the 2011 TIMSS/PIRLS study. This allows to examine achievement gaps in mathematics and reading for identical student populations. The analysis draws on the student and parent questionnaire of TIMSS/PIRLS. To deal with substantial amounts of missing data, especially for the parent questionnaire, data were first imputed using nested multiple imputations to properly handle the combination of missing data and previously calculated multiple plausible values for achievement scores. For the main analysis two matching techniques are applied in comparison to identify the average treatment effect (ATE) for children instructed in another language than their home language: Propensity score matching as implemented by Sekhon (2011) and a potential outcome approach, as it is implemented by Freunberger, Robitzsch, and Pham (2014) for the Austrian national assessments. The presentation aims to match not only students by their individual characteristics, but to match pairs of students who received primary education in the same class rooms allowing to control many other factors contributing to achievement gaps. This approach is limited by substantial measurement errors due to relatively small numbers of successful matches, but can enrich the discussion on international level.

The presentation shows for countries in TIMSS/PIRLS partially substantial reductions in achievement gaps when controlling for other student characteristics and discusses how the interplay of language and poverty could be better reflected in education monitoring. It also discusses the limitations of TIMSS/PIRLS data with respect to capture the social background of students.

Agirdag, O., & Vanlaar, G. (2016). Does more exposure to the language of instruction lead to higher academic achievement? A cross-national examination. International Journal of Bilingualism, 22(1), 123–137. https://doi.org/10.1177/1367006916658711 Breit, S., Bruneforth, M., & Schreiner, C. (Eds.). (2016). Standardüberprüfung 2015 Deutsch, 4. Schulstufe - Bundesergebnisbericht. Salzburg. Retrieved from https://www.bifie.at/node/3360 Freunberger, R., Robitzsch, A., & Pham, G. (2014). Hintergrundvariablen und spezielle Analysen. Der ursprüngliche Titel der Publikation lautete: Hintergrundvariablen und spezielle Analysen in der BIST-Ü-M4. Retrieved from https://www.bifie.at/node/2782 OECD. (2015). Immigrant Students at School: Easing the Journey towards Integration. Paris: OECD Publishing. Sekhon, J. S. (2011). Multivariate and Propensity Score Matching Software with Automated Balance Optimization: The Matching Package for R. Journal of Statistical Software, 42(7). UNESCO. (2015). Education 2030. Incheon Declaration and Framework for Action. Paris: UNESCO. Retrieved from http://unesdoc.unesco.org/images/0024/002456/245656e.pdf UNESCO (Ed.). (2016). Global Education Monitoring Report 2016: Education for people and planet: Creating sustainable futures for all (Vol. 2016). Paris: Unesco Publishing. Retrieved from http://unesdoc.unesco.org/images/0024/002457/245752e.pdf UNESCO Institute for Statistics. (2017). Metadata for the global and thematic indicators for the follow-up and review of SDG 4 and Education 2030. Retrieved from http://uis.unesco.org/sites/default/files/documents/metadata-global-thematic-indicators-sdg4-education2030-2017-en_1.pdf