The relationship between young people’s Economic, Cultural and Social status (ECSS) and learning outcomes is a common and well-documented subject in educational research (see for example, Broer, Bai, & Fonseca, 2019; Conger, Conger, & Martin, 2010; Hattie, 2009; O’Connell, 2019; Yang-Hansen, 2008). Furthermore, associations of children’s and young people’s ECSS with a variety of outcomes in a range of other areas and stages of life have been extensively explored. 

The growth of national, regional and international large‑scale assessments in education has furthered the use young people’s ECSS in quantitative research to investigate the relationship between ECSS and education outcomes (Broer et al., 2019; Lietz, Cresswell, Rust, & Adams, 2017; OECD, 2018). Thus, the results from comparative large-scale assessments have great potential to inform the development of strategies and policies to improve and ensure equity in education.

However, definitions and operationalisations of young people’s ECSS across different large-scale assessments are seldom consistent (Osses, Adams, & Schwantner, Forthcoming). These inconsistencies pose significant challenges for comparing results between large-scale assessments, and they limit the usability of findings in addressing policy issues concerning equity in education.

There is broad agreement in the literature, and in most large-scale assessments, on the essential components of ECSS and its operationalisation through key indicators of education, occupation and wealth (American Psychological Association [APA] Task Force on Socioeconomic Status, 2007; Bollen, Glanville, & Stecklov, 2001; Grusky, 2008; Hauser, 1994; Mueller & Parcel, 1981). However, inconsistencies occur in the components included and how each component is operationalised, which is limiting researchers’ capacity to a build a sound evidence base concerning educational outcomes and equity-related background factors. 

The aim of this paper is to investigate whether the interpretations concerning equity in education vary when different indicators of ECSS are used for the same set of students and countries. We expect that findings provide evidence to foster discussion about the need of developing well-defined and consistent contextual indicators that allow policymakers to use findings for implementing actions to address policy issues of equity in education.

Data from PISA 2018 are used to assess variations in the relationship between student’s reading achievement and ECSS when two different indicators of this construct are used. The first indicator is the PISA Index of Economic, Social and Cultural Status. The second indicator is the TIMSS 2018 Index of Home Educational Resources (PISA-HER). We replicate the PISA analyses concerning the relationship between students’ ECSS and reading achievement for the PISA-ESCS and PISA-HER indicators and compare findings. 

Our preliminary results indicate that both indicators – PISA-ESCS and PISA-HER, provide evidence of an existing relationship between reading achievement and student’s ECSS. For example, in OECD countries an average 12% and 14% of the variation in reading achievement is explained when using the PISA-ESCS indicator and the PISA-HER indicator, respectively. However, differences in the interpretation of equity in education are found when looking at individual countries. Among the 37 OECD countries participating in PISA 2018, 12 countries are classified as high achievers (above OECD average in reading performance) and equitable (below the OECD average variation in reading achievement explained by ECSS) when considering the PISA-ESCS indicator. Seven of these countries show low equity in education when the PISA-HER indicator is considered. Inconsistencies such as these emphasise the need for harmonising contextual indicators between studies to provide sound evidence for the design of policy actions aimed at improving equity in education.

Data: to address our research aim, we use PISA data on reading achievement comprising 79 countries of which 37 are OECD countries. The PISA database contains the Index of Economic, Social and Cultural Status (PISA-ESCS) for each student. For this research, PISA data are recoded to obtain variables identical to those used in the TIMSS Home Educational Resources scale for grade 8 students and the PISA-HER indicator, which is constructed following the procedures detailed in the TIMSS 2019 technical report (Martin, von Davier, & Mullis, 2020). PISA and TIMSS use a similar set of indicators to operationalise student’s ECSS – parents’ education and occupation and family wealth (Martin et al., 2020; OECD, 2020). However, there are differences that we hypothesise may have an impact on the observed relationship between ECSS and reading achievement and the subsequent interpretation of findings about equity in education. Both TIMSS and PISA derive their indicator of parents’ education from items coded using the International Standard Classification of Education (ISCED). However, both assessments programs treat the information differently in the Index construction: while PISA converts the information to a continuous indicator of years of education, TIMSS uses a discrete indicator of seven categories. The number of books at home is treated as a discrete categorical variable in PISA and TIMSS, but each with a different number of categories. Possessions at home is also an indicator used by both assessments. However, the TIMSS-HER scale only considers the possession of internet and student’s own room, while PISA includes a list of more than 20 items. Parents’ occupation is an information only included in the PISA-ESCS index. Following the analysis approach outlined in the PISA 2018 results’ report (OECD, 2019), we perform correlation and regression analyses between reading achievement and the PISA-ESCS indicator. The same analyses are undertaken for the PISA-HER indicator. Results of PISA-ESCS and PISA-HER scales are compared in relation to the strength and the slope of their relationship with reading achievement and the difference in PISA score points between students in different quarters of each scale.

Preliminary results show that both indicators – PISA-ESCS and PISA-HER, provide relatively consistent evidence of an existing relationship between reading achievement and student’s ECSS. However, we found that results about the strength of the relationships between reading achievement and ECSS vary for individual countries when using different indicators of the ECSS construct. When different indicators of ECSS are used, the observed results lead to different interpretations about how equitable education systems are. In OECD and Partner countries, the differences in results lead to contradicting interpretations about equity in education: some countries are classified as equitable by one indicator and not equitable by the other. It is expected that the use of different indicators of ECSS will also lead to different findings in relation to differences in PISA score points between most advantaged and most disadvantaged students in terms of ECSS. Inconsistencies such as these provide conflicting evidence to governments and other education stakeholders interested in deriving conclusions that could be used for planning actions aimed at improving equity in education. Therefore, these inconsistencies emphasise the need of turning our attention to the construction of well-defined and consistent indicators of ECSS between different studies.

Broer, M., Bai, Y., & Fonseca, F. (2019). Socioeconomic Inequality and Educational Outcomes. Evidence from Twenty Years of TIMSS: SpringerOpen. Conger, R. D., Conger, K. J., & Martin, M. J. (2010). Socioeconomic status, family processes, and individual development. Journal of Marriage & Family, 72, 685–704. Education Equity Research Initiative. (2016). Measuring Equity in Education: Review of the Global and Programmatics Data Landscape. Retrieved from https://learningportal.iiep.unesco.org/en/library/measuring-equity-in-education-review-of-the-global-and-programmatic-data-landscape Hattie, J. (2009). Visible Learning: a synthesis of over 800 meta-analyses relating to achievement. London, Ney York: Routledge. Lietz, P., Cresswell, J., Rust, K. F., & Adams, R. J. (Eds.). (2017). Implementation of Large‐Scale Education Assessments. Chichester: John Wiley and Sons. Martin, M. O., von Davier, M., & Mullis, I. V. S. (Eds.). (2020). Methods and Procedures: TIMSS 2019 Technical Report. Chestnut Hill: TIMSS & PIRLS International Study Center. O’Connell, M. (2019). Is the impact of SES on educational performance overestimated? Evidence from the PISA survey. Intelligence, 75, 41-47. OECD. (2018). Equity in Education: Breaking down barriers to social mobility. Paris: OECD Publishing. OECD. (2019). PISA 2018 Results (Volume II): Where All Students Can Succeed. Retrieved from https://doi.org/10.1787/b5fd1b8f-en [1 September, 2020] OECD. (2020). Chapter 16. Scaling procedures and construct validation of context questionnaire data - PISA 2018. Retrieved from https://www.oecd.org/pisa/data/pisa2018technicalreport/PISA2018_Technical-Report-Chapter-16-Background-Questionnaires.pdf Osses, A., Adams, R. J., & Schwantner, U. (Forthcoming). Monitoring equity in education: Towards improved indicators of economic, cultural and social status: Conceptual considerations to inform globally comparable and context specific indicators. Centre for Global Education Monitoring, ACER. Yang-Hansen, K. (2008). Ten-Year Trend in SES Effects on Reading Achievement at School and Individual Levels: A Cross-Country Comparison. Educational Research and Evaluation, 14(6), 521-537. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&db=eric&AN=EJ820672&site=ehost-live