Broadly describing the ranking of individuals and families on a hierarchy according to their control of various commodities such as wealth, power, or social status, socioeconomic status (SES) is one of the commonly used contextual variables in statistical analysis in the social and educational sciences (Mueller & Parcel, 1981; Sirin, 2005; White, 1982). SES can be deployed in statistical analysis as a control variable to adjust for background differences in the sample or as an independent causal agent of educational outcomes (White, 1982), uses which are drawn from the well-established finding that higher social classes dominate the strongest educational outcomes. Sirin (2005), in her meta-analysis of studies of SES and academic achievement between 1990 and 2000, notes that SES has three traditional components, parental income, parental education, and parental occupation, with a fourth component, home resources, being less commonly deployed.

Although there is consensus that aforementioned components constitute SES, Bourdieu (1998) cautions against the uncritical use of possessions as totems of social position across cultural contexts or at different points in time, as behaviours and preferences of a social group are not fixed characteristics and “at every moment of each society, one has to deal with a set of social positions which is bound by a relation of homology to a set of activities (the practice of golf or piano) or of goods (a second home or an old master painting) that are themselves characterized relationally” (p. 5). Besides such qualitative changes, the meaning of SES indicators can change gradually due to a shift in the distribution of positional goods. In this vein, Goldthorpe (1996) argues that such a blanket approach fails to take into account the advances and reforms in society and education, including widening participation in higher education and the corresponding shifting of the social capital associated with varying tiers of education. Accordingly, the items used as indicators of social status and position need to be considered systematically, rather than in isolation.

International large-scale assessments such as TIMSS and PISA are repeated every three to five years to trace developments over time. The SES construct in ILSAs is derived from items administered to participants, which raises a concern in the cross-cultural context whereby items that indicate differential levels of wealth in one country are ubiquitous in another. A challenge in designing questionnaires for ILSAs is the need to collect data that can generate meaningful cross-cultural comparisons, as the differentiating abundance and scarcity of certain items across countries and time points makes the continual reapplication of the same measurement instrument inappropriate, and requires us to question the inclusion of some perennial items. However the variation of the items used to identify these constructs over time can undermine their usefulness in making comparisons, particularly when it comes to items indicating home possessions (Pokropek, Borgonovi, & McCormick, 2017; Rutkowski & Rutkowski, 2013), which has been the focus of more research than the other components.

Against this background, the following research questions are considered:

1. How stable are the instruments used to measure socioeconomic status in PISA and TIMSS?
2. Is the inclusion of perennial items in the measurement of socioeconomic status in PISA and TIMSS justifiable empirically and theoretically?

I examine SES indicators in international assessments in three steps. First, I will review SES items that have been used in PISA and TIMSS (grade 8) and identify which common items (i.e. excluding national adaptations) are replicated across contextual questionnaires for the various cycles of international assessments. Second, I will conduct content analyses using Sirin’s four components of SES as a conceptual framework to examine the content validity of SES measures in PISA and TIMSS with a special focus on perennially deployed items. Third, properties of these common indicators of socioeconomic status across multiple cycles of international assessments will be examined. Response patterns for a selected group of countries participating in multiple cycles of these studies – representing OECD and non-OECD countries and distributed across the globe – will be interrogated in further detail to justify the empirical and theoretical inclusion of the items in measures of SES and application in a cross-cultural context.

Preliminary results suggest that there are differences in the distributions of common indicators over time and between countries. A striking result is that the number of books in the home is the only staple SES item in across all cycles of PISA and TIMSS. However, the meaning of this indicator has probably changed due to the development of e-reader technology and reading apps. Changes in the empirical distribution of the item support this supposition: when examining the responses of, for example, Swedish grade 8 students to the question “About how many books are there in your home? (Do not count magazines, newspapers, or your school books)” for the 1995 and 2015 cycles of TIMSS, a decline in the mean for the item from 3.93 in 1995 to 2.06 in 2015 was observed. Calculating the standardized mean differences reveals very large differences (Cohen, 1988) in these items (Cohen’s d 1.54). Besides the number of books in the home, TIMSS repeats only one further item measuring home possessions with consistency across all cycles: possession of a study desk. This item, however, has almost no variation in developed countries. Contextual questionnaires for PISA provide more consistency in item inclusion. Nonetheless, initial analysis of the items used to indicate SES are interesting. Examination of the highest parental education level recoded in PISA in 2000 and 2015 showed that this measure was stable in Indonesia (Cohen’s d .00), while a notable decline in the mean response (Cohen’s d .29) was observed among American respondents. It is expected that the study will identify variability in the stability of repeated items over time.

Bourdieu, P. (1998). Practical reason: on the theory of action. Oxford: Oxford : Polity. Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2. ed.. ed.). Hillsdale: Hillsdale : L. Erlbaum Associates. Goldthorpe, J. H. (1996). Class Analysis and the Reorientation of Class Theory: The Case of Persisting Differentials in Educational Attainment. The British Journal of Sociology, 47(3), 481-505. doi:10.2307/591365 Mueller, C. W., & Parcel, T. L. (1981). Measures of Socioeconomic Status: Alternatives and Recommendations. Child Development, 52(1), 13-30. doi:10.1111/1467-8624.ep8861601 Pokropek, A., Borgonovi, F., & McCormick, C. (2017). On the Cross-Country Comparability of Indicators of Socioeconomic Resources in PISA. Applied Measurement in Education, 30(4), 243-258. doi:10.1080/08957347.2017.1353985 Rutkowski, D., & Rutkowski, L. (2013). Measuring Socioeconomic Background in PISA: One Size Might not Fit all. Research in Comparative and International Education, 8(3), 259-278. doi:10.2304/rcie.2013.8.3.259 Sirin, S. R. (2005). Socioeconomic Status and Academic Achievement: A Meta-Analytic Review of Research. Review of Educational Research, 75(3), 417-453. doi:10.3102/00346543075003417 White, K. R. (1982). The relation between socioeconomic status and academic achievement. Psychological Bulletin, 91(3), 461-481. doi:10.1037/0033-2909.91.3.461