Inequality in education continues to be a challenge that affects both policy and practice in contemporary education systems (OECD, 2019a; United Nations, 2015). Several studies showed that the lower the family socio-economic background, the lower the academic success of students in all degrees of education (Langenkamp & Carbonaro, 2018; Nunes et al., 2022; Rodríguez-Hernández et al., 2020; Tan et al., 2023). This effect is problematic for the effort of many countries to fulfill the potential of each pupil to its maximum and to stop the replication of socioeconomic status from parents to children. 

Despite their disadvantaged socio-economic background, some students are academically successful. They are called academic resilient students. Academic resilience is students’ capacity to be successful at school despite their disadvantaged background (OECD, 2011; Rudd et al., 2021; Ye et al., 2021). This construct has recently gained significant attention in research due to its potential to address factors that may enhance students' success (Hunsu et al., 2023; OECD, 2011).

In spite of the growing popularity of the construct, studies significantly differ in the way in which they operationalize and measure academic resilience (Rudd et al., 2021; Tudor & Spray, 2017; Ye et al., 2024). Only two empirical studies have empirically compared these approaches in terms of their influence on analyses measuring the effects of protective factors on academic resilience (Rudd et al., 2023; Ye et al., 2024), and none of them is complex enough to evaluate the effect of the approaches on results. However, if the influence of operationalization on scientific findings were to exist, it could have serious implications for the consideration of the validity of existing research. 

Together with the effect of the operationalization of academic resilience on findings, there is another issue related to the selection of protective factors added to the model. Recent studies have examined many potential protective factors (Hunsu et al., 2023; Ye et al., 2024) usually without any theoretical framework. Given the fact that most of the studies have had cross-sectional designs (Ye et al., 2024) lacking research pre-registration, the validity of the effects of those factors is in question.   

This study aims to examine the relationship between one of the most examined protective factors, motivation (Ye et al., 2024), on academic resilience. Motivation factors are derived from the Expectancy-value theory of motivation framework (Atkinson, 1957; Eccles & Wigfield, 2020). This theory postulates that the decision to participate in a task depends on the perceived expectation of success and the value one gives to the task (Eccles & Wigfield, 2020). Using this theoretical framework could prevent selection bias. 

The main goal is to explore the effect of the operationalization choice of academic resilience on results. The present study examines four types of operationalization according to a systematic review by Rudd et al. (2021) and three thresholds commonly used in academic studies. Considering that OECD has recently been one of the main leaders of academic resilience research (OECD, 2011), this research aims to explore the effect of operationalization of academic resilience in the PISA dataset. This study seeks to examine these effects in diverse contexts, utilizing data from all countries participating in the PISA testing. Only when we better understand the effect of different operationalization of academic resilience of findings could we better rely on the results and consequently, create effective interventions aimed at boosting academic resilience in pupils.  

In this study, we utilize OECD's Programme for International Student Assessment 2018 dataset (PISA; OECD, 2019b) which gathered data from more than 600 thousand 15-year-old students from 79 countries. Academic achievement is operationalized as a score in a PISA reading test, and risk factor as a low socio-economic status represented by the ESCS index. Motivational variables assessed by self-report include perceived competence of own reading skills, level of reading enjoyment, the perceived value of school in general, and prestige of expected occupation. Except for the latest one, motivational factors are assessed as latent variables each loaded with three to five indicators. We tested four approaches described in the systematic review by Rudd et al. (2021). Measurement approaches 1 and 2 identify academically resilient students based on criteria for achievement and risk and then the analyses are conducted. Academic resilience is a binary dependent variable. Contrary to the first model, the second one selects those students based on their positive residuum. Approaches 3 and 4 use reading performance as a proxy of academic resilience. Academic achievement is a continuous dependent variable, and the risk factor is added as a continuous predictor or an interaction term. Contrary to model 4, model 3 reduces the sample to students with low socio-economic status. For each model, several thresholds were used for identifying academically resilient students. Given the research questions focusing on comparing different approaches to measurement, we chose the structural equation modeling approach as the most flexible tool. We conducted analyses separately for each country and later put them together using a meta-analytic strategy.

We hypothesize that the operationalization approach to academic resilience influences the findings related to the effects of motivational protective factors on academic resilience. Given the explorative nature of this study, we did not assume any specific relationships between the approach and the found effects. We anticipate that the sample size, the choice between a binary or continuous dependent variable and the strictness of the thresholds influence statistical power and therefore impact the findings. Based on the preliminary meta-analysis results, the final effects seemed quite robust against the choice of the operationalization approach. In most of the models, the effects of all motivational variables are significant. However, the variability between countries is substantial and the results on a country level are influenced by the choice of operationalization approach. The effects of thresholds are apparent in a meta-analytical view and on a country level. The results have consequences for academic research, particularly the one using PISA data, in terms of the choice of academic resilience operationalization approach. This work was supported by the NPO ‘Systemic Risk Institute’ number LX22NPO5101, funded by European Union—Next Generation EU (Ministry of Education, Youth and Sports, NPO: EXCELES) and by the project Mediated Society (MEDIS:ON) CZ.02.01.01/00/23_025/0008713 co-financed/supported by the European Union and Czech Republic.

Atkinson, J. W. (1957). Motivational determinants of risk-taking behavior. Psychological review, 64(6p1), 359–372. Eccles, J. S., & Wigfield, A. (2020). From expectancy-value theory to situated expectancy-value theory: A developmental, social cognitive, and sociocultural perspective on motivation. Contemporary Educational Psychology, 61, 101859. https://doi.org/10.1016/j.cedpsych.2020.101859 Hunsu, N. J., Oje, A. V., Tanner-Smith, E. E., & Adesope, O. (2023). Relationships between risk factors, protective factors and achievement outcomes in academic resilience research: A meta-analytic review. Educational Research Review, 41, 100548. https://doi.org/10.1016/j.edurev.2023.100548 Langenkamp, A. G., & Carbonaro, W. (2018). How School Socioeconomic Status Affects Achievement Growth across School Transitions in Early Educational Careers. Sociology of Education, 91(4), 358–378. https://doi.org/10.1177/0038040718802257 Nunes, C., Oliveira, T., Santini, F. D. O., Castelli, M., & Cruz-Jesus, F. (2022). A Weight and Meta-Analysis on the Academic Achievement of High School Students. Education Sciences, 12(5), 287. https://doi.org/10.3390/educsci12050287 OECD. (2011). Against the Odds: Disadvantaged Students Who Succeed in School. OECD. https://doi.org/10.1787/9789264090873-en OECD. (2019a). PISA 2018 Results (Volume I): What Students Know and Can Do. OECD. https://doi.org/10.1787/5f07c754-en OECD. (2019b). Programme Programme for International Student Assessment 2018 (PISA 2018) [Dataset]. https://www.oecd.org/en/data/datasets/pisa-2018-database.html Rodríguez-Hernández, C. F., Cascallar, E., & Kyndt, E. (2020). Socio-economic status and academic performance in higher education: A systematic review. Educational Research Review, 29, 100305. https://doi.org/10.1016/j.edurev.2019.100305 Rudd, G., Meissel, K., & Meyer, F. (2021). Measuring academic resilience in quantitative research: A systematic review of the literature. Educational Research Review, 34, 100402. https://doi.org/10.1016/j.edurev.2021.100402 Rudd, G., Meissel, K., & Meyer, F. (2023). Investigating the measurement of academic resilience in Aotearoa New Zealand using international large-scale assessment data. Educational Assessment, Evaluation and Accountability, 35(2), 169–200. https://doi.org/10.1007/s11092-022-09384-0 Tan, C. Y., Hong, X., Gao, L., & Song, Q. (2023). Meta-analytical insights on school SES effects. Educational Review, 1–29. https://doi.org/10.1080/00131911.2023.2184329 Tudor, K. E., & Spray, C. M. (2017). Approaches to measuring academic resilience: A systematic review. International Journal of Research Studies in Education, 7(4). https://doi.org/10.5861/ijrse.2017.1880 UUnited Nations. (2015). Transforming our world: The 2030 Agenda for Sustainable Development. United Nations. https://sustainabledevelop- ment.un.org/post2015/transformingourworld Ye, W., Strietholt, R., & Blömeke, S. (2021). Academic resilience: Underlying norms and validity of definitions. Educational Assessment, Evaluation and Accountability, 33(1), 169–202. https://doi.org/10.1007/s11092-020-09351-7 Ye, W., Teig, N., & Blömeke, S. (2024). Systematic review of protective factors related to academic resilience in children and adolescents: Unpacking the interplay of operationalization, data, and research method. Frontiers in Psychology, 15, 1405786. https://doi.org/10.3389/fpsyg.2024.1405786