A large amount of research has shown that individuals’ beliefs in their ability have a major impact on their actions in different situations (Bandura, 1986; Marsh et al., 2019; Usher & Pajares, 2006). Students with different self-beliefs demonstrate different level of cognitive engagement as well as well-being in school (Bong & Skaalvik, 2003), which in turn, play a vital role in their academic achievements. Self-belief of competence is a broad conception that involves several facets and constructs, among which, self-efficacy and self-concept have been given considerable attention in their prediction of academic performance (see e.g. Bong & Skaalvik, 2003; Marsh, 1987; Shavelson et al., 1976).

Self-efficacy, built through previous experiences, refers to one’s perception of and belief in the capability of oneself. In the social cognitive theory (Bandura, 1986), self-efficacy captures the belief in one’s competence on a specific task or process. Mathematics self-efficacy in Programme for International Student Assessment (PISA), for example, measures student’s expectation and conviction of what can be accomplished when they need to solve pure and/or applied mathematics tasks. The students are asked to report on their perceived ability by responses of the extent to which they feel confident (OECD, 2013).

Self-concept, on the other hand, is defined as ones’ general perception of themselves, which is established through experiences, interpretations of social atmosphere, and with reference with their peers (Rosenberg, 1979; Marsh, 1987; Parker et al., 2014). Unlike self-efficacy, self-concept captures the perceived general competence of a subject matter that is not linked to any specific tasks in that subject. Mathematics self-concept in PISA is a constructed index based on students’ degree of agreement about their perceived competence in mathematics (OECD, 2013).

In PISA studies, self-reported questionnaire data is used to assess student mathematics self-efficacy and self-concept across different education systems and measurement cycles. The comparability of these constructs is thus essential for the validity of the inferences drawn from any cross-nation and cross-cycle analyses involving these constructs. Invariance is a statistical property of measurement, indicating the measured construct being equivalence across groups and/or over time (Vandenberg and Lance, 2000). Therefore, studies aiming to make cross-national/cultural comparisons or comparisons over time, measurement invariance has to be reached to make such comparisons appropriate and proper.

The aim of the present study was to assess the measurement invariance of mathematics self-efficacy and self-concept employed in the PISA 2003 and 2012 cycles, and across 40 countries and economies (by cohort by country). The study was intended to identify patterns of non-invariance in the measurement of student mathematics self-efficacy and self-concept related to participant’s country and cohort, and assess whether these two constructs can be comparably measured across groups.

The study considered 40 countries and economies participated in both the PISA 2003 and 2012 cycles, giving a total sample size 605,564 respondents. Eight items were used to assess student mathematics self-efficacy (MSE). The students were asked to report on their perceived ability and whether they feel very confident, confident, not very confident or not at all confident (a 4-point Likert type) on pure and applied mathematical tasks such as “using a train timetable”, “calculating TV discount”, and “calculating square metres of tiles”. Mathematics self-concept (MSC) was measured by five items, student were supposed to report whether they strongly agree, agree, disagree or strongly disagree (a 4-point Likert type) with the statements towards e.g. “get good grades”, “learn quickly”, and “not good at math” (OECD, 2013). The alignment method proposed by Muthén and Asparouhov (2013) was applied to assess measurement invariance and compare factor means and factor variances across groups in the current study. The advantage of alignment method is that it minimises the amount of measurement non-invariance by estimating the factor means and factor variance (Muthén and Asparouhov, 2013; Asparouhov and Muthén, 2014). As a first step in the alignment method, it is important to establish good-fit configural models for MSE and MSC by using multi-group confirmatory factor analysis (MGCFA, Muthén and Asparouhov, 2013). Then, the alignment optimisation method was used to identify the potential best-fitting solution for the data while assuming more practical partial measurement invariance. In this step, the baseline models of mathematics self-efficacy and self-concept were applied in an alignment scaling procedure. In this study, a 40-group alignment analysis of the two constructs MSE and MSC was performed for the 40 countries and economies in each of the two PISA cycles 2003 and 2012, followed by an 80-group alignment analysis of the two cycles jointly. The joint analysis makes it possible to compare factor means and factor variances across countries, as well as across the two cycles (Muthén and Asparouhov, 2014). Last, Monte Carlo simulation was conducted to check how well alignment method works under various conditions where groups, group sample size, and degree of measurement non-invariance were different. The simulation study was conducted for group sizes, N = 500, N = 1,000 and N = 10,000, the latter being close to the real-data group size, and each with 100 replications. SPSS 25 and Mplus 8 (Muthén & Muthén, 1998-2019) were used for data management and analysis.

The measurement models of mathematics self-efficacy and self-concept fitted the data fairly well, across 40 groups in the separate cycles, as well as across 80 groups (by cohort by country) in the joint analysis. Results from the multiple group analysis indicated that MSE and MSC held metric invariance. The alignment analysis however showed that the proportion of non-invariance was rather high. In the joint analysis, the proportion of non-invariance of MSE and MSC was found to have 42.5% and 70.75% across 80 groups. The simulation study results demonstrated that the aligned models of MSE and MSC were well fitted to the data of 40 countries and economies participated in both PISA 2003 and 2012 cycles, despite the existence of large proportion of non-invariance. Biemer and Lyberg (2003) argued that the comparability of survey quality across space and over time was a crucial consideration for cross-national studies. It is necessary to address measurement invariance issues due to culture or other country differences in the large-scales assessment studies. By utilising conventional measurement invariance testing as well as the alignment method, this study provided the evidence that a large degree of non-invariance existed in student mathematics self-efficacy and self-concept across countries and over time. The outcomes have methodological significance, given their potential to improve the quality of MSE and MSC scales in PISA. Findings about non-invariance may have substantive significance for identifying the issues that need to be addressed for the comparability among various countries and culture, and over time. The ordering and patterns of factor means for the two constructs could contribute to further comparative studies, relating to student mathematics achievement for instance.

Asparouhov, T., & Muthén, B. (2014). Multiple-group factor analysis alignment. Structural Equation Modeling: A Multidisciplinary Journal, 21(4), 495-508. Bandura, A. (1986). Social foundations of thought and action. Englewood Cliffs, NJ, 1986. Biemer, P. P., & Lyberg, L. E. (2003). Introduction to survey quality (Vol. 335): John Wiley & Sons. Bong, M., & Skaalvik, E. M. (2003). Academic self-concept and self-efficacy: How different are they really? Educational psychology review, 15(1), 1-40. Lee, J. (2009). Universals and specifics of math self-concept, math self-efficacy, and math anxiety across 41 PISA 2003 participating countries. Learning and Individual Differences, 19(3), 355-365. doi:10.1016/j.lindif.2008.10.009 Marsh, H. W. (1987). The big-fish-little-pond effect on academic self-concept. Journal of educational psychology, 79(3), 280. Marsh, H. W., Pekrun, R., Parker, P. D., Murayama, K., Guo, J., Dicke, T., & Arens, A. K. (2019). The murky distinction between self-concept and self-efficacy: Beware of lurking jingle-jangle fallacies. Journal of educational psychology, 111(2), 331. Muthen, B., & Asparouhov, T. (2013). New methods for the study of measurement invariance with many groups. Mplus (www. statmodel. com). Muthén, B., & Asparouhov, T. (2014). IRT studies of many groups: the alignment method. Frontiers in Psychology, 5, 978. Muthén, L. K., & Muthén, B. O. (1998). Mplus user’s guide (1998-2019, Version 8). Los Angeles, CA: Author. OECD (2013), PISA 2012 Results: Ready to Learn: Students’ Engagement, Drive and Self-Beliefs (Volume III), PISA, OECD Publishing. http://dx.doi.org/10.1787/9789264201170-en Parker, P. D., Marsh, H. W., Ciarrochi, J., Marshall, S., & Abduljabbar, A. S. (2014). Juxtaposing math self-efficacy and self-concept as predictors of long-term achievement outcomes. Educational Psychology, 34(1), 29-48. Rosenberg, M. (1986). Conceiving the self: RE Krieger. Shavelson, R. J., Hubner, J. J., & Stanton, G. C. (1976). Self-concept: Validation of construct interpretations. Review of educational research, 46(3), 407-441. Usher, E. L., & Pajares, F. (2006). Sources of academic and self-regulatory efficacy beliefs of entering middle school students. Contemporary educational psychology, 31(2), 125-141. Vandenberg, R. J., & Lance, C. E. (2000). A review and synthesis of the measurement invariance literature: Suggestions, practices, and recommendations for organizational research. Organizational research methods, 3(1), 4-70.