Session Information
09 SES 12 C, Psychometric Modelling, Adaptive Testing and Item Complexity
Paper Session
Contribution
Confidence in science is a central affective construct that has been consistently linked to students’ engagement, persistence, and achievement in science learning (Berger et al., 2025; Lee & Stankov, 2018; Wu & Wu, 2022). Students with higher confidence in science are more likely to participate actively in learning activities (OECD, 2019) and to pursue further studies in science-related fields (Wang & Degol, 2013). Recently released results from TIMSS 2023 provide further evidence that eighth-grade students who report being very confident in science demonstrate substantially higher average science achievement (von Davier et al., 2024). Consequently, understanding how confidence in science is conceptualised and distributed among students is of critical importance for educational research.
In the TIMSS 2023 Assessment Frameworks (Reynolds et al., 2021), confidence in science is conceptualised under the students’ attitudes toward science. It is commonly measured using self-report Likert-type scales. The scale scores are reported using three categorical levels (e.g., not confident, somewhat confident, and very confident) defined through cut scores that are determined using Latent Class Analysis (LCA; Lazarsfeld & Henry, 1968). While the LCA-based approach adopted in TIMSS offers a data-driven method for identifying qualitatively distinct groups of students, it conceptualises confidence in science, assuming measurement homogeneity within each identified group. This assumption may be limiting in large-scale assessments, where latent subgroups may differ in overall levels of confidence and in how the construct is structured and measured.
Factor Mixture Modeling (FMM) provides an integrative framework that combines confirmatory factor analysis (CFA) and latent class analysis (LCA), allowing researchers to simultaneously examine categorical heterogeneity and within-class variation (Muthén, 2006). Importantly, FMM provides a flexible approach to investigate potential differences in the latent construct and measurement properties of confidence in science across unobserved subgroups, providing a means to investigate measurement equivalence.
The present study focuses on a comparison between Türkiye and England, two countries that differ substantially in their educational systems, instructional traditions, and sociocultural contexts (OECD, 2023). According to TIMSS 2023 results, the average science achievement scores for Grade 8 are highly similar in the two countries (von Davier et al., 2024), making this comparison potentially informative for examining whether comparable performance levels reflect comparable latent constructs. Using FMM, this study examines whether the same number of latent classes emerges in both countries and whether the measurement structure of confidence in science operates equivalently across these latent classes. Addressing these issues is essential for evaluating measurement equivalence and for informing the comparability of the scale scores across countries.
Accordingly, the study addresses the following research questions:
- Is the confidence in science scale unidimensional?
- Are there distinct latent classes in students’ confidence in science?
- Is the measurement model invariant across latent classes within the FMM framework?
- To what extent do latent class structures and measurement properties vary across countries?
Method
To address the research questions, data were retrieved from the TIMSS 2023 international database and drawn from the Grade 8 student context questionnaire administered in Türkiye and England. The Turkish sample comprised 4,925 students initially, with 4,508 retained after data cleaning. The sample comprised of 50.4% girls (n = 2271), and 49.6% boys (n = 2237). The mean age of students was 13.87 years (SD = 0.41). The English sample comprised 4,239 students initially, with 4,190 retained after data cleaning. The sample comprised of 51.4% girls (n = 2155), and 48.6% boys (n = 2035). The mean age of students was 14.04 years (SD = 0.30). The TIMSS 2023 Students Confident in Science scale captures students’ self-perceptions of their ability to learn and succeed in science, which are informed by prior experiences and comparative self-evaluations (Reynolds et al., 2021). These items were scored on a 4-point Likert-type scale, ranging from 1 = agree a lot to 4 = disagree a lot. At this stage, preliminary analyses and FMMs were first conducted for the Turkish sample to inform model specification. Corresponding analyses are planned for the English sample. The analytical procedure followed a stepwise modeling strategy (Clark et al., 2013). Baseline models were evaluated using CFA to assess unidimensionality and LCA to identify latent classes. In the most restrictive specification (FMM-1), factor loadings and item thresholds were constrained to be invariant across latent classes. Subsequent models (FMM-2 to FMM-4) progressively relaxed these constraints by allowing class-specific factor variances, thresholds, and loadings, with factor means fixed for model identification. Based on these preliminary findings, a sequence of increasingly flexible FMMs will subsequently be evaluated within a multigroup framework. Measurement invariance across latent classes will be examined by treating country as a known grouping variable, provided that an acceptable baseline model is established in each country. Acceptable model fit for CFA was indicated by RMSEA and SRMR values below 0.08, together with CFI and TLI values exceeding 0.90 (Hu & Bentler, 1999). For LCA and FMM, model selection was informed by the BLRT, LMR, and VLMR tests, where significant p-values indicate improved fit for models with additional classes (Nylund et al., 2007). Information criteria (AIC, the BIC, and the SABIC) were also examined, with lower values indicating better relative fit. Classification quality was evaluated using entropy, with values closer to 1 indicating greater classification precision.
Expected Outcomes
At this stage, conclusions are based on the Turkish sample; analyses involving the English sample and multigroup FMM will be undertaken in a subsequent phase of the study. In the Turkish sample, one- and two-factor models was evaluated to examine factor structure. CFA results indicated that both models demonstrated acceptable fit; however, the one-factor model was preferred due to its more parsimonious structure and comparable fit indices (CFI = .99, TLI = .99, RMSEA = .064, SRMR = .016). LCA findings indicated that a three-class solution provides a reasonable representation of the underlying heterogeneity. Preliminary FMM results indicate systematic improvements in model fit as measurement constraints were progressively relaxed. In both the two- and three-class solutions, more flexible specifications allowing measurement noninvariance (FMM-3 and FMM-4) yielded lower information criteria and higher classification quality. This pattern suggests that the observed differences between classes are not solely attributable to differences in the underlying factor (Lubke & Muthén, 2005), but may also reflect class-specific measurement characteristics. Accordingly, differences between the latent classes may reflect qualitative distinctions rather than purely quantitative differences. These findings highlight the potential utility of FMM for examining class-specific measurement structures within heterogeneous samples.
References
Berger, N., Fitzmaurice, O., Ryan, V., Mackenzie, E., & Holmes, K. (2025). Secondary students’ attitudes towards mathematics and science in Ireland: a latent profile analysis of TIMSS 2019 using situated expectancy-value theory. Irish Educational Studies, 1-23. https://doi.org/10.1080/03323315.2025.2549098 Clark, S. L., Muthén, B., Kaprio, J., D'Onofrio, B. M., Viken, R., & Rose, R. J. (2013). models and strategies for factor mixture analysis: an example concerning the structure underlying psychological disorders. Structural Equation Modeling: A Multidisciplinary Journal, 20(4), 681-703. http://dx.doi.org/10.1080/10705511.2013.824786 Lazarsfeld, P. F., & Henry, N. W. (1968). Latent structure analysis. New York: Houghton Mifflin. Lee, J., & Stankov, L. (2018). Non-cognitive predictors of academic achievement: Evidence from TIMSS and PISA. Learning and Individual Differences, 65, 50-64. https://doi.org/10.1016/j.lindif.2018.05.009 Lubke, G. H., & Muthén, B. (2005). Investigating population heterogeneity with factor mixture models. Psychological Methods, 10(1), 21-39. https://doi.org/10.1037/1082-989X.10.1.21 Muthén, B. (2006). Should substance use disorders be considered as categorical or dimensional? . Addiction, 101(1), 6-16. https://doi.org/10.1111/j.1360-0443.2006.01583.x Nylund, K. L., Asparouhov, T., & Muthén, B. O. (2007). Deciding on the number of classes in latent class analysis and growth mixture modeling: a monte carlo simulation study. Structural Equation Modeling: A Multidisciplinary Journal, 14(4), 535-569. https:// doi.org/ 10. 1080/ 10705 51070 1575396 OECD. (2019). PISA 2018 Results (Volume II): Where All Students Can Succeed. Paris: OECD Publishing. https://doi.org/10.1787/b5fd1b8f-en OECD (2023). Education at a Glance 2023: OECD Indicators. Paris: OECD Publishing. https://doi.org/10.1787/e13bef63-en. Reynolds, K. A., Mullis, I. V., & Martin, M. O. (2021). Chapter 3: TIMSS 2023 Context Questionnaire Framework. In I. V. Mullis, M. O. Martin, & M. von Davier, TIMSS 2023 Assessment Frameworks (s. 46-71). TIMSS & PIRLS International Study Center: Boston College. https://doi.org/10.6017/lse.tpisc.timss.rs6460 von Davier, M., Kennedy, A., Reynolds, K., Fishbein, B., Khorramdel, L., Aldrich, C., . . . Yin, L. (2024). TIMSS 2023 International Results in Mathematics and Science. Boston College: TIMSS & PIRLS International Study Center. https://doi.org/10.6017/lse.tpisc.timss.rs6460 Wang, M. T., & Degol, J. (2013). Motivational pathways to STEM career choices: Using expectancy–value perspective to understand individual and gender differences in STEM fields. Developmental Review, 33(4), 304-340. https://doi.org/10.1016/j.dr.2013.08.001 Wu, X., & Wu, Q. (2022). Can students be more engaged and confident? A multiple membership multilevel analysis of science engagement and confidence and their effects on science achievement in East Asia. Studies in Educational Evaluation, 73(101147). https://doi.org/10.1016/j.stueduc.2022.101147
Update Modus of this Database
The current conference programme can be browsed in the conference management system (conftool) and, closer to the conference, in the conference app.
This database will be updated with the conference data after ECER.
Search the ECER Programme
- Search for keywords and phrases in "Text Search"
- Restrict in which part of the abstracts to search in "Where to search"
- Search for authors and in the respective field.
- For planning your conference attendance, please use the conference app, which will be issued some weeks before the conference and the conference agenda provided in conftool.
- If you are a session chair, best look up your chairing duties in the conference system (Conftool) or the app.