Comparing latent constructs across multiple countries is highly valued in cross-cultural operational research and academic research especially in international large-scale assessments (ILSAs) for different scales/constructs (Svetina et al., 2020). In order for latent constructs to be comparable among countries, participants who are observed to have identical characteristics should obtain identical values (He and Van de Vijver, 2013). This characteristic is called measurement invariance (Vanderberg & Lance, 2000; Kline, 2011).

A widely used method for testing measurement invariance is Multiple-Group Confirmatory Factor Analysis (MG-CFA) (Vanderberg & Lance, 2000). The MG-CFA is also often used to test measurement invariance when different levels of comparability need to be examined, such as configural invariance, metric invariance, and scalar invariance (Kline, 2011). It is important that a construct is scalar level invariant in order to make meaningful comparisons across groups.

In the literature, recently there has been a wide range of studies that test measurement invariance using MG-CFA in the context of international large-scale assessments (ILSAs), and these studies failed to reach scalar level invariance (Eryilmaz et al., 2020; Sandoval-Hernandez et al., 2019; Pedrero and Manzi, 2020). 

There has been recently used a relatively novel approach known as Alignment Optimization (Asparouhov & Muthen, 2014; Muthen & Asparouhov, 2018) to test latent mean comparisons of constructs across countries. The alignment optimization procedure is another method of dealing with a non-invariant scalar measure (Asparouhov & Muthen, 2014; Muthen & Asparouhov, 2018). As highlighted in the above studies, alignment optimization has benefits over Multi-Group Confirmatory Factor Analysis (MG-CFA) since it is almost hard to reach scalar level invariance using MG-CFA. Also, over traditional measurement invariance methods, such as partial invariance, alignment optimization offers an advantage. The partial measurement invariance approach is a bit arduous, especially when there are many groups or many items within each factor in the model since it requires manual adjustments as guided by the modification indexes. Overall, the alignment optimization provides usefulness to make cross-cultural comparisons across countries.

Yet relatively few studies have utilized alignment optimization within the context of International Large-Scale Assessments (ILSAs) (Glassow et al., 2021; Munck et al., 2018; Zakariya et al., 2020). For example, Munck et al. (2018) applied the alignment optimization for the European youth’s attitudes scale using data from data set from the 1999 International Association for the Evaluation of Educational Achievement (IEA) Civic Education Study and the 2009 IEA International Civics and Citizenship Education Study to establish cross-cultural comparisons across 92 groups. In their study of job satisfaction, Zakariya et al. (2020) used the Teaching and Learning International Survey (TALIS 2018) data and used alignment optimization to compare latent means across 48 countries. However, since the usage of alignment optimization is relatively recent, there is very little instructional information that is available for applied cross-cultural researchers who are working with multiple groups especially in the context of ILSAs.

To that end, the main aim of this paper is to ensure an illustration of Alignment Optimization for determining measurement invariance that would be more suitable for conditions with many diverse groups. We illustrate the Alignment Optimization approach to administering measurement invariance using Mplus.

In the following section, we provide a brief background summary of how alignment optimization is defined, with the focus on benefiting from Asparouhov & Muthen (2014) and Muthen & Asparouhov (2018)’s studies for examining measurement invariance. Later, we present a step-by-step tutorial of alignment optimization for researchers and practitioners using Mplus 8.2 (Muthen & Muthen, 1998). Lastly, we provide a concluding remark by discussing the advantages and disadvantages of this method.

It is worth noting that measurement invariance consists of three levels: configural, metric, and scalar. Scalar invariance is needed to compare latent variable means and variances across subgroups (Kline, 2011). However, in many cases, reaching scalar level is often impossible in International Large-Scale Assessments (ILSAs). Furthermore, when comparing models with large samples and groups, the traditional Chi-square difference test should not be used since it is highly sensitive to small changes (Rutkowski and Svetina, 2014). This alignment optimization approach uses an exploratory multiple-group factor analysis to discover the optimal measurement invariance design with the most satisfactory means and variances despite the existence of non-invariances between the groups to compare group mean values (Asparouhov & Muthen, 2014). The alignment optimization process consists of two steps. A configural invariance model is fitted in the first step in which factor loadings and intercepts need to be free across groups, whereas factor means, and factor variances should be fixed at zero and one respectively. Secondly, the factor means, and variances are free, which means we are not forced to assume measurement invariance and are free to estimate the factor mean and variance parameters in each group in order to identify the most optimal measurement invariance pattern. The simplicity function is very similar to the rotation criteria used with exploratory factor analysis (EFA) (Asparouhov & Muthen, 2014). Last but not least, Asparouhov and Muthen (2014, p.499) conclude that "after the alignment estimation is completed, the position of the invariant measurement parameters can be evaluated both visually and by counting the misfit for factor loadings and intercepts". This study uses data from the OECD’s Teaching and Learning International Survey (TALIS) 2018 survey. TALIS is an international survey that collects information from teachers and school principals about working conditions and learning environments in their schools to help countries confront various challenges. The last cycle of TALIS (2018) was administered in 48 countries. In each participating country, approximately 200 schools, and 20 teachers within each school, were selected using a probabilistic sampling technique (OECD, 2019a). Our focus is on “ISCED level 2” (i.e., lower secondary school) since ISCED level 2 includes all countries included at ISCED level 1 and ISCED level 3, as well as additional countries. The final sample size is 9247 principals from 48 countries.

There are some advantages of the alignment optimization method. First, it ensures scalar level invariance to make cross-cultural comparisons across countries. Therefore, in the current study, By employing an alignment optimization method that applies approximate scalar invariance, we are able to compare latent mean principal distributed leadership across participating economies and countries in TALIS 2018. Second, alignment optimization allows the researchers ranking the countries in the analysis. One unique contribution of alignment optimization in this study to knowledge is the ranking of countries and economies by the mean distributed leadership of secondary school principals. Third, alignment optimization provides a framework for comparison between countries which means that we can compare a country to another. There are also some limitations of alignment optimization that are worth mentioning. First, it fails to explain why some certain countries are ranked higher, the other countries are ranked lower. Second, the method is only applicable to one dimension of the construct. Due to this restriction, the alignment approach cannot be used to compare means in multidimensional scales. Thus, this paper will help and guide the researchers to conduct this kind of analysis.

Asparouhov, T., & Muthén, B. (2014). Multiple-group factor analysis alignment. Structural Equation Modeling: A Multidisciplinary Journal, 21(4), 495-508. Eryilmaz, N., Rivera-Gutiérrez, M., & Sandoval-Hernández, A. (2020). Should different countries participating in PISA interpret socioeconomic background in the same way? A measurement invariance approach. Rev. Iberoam. Educ, 84(1), 109-133. Glassow, L. N., Rolfe, V., & Hansen, K. Y. (2021). Assessing the comparability of teacher-related constructs in TIMSS 2015 across 46 education systems: an alignment optimization approach. Educational Assessment, Evaluation and Accountability, 33(1), 105-137. He, J., and F. J. Van de Vijver (2013). “Methodological Issues in Cross-cultural Studies in Educational Psychology.“ In Advancing Cross-cultural Perspectives on Educational Psychology: A Festschrift for Dennis McInerney, edited by Leithwood, K., Sun J., and Pollock, K., 39–56. Kline, R.B., (2011). Principles and Practices of Structural Equation Modelling. 3rd ed. New York: The Guilford Press. Munck, I., Barber, C., & Torney-Purta, J. (2018). Measurement invariance in comparing attitudes toward immigrants among youth across Europe in 1999 and 2009: The alignment method applied to IEA CIVED and ICCS. Sociological Methods & Research, 47(4), 687-728. Muthén, B., & Asparouhov, T. (2018). Recent methods for the study of measurement invariance with many groups: Alignment and random effects. Sociological Methods & Research, 47(4), 637-664. Muthén, L. K., & Muthén, B. O. (1998-2017). Mplus User’s Guide (Eighth ed.). Los Angeles, CA: Muthén & Muthén. Pedrero, V., & Manzi, J. (2020). Self-beliefs, engagement and motivation in science and mathematics: Are they universal?. International Journal of Educational Research, 101, 101562. Rutkowski, L., & Svetina, D. (2014). Assessing the hypothesis of measurement invariance in the context of large-scale international surveys. Educational and Psychological Measurement, 74(1), 31-57. Sandoval-Hernández, A., David, R., Matta, T., & Miranda, D. (2019). Back to the drawing board: Can we compare socioeconomic background scales?. Revista de Educacion, 2018(383), 37-61. Svetina, D., Rutkowski, L., & Rutkowski, D. (2020). Multiple-group invariance with categorical outcomes using updated guidelines: an illustration using Mplus and the lavaan/semtools packages. Structural Equation Modeling: A Multidisciplinary Journal, 27(1), 111-130. Vanderberg, R. J., & Lance, C. E. (2000). A Review and Synthesis of the Measurement Invariance Literature: Suggestions Practices, and Recommendations for Organizational Research. Organizational Research Methods, 3(1), 4-70. Zakariya, Y. F., Bjørkestøl, K., & Nilsen, H. K. (2020). Teacher job satisfaction across 38 countries and economies: An alignment optimization approach to a cross-cultural mean comparison. International Journal of Educational Research, 101, 101573.