Abstract

The work presents the results of a study of the psychometric quality of an instrument for measuring professional well-being of Russian teachers (2014 respondents). Based on exploratory structural equation modeling (ESEM) and confirmatory factor analysis (CFA) methodology of the data obtained, it was revealed that the theoretical factor structure of the instrument, which includes four correlated factors, is not fully confirmed, but the data show that the instrument functions well within revised more complex structure (CFI = 0.998; TLI = 0.993; RMSEA = 0.012). The results demonstrate that teachers' professional well-being is a complex construct which includes the general factor of professional well-being and specific factors related to different aspects of professional attitudes such as attitudes toward duties and responsibility of a teacher.  The methodology of the analysis as well as the fact that our policy decisions on the construction of monitoring studies are faced with the reality of the data we receive will be discussed

Introduction

Teachers' professional well-being is an important subject of research around the world (Brouwers & Tomic, 2000; OECD, 2005; Pines & Aronson, 1993; Wood, 2002). Professional well-being can vary from conditionally positive to conditionally negative values ​​(what can be called "burnout") (Dworkin, Saha, & Hill, 2003). Some relate teachers’ professional well-being to their motivation, pedagogical attitudes, productivity and academic performance of students (as a result), their emotional well-being and involvement, social mobility etc.(Klusmann, Kunter, Trautwein, Lüdtke, & Baumert, 2008; OECD, 2016) .

The structure of professional well-being has been studied by a variety of researchers for a long time (Dworkin, 2009; Maslach, Rutgers, & Leiter, 1997). The motivation and professional well-being of teachers, satisfaction with the professional and social situation is the subject of monitoring by leading Russian research centers. However, in Russian state monitoring studies a thorough analysis of the construct validity of the used questionnaires is rare thing.

In our study, we look at professional well-being / burnout among school teachers. Burnout is understood as a loss of enthusiasm, emotional devastation, loss of ideals and disappointment in moral values ​​(Cherniss, 1992; Maslach, Rutgers, Leiter, 1997; Matheny, Gfroerer, Harris, 2000). Described monitoring suggested a certain theoretical structure of the burnout construct (attitudes toward students; relationship with administration; attitudes toward professional duties; professional identity), which is not confirmed in our analysis. However, the analysis shows that the instrument can be called good if we consider its factor structure differently.

The purpose of the present study was to examine the validity of the teachers’ professional well-being questionnaire applied in monitoring research. More precisely, we are going to focus on the structural validity and investigate the internal structure of the questionnaire in detail. According to the initial framework, the questionnaire was considered as inherently multidimensional and included four correlated factors.  Toward this end, exploratory structural equation modeling (ESEM) and its extensions (bi-factor and second-order models) was applied as a flexible framework for modelling complex relationship among partially overlapping constructs. ESEM integrates features of confirmatory and exploratory factor analysis in a way that being less restricted,  ESEM allow to estimate theoretically expected factor loadings and additional cross-loadings extracted from the data simultaneously (Morin, Arens, & Marsh, 2016). The benefit of ESEM compared to CFA lies in the fact that due to inclusion cross-loadings into the model factor correlation estimates are not inflated and discriminant validity of the factors can be estimated more precisely (Marsh, Morin, Parker, & Kaur, 2014).

Method The information base of the study was one of the Russian large monitoring studies, which covers 8 federal districts of Russia, and is supposed to be representative at the state level. The sample was recruited by the two-step stratification method. The monitoring sample included 2014 teachers of Russian schools who filled out paper questionnaires. Among other things, the monitoring questionnaire includes our instrument for measuring professional well-being. The instrument consisted of 12 items which were rated on 5-pont Likert-type scale. Preliminary analysis demonstrated that there were significant departures from normality for two items (skewness were -1,81 and -1,21; kurtosis = 3.16 and 2,18, respectively). Other items demonstrated mild departures from normality. All items demonstrated fairly low difficulty values (M = 3.69; SD= 1,03). The robust maximum likelihood estimator (MLR) was used due to its robustness to the non-normality of the data and suitability for 5-points scale (Rhemtulla et al., 2012). Firstly, the original factor structure was verified by conducting multidimensional confirmatory factor analysis (CFA). Secondly, the analysis of the factor structure was carried out by ESEM based on geomin oblique rotation which makes ESEM conceptually closer to EFA (Asparouhov & Muthén, 2009). Thirdly, ESEM bi-factor model and second-order models (ESEM-within-CFA) were applied. Bi-factor model allows to estimate the general factor and the uncorrelated specific factors. The general factor reflects the variance shared by all items and the specific factors reflect the variance of the items that not explained by the general factor (Morin, Arens, & Marsh, 2016). Second-order model indicates whether first-order factors related to a single higher-order factor (Morin & Asparouhov, 2018). Based on target rotation as the most appropriate for these models, items were loaded on their respective factor, whereas cross-loadings were targeted (but not fixed) to be close to zero. Following Brown's work (2006) we have chosen the following critical values to assess goodness of fit: CFI (> 0.9), TLI (> 0.9) and RMSEA (< 0.6).

Results The first step of the analysis showed that the originally expected structure fits the data poorly (ꭓ2 (48) = 795.33, p < 0.00; CFI = 0.751; TLI = 0.656; RMSEA = 0.088). On the second step, based on the result of ESEM new four-factor structure with correlated factors was obtained (ꭓ2 (24) = 109.82, p < 0.00; CFI = 0.976; TLI = 0.934; RMSEA = 0.042). The distribution of items per factors differed from the original structure. On the next step the second-order and bi-factor models were contrasted. Second-order model demonstrated slightly worse fit than the model with four correlated factors (ꭓ2 (26) = 111.79, p < 0.00; CFI = 0.969; TLI = 0.922; RMSEA = 0.04). Bi-factor model demonstrated significantly better fit (ꭓ2 (16) = 21.008, p < 0.00; CFI = 0.998; TLI = 0.993; RMSEA = 0.012). All items significantly loaded on the general factor (factor loadings varied from 0.27 to 0.54), whereas, the items of the third factor insignificantly loaded on their specific factor. We believe that the value of our work lies in the following important points: • Teachers’ professional well-being is an important characteristic with a complex structure. The structure includes the general factor, which reflects professional attitudes and emotional characteristic (enthusiasm and aspiration), and specific factors related to professional attitudes toward formal attitude to work; the responsibility for actions as a teacher, formal and non-formal professional duties. More research related to the investigation of internal structure of this construct is need. • Educational policy makers responsible for monitoring research need to consider the results of research tools to improve the quality of the resulting assessments • Improved assessment can help us to prepare better interventions and adjust the country's educational policy.

Brouwers, A., & Tomic, W. (2000). A longitudinal study of teacher burnout and perceived self-efficacy in classroom management. Teaching and Teacher Education, 16(2), 239–253. https://doi.org/10.1016/S0742-051X(99)00057-8 Brown, T. A. (2006). Confirmatory Factor Analysis for Applied Research. Guilford Press. Извлечено от https://books.google.co.in/books?id=KZwDkH2G2PMC Cherniss, C. (1992). Long‐term consequences of burnout: An exploratory study. Journal of Organizational Behavior, 13(1), 1–11. https://doi.org/10.1002/job.4030130102 Dworkin, A. G. (2009). Teacher Burnout and Teacher Resilience: Assessing the Impacts of the School Accountability Movement. In International Handbook of Research on Teachers and Teaching (pp. 491–502). Boston, MA: Springer US. https://doi.org/10.1007/978-0-387-73317-3_32 Dworkin, Saha, L. J., & Hill, A. N. (2003). Teacher burnout and perceptions of a democratic school environment. International Education Journal. Klusmann, U., Kunter, M., Trautwein, U., Lüdtke, O., & Baumert, J. (2008). Teachers’ occupational well-being and quality of instruction: The important role of self-regulatory patterns. Journal of Educational Psychology, 100(3), 702–715. https://doi.org/10.1037/0022-0663.100.3.702 Marsh, H. W., Morin, A. J. S., Parker, P. D., & Kaur, G. (2014). Exploratory Structural Equation Modeling: An Integration of the Best Features of Exploratory and Confirmatory Factor Analysis. Annual Review of Clinical Psychology, 10(1), 85–110. https://doi.org/10.1146/annurev-clinpsy-032813-153700 Maslach, C., Rutgers, S. E. J., & Leiter, M. (1997). The Maslach Burnout Inventory Manual Maslach Burnout Inventory Manual, 4th Edition View project. Retrieved from https://www.researchgate.net/publication/277816643 Matheny, K.B., Gfroerer, C.A., & Harris,K. (2000). Work Stress, Burnout, and Coping at the Turn of the Century: An Individual Psychology Perspective. Journal of Individual Psychology. McKenzie, P., Santiago, P., & OECD. (2005). Teachers matter : attracting, developing and retaining effective teachers. Organisation for Economic Co-operation and Development. Retrieved from http://www.oecd.org/education/school/attractingdevelopingandretainingeffectiveteachers-finalreportteachersmatter.htm Morin, A. J. S., Arens, A. K., & Marsh, H. W. (2016). A Bifactor Exploratory Structural Equation Modeling Framework for the Identification of Distinct Sources of Construct-Relevant Psychometric Morin, A.J.S., & Asparouhov, T. (2018). Estimation of a hierarchical Exploratory Structural Equation Model (ESEM) using ESEM-within-CFA. Structural Equation Modeling: A Multidisciplinary Journal, 23(1), 116–139. https://doi.org/10.1080/10705511.2014.961800 OECD. (2016). Supporting Teacher Professionalism. OECD Publishing. https://doi.org/10.1787/9789264248601-en Pines,A., & Aronson,E. (1993). Career burnout: causes and cures, in professional burnout: recent developments. In Theory and Research (p. 9). Wood, T.-M.C. (2002). Understanding and Preventing Teacher Burnout. ERIC Digest. Retrieved from https://www.ericdigests.org/2004-1/burnout.htm