School leaders’ concerns about a comprehensive school reform in Germany: Reevaluation of the Stages of Concern Questionnaire
Author(s):
Susanne Boese (presenting / submitting) Marko Neumann Michael Becker Kai Maaz
Conference:
ECER 2016
Network:
26. Educational Leadership
Format:
Paper

Session Information

26 SES 02 A, Leadership and System Reform

Paper Session

Time:
2016-08-23
15:15-16:45
Room:
OB-H1.51
Chair:
Ulf Leo

Contribution

Objective

The study presented here drew on the Stages of Concern Questionnaire (SoCQ) which presents a stage-wise assessment of individual actors’ degrees of concerns regarding an innovation. This instrument has been used in the context of different studies. Regardless of its frequent implementation in innovation and reform processes, the concept and psychometric characteristics of the SoCQ have been a matter of critical debate. This paper therefore aims to assess the psychometrical validity of the Stages of Concern approach and  find out if and how the instrument can be applied to assessing the implementation of school reforms in the context of a school leader survey.

 

Theoretical framework

The success of implementing school reforms is linked to its coupling of individual school and system levels. Thus, an essential role is attributed to school leaders (Desimone, 2002). Owing to his or her particular function, the school leader acts as a mediator operating between educational administration and individual school (Bonsen, 2010). School leaders are capable of preparing the (organizational) ground for a successful implementation, for instance by determining common long-term goals or structures, processes and procedures for the validation of outcomes (Fullan, 2007). Therefore, school leaders can be regarded as central actors in the support of reforms, and their attitudes may affect teachers, the student population, parents and even the school environment (Hallinger, 2003). Empirical research has demonstrated that school leaders have a strong influence on individual goals and professional convictions of teachers. They can provide teachers with incentives to change and inspire them with a clear vision for change (Geijsel, Sleegers, Leithwood, & Jantzi, 2003). Therefore it is important to investigate how school leaders perceive a reform and how they deal with it.

A model focusing on how individual persons deal with innovation in the field of education is the Concerns-Based Adoption Model (CBAM, Hall & Hord, 2011). The main component of the CBAM consists of the Stages of Concern and the relevant instrument, i.e. the SoCQ. This instrument concentrates on actors involved in a reform process, describing their stage-wise dealing with innovation in the course of a change process (Hord & Huling-Austin, 1986). In the past, the SoCQ has been implemented in various longitudinal and cross-sectional studies and it has been established as a viable empirical instrument (e.g. Bitan-Friedlander, Dreyfus, & Milgrom, 2004; Pant, Vock, Pöhlmann, & Köller, 2008) even if it was not always possible to replicate the theoretically assumed model entirely (e.g. Bailey & Palsha, 1992).

The aim of our study is to assess whether the theoretically postulated seven Stages of Concern can be empirically replicated for a CSR in Berlin. Based on findings from other studies which also applied the SoCQto assessing a complex (curricular) reform without confirming all stages (e.g. Cheung, Hattie, & Ng, 2001), the following analyses are also expected to reveal that it is not necessarily possible to model all stages in line with theoretical postulations. As already demonstrated by Hall, George, and Rutherford (1977), stronger connections may emerge between adjacent stages. This approach can also be referred to as a simplex structure (Guttman, 1954). Based on earlier findings (e.g. Cheung et al., 2001), this study aims to contribute to the current knowledge about the simplex structure of the Stages of Concern approach in the context of a CSR and a school leader survey.

Method

Methods We used exploratory factor analyses (EFA) to gain a first heuristic insight into the data structure. In a next step, different models were tested in confirmatory factor analyses (CFA) regarding data fit, using Mplus 7. To investigate the simplex structure of the SoC, we used the computer program CIRCUML, which tested the simplex structure with a factor analytic approach. For the purpose of including all school leaders in the analysis and to avoid a possible distortion of findings owing to an exclusion of missing data, analyses with Mplus drew on the FIML option (Full-Information Maximum Likelihood). The maximum likelihood estimator was used with robust standard errors (MLR), which is robust against violations of the normal distribution assumption. Data sources Sample This study is based on a scientific monitoring of a CSR in Berlin, which became effective in the academic year 2010/2011. The main concern of this CSR was that the secondary school system changed from a four-track system to a two-track system. The study pursues a quasi-experimental design involving several cohorts. In this context, school leaders of nearly all public secondary schools in Berlin were surveyed. A total number of N = 196 school leaders from N = 211 secondary schools participated (return rate: 92.9%). School leaders were questioned approximately a year after imple-mentation of the school reform. Instruments The presented analyses are based on the SoCQ by Hall and Hord (2011) in an adapted version (seven subscales with a total of 31 items), drawing on the translation by Pant et al. (2008). Adaptations resulted in a slight modification of items taking the situa-tion of the CSR in Berlin into account as well as the situation of school leaders. Moreo-ver, four items were excluded a priori because they were irrelevant in the context. All seven Stages of Concern-subscales were assessed by four to five 7-point Likert-type items ranging from 1 = currently does not apply to 7 = currently fully applies showing acceptable to good internal consistencies (α = .70 - .89). Only the subscale at Stage 1, i.e. Informational, displays a very low level of internal consistency (α = .48).

Expected Outcomes

Results The EFAs revealed that a 6- to 8-factorial solution showed the best absolute and relative model fit values. To assess the theoretically intended 7-factor structure of the SoC, confirmatory factor analyses were conducted. The several fit indices did, however, indicate a poor model fit. Therefore, inter-scale correlations were inspected for the individual subscales. A remarkably high factor correlation of r = 0.90 was revealed between subscales Infor-mational and Collaboration, indicating that these aspects are not empirically separable. Therefore, both subscales were comprised in a single scale. Factor loadings of the individual items were then tested. Here, only the subscale of Awareness could be adopted from the original. Regarding all other subscales, a total number of eight items was excluded from estimation owing to low factor loadings. Looking at the fit indices for the reduced 6-factor model, they revealed accepta-ble to good scores. Even though the 6-factor model fits the data best, the simplex corre-lation matrix (RMSEA = .45) indicated that the hypothesized simplex structure of the SoC was not empirically supported. Discussion Our results did not confirm the assumption of the SoC as a developmental mod-el. The pattern of findings can possibly be explained by the background of this study. The structural decision for the CSR as such was initiated by school administration in-stead of the school leaders themselves. Hence, school leaders are not free to decide on whether they will actually implement the reform as they are immediately facing it. In particular, this is evident from the difficulty of replicating the conceptual phases. The question emerges in how far the theoretical concept of SoC might be apt for specific types of innovation respectively implementation strategies and if it should generally be modified to specific purposes of application.

References

Bailey, D. B., & Palsha, S. A. (1992). Qualities of the Stages of Concern Questionnaire and Implications for Educational Innovations. The Journal of Educational Research, 85(4), 226–232. Bitan-Friedlander, N., Dreyfus, A., & Milgrom, Z. (2004). Types of "teachers in train-ing": the reactions of primary school science teachers when confronted with the task of implementing an innovation. Teaching and Teacher Education, 20(6), 607–619. Bonsen, M. (2010). Schulleitungshandeln [School leadership actions]. In H. Altrichter & K. Maag Merki (Eds.), Handbuch Neue Steuerung im Schulsystem. (pp. 277–294). Wiesbaden: VS Verlag für Sozialwissenschaften. Cheung, D., Hattie, J., & Ng, D. (2001). Reexamining the Stages of Concern Question-naire: A Test of Alternative Models. The Journal of Educational Research, 94(4), 226–236. Desimone, L. (2002). How can Comprehensive School Reform Models Be Succesfullly Implemented? Review of Educational Research, 72(3), 433–479. doi:10.3102/00346543072003433 Fullan, M. (2007). The New Meaning of Educational Change (4th ed.). London and New York: Routledge Taylor & Francis Group. Geijsel, F., Sleegers, P., Leithwood, K., & Jantzi, D. (2003). Transformational leadership effects on teachers’ commitment and effort toward school reform. Journal of Educational Administration, 41(3), 228–256. doi:10.1108/09578230310474403 Guttman, L. (1954). A new approach to factor analysis: the Radex. In P. F. Lazarsfeld (Ed.), Mathematical thinking in the social sciences (pp. 258–348). New York: Free press. Hall, G. E., George, A. A., & Rutherford, W. L. (1977). Measuring Stages of Concern about the Innovation: A Manual for the Use of the SOC Questionnaire. Austin: The Research and Development Center for Teacher Education. Hall, G. E., & Hord, S. M. (2011). Implementig change.: Patterns, Principles and Pot-holes. (3rd ed.). Boston: Pearson Education, Inc. Hallinger, P. (2003). Leading Educational Change: reflections on the practice of instruc-tional and transformational leadership. Cambridge Journal of Education, 33(3), 329–352. doi:10.1080/0305764032000122005 Hord, S. M., & Huling-Austin, L. (1986). Effective Curriculum Implementation - Some Promising New Insights. Elementary School Journal, 87(1), 96–115. doi:10.1086/461482 Pant, H. A., Vock, M., Pöhlmann, C., & Köller, O. (2008). Offenheit für Innovationen. Befunde aus einer Studie zur Rezeption der Bildungsstandards bei Lehrkräften und Zusammenhänge mit Schülerleistungen [Open for innovations. Findings from a study on the reception of educational standards in teachers and correlations with student achievement]. Zeitschrift für Pädagogik, 54(6), 827–845.

Author Information

Susanne Boese (presenting / submitting)
German Institute for International Educational Research
Department for Educational Governance
Berlin
Marko Neumann
German Institute for International Educational Research, Germany
Michael Becker
German Institute for International Educational Research, Germany
Kai Maaz
German Institute for International Educational Research, Germany

Update Modus of this Database

The current conference programme can be browsed in the conference management system (conftool) and, closer to the conference, in the conference app.
This database will be updated with the conference data after ECER. 

Search the ECER Programme

  • Search for keywords and phrases in "Text Search"
  • Restrict in which part of the abstracts to search in "Where to search"
  • Search for authors and in the respective field.
  • For planning your conference attendance, please use the conference app, which will be issued some weeks before the conference and the conference agenda provided in conftool.
  • If you are a session chair, best look up your chairing duties in the conference system (Conftool) or the app.