Implementing large scale instructional change in secondary schools? An experimental study.

Author(s):

Sébastien Dellisse(presenting / submitting)Xavier Dumay Benoit Galand Vincent Dupriez

Conference:

ECER 2016

Network:

23. Policy Studies and Politics of Education

Format:

Paper

Session Information

23 SES 10 D, Policy Reforms and Implementation Processes (Part 1)

Paper Session to be continued in 23 SES 11 D

Time:

2016-08-25

15:30-17:00

Room:

NM-J110

Chair:

Janne Varjo

Contribution

The literature on policy implementation in education has largely concluded that implementing instructional change at a large-scale level is a substantial challenge, especially when the change is top-down (Tyack & Tobin, 1994; Hamilton & al. 2003; Dumay, 2009). Many reforms fail in their twin goals to change teaching practices and improve student achievement (Dupriez, 2015).

For a long time, researchers have aimed at identifying the mechanisms explaining the deviations between the political intentions of the reforms and the teachers’ work. They have in particular shown that the level of political congruence, pervasiveness, intensity, and voluntariness mattered to make sense of the differences in the level of policy implementation within schools. To respond to this failure, in the USA, several initiatives most often-labelled ‘Comprehensive School Reforms’ (CSR) have emerged with the objective to promote students’ learning in high-poverty schools. The CSR (for instance the ‘Success for All’, the ‘America’s choice’ …) provided opportunities for studying the effects of alternative modes of policy implementation on the teachers’ work. In particular, Rowan and Miller (2007) have shown that reforms using a professional control (based on collaboration and colleagues’ support) or a procedural one (based on precise guidelines) are more successful than a cultural control (based on teachers shared understanding of reform intentions). Likewise, some facilitators should be ensured, for example trainers could guide the teachers and give them an easy access to support and equipment needed while they implement the program. Finally, programs that follow structured approaches to instructional change are more likely than the adaptive ones to implement new instructional practices in schools (Rowan & Miller, 2007).

It remains unclear, however, what are the processes underlying the greater efficiency of procedural and professional modes of implementation. The major objective of this presentation will be to explore this issue based on the analyses of experimental data of the program implementation.

An experimental study was conducted in a sample of secondary schools (n = 19) and classes (n = 82) with the objective to foster the competencies in reading comprehension of 1^st year students. Secondary schools were randomly assigned either to the control group or to experimental conditions. Two experimental conditions were created that reflected procedural and/or professional controls. In experimental condition 1, teachers received the structured didactic intervention with a training adapted to its use; in experimental condition 2, teachers received the treatment, adapted training and guidance through coaching sessions.

Through our research, we try to understand how procedural and professional control could foster the implementation of alternative teaching practices. However, we also attempt to find different levels of instructional change depending on the type of control. To understand this, we have explored another piece of literature interested in factors affecting implementation (Durlak & Dupre, 2008). This literature shows among others the importance of individual characteristics and collective dimensions.

First, we hypothesize that procedural and professional control, in comparison with the control group, would promote instructional change through the implementation of the program because it is structuring for practices and it enables the introduction of new routines (Marz, 2014). However, we believe that this effect would be mediated by teachers’ characteristics such as self-efficacy and task values (Dumas & al., 2001; Dusenbury & al., 2003).

Second, we hypothesize that professional control would promote a higher level of implementation that procedural control thanks to coaching sessions which take place in experimental condition 2. Coaching sessions is attempted to foster the instructional change because they were built to develop collective dimensions such as collaboration, colleagues support and shared-standards inside the educative team (Durlak & Dupre, 2008).

Method

Data collection was organized around three times: (1) before the intervention to assess students’ performance, teachers and school characteristics, (2) during the school year to collect information on the implementation process, and (3) after the intervention to assess again students’ performance and see their evolution. Data were collected on the basis of self-reported questionnaires validated by factor analysis and Cronbach’s alphas. To assess implementation, we have based our measurement on its two dimensions: quantity and quality (Rohrbach et al., 1993; Basen-Engquist et al., 1994, In Resnicow et al., 1998). Quantity refers to the intensity in which the program is delivered by teachers (Payne et al., 2006). In our research, we have asked teachers about the activities they have realised thanks to a checklist. The quality of implementation could be understood as the fidelity with which teachers deliver the program. In that way, ‘an intervention can be said to satisfy fidelity requirements if it can be shown that each of its components is delivered in a comparable manner to all participants and is true to the theory and goals underlying the research’ (Dumas et al., 2001, p. 38). To measure this dimension, we used a seven-point Likert scale and asked teachers about their practices during the school year. Questions were constructed around three dimensions based on practices promoted by the program: teacher guidance, collaboration among students, and teaching strategies for reading and writing. Teachers’ characteristics were measured at two levels: a general one and a specific one related to the program implemented. At the general level, teachers were asked about their self-efficacy related to general teaching abilities. We also asked teachers about the value they give to teaching reading and writing strategies related to informative texts. At the specific level, we have adapted our self-efficacy scale and our task-value scale to the program implemented. Finally, organizational capacity (like collaboration, shared-standards, colleagues and direction support…) was also measured throughout Likert-scales at two levels. First, we asked teachers about these dimensions at school level in general and, after the intervention, teachers in experimental conditions were also asked about the organizational capacity of their school around the program.

Expected Outcomes

Despite analyses are still ongoing, preliminary results of multilevel analyses show that there is a significant impact of the experimental conditions on the fidelity of implementation (e.g. explicit teaching, collaborative learning, and metacognition), but no significant difference between experimental condition 1 (training) and 2 (training and coaching). Further analyses are looking at individual and collective dimensions that mediate the instructional change in experimental conditions. They show that there is a positive and significant correlation between the general value given by teachers to reading and writing strategies and the teaching practices promote by the program. Correlations have also highlighted that teachers who feel more self-confident with the program itself (using books, planning and implementing activities) do more activities than the others. But if teachers feel more self-confident with students’ management around activities proposed in the program, they seem to implement more faithfully practices linked to teaching reading and writing strategies. Finally, teachers who collaborate around program’s activities are more likely to increase the implementation fidelity, both for their intervention among students than for teaching the strategies. We also found that teachers do more activities if they share the same standards.

References

Bandura, A. (2007). Auto-efficacité : le sentiment d’efficacité personnelle. Paris : De Boeck University. Dumas, J., Lynch, A., Laughlin, J., Smith, E., & Prinz, R. (2001). Promoting Intervention Fidelity: Conceptual Issues, Methods, and Preliminary Results from the EARLY ALLIANCE Prevention Trial. American Journal of Preventive Medicine, 20 (1S), 38—47. Dumay, X. (2009). Efficacité des modes locaux de coordination et de gestion des établissements. Revue française de pédagogie, 167, pp. 101-129. Dupriez, V. (2015). Peut-on réformer l’école ? Approches organisationnelle et institutionnelle du changement pédagogique. Bruxelles : de Boeck. Durlak, J. & DuPre, E. (2008). Implementation Matters: A Review of Research on the Influence of Implementation on Program Outcomes and the Factors Affecting Implementation. American Journal of Community Psychology, (41), 327–350. DOI: 10.1007/s10464-008-9165-0 Dusenbury, L., Brannigan, R., Falco, M. & Hansen, W. (2003). A review of research on fidelity of implementation: implications for drug abuse prevention in school settings. Health Education Research, 18 (2), 237—256. Eccles, J. & Wigfield, A. (2000). Expectancy-Value theory of achievement motivation. Contemporary Educational Psychology, 25 (1), 68-81. Hamilton, L. S., McCaffrey, D. F., Stecher, B. M., Klein, S., P., Robyn, A. & Bugliari, D. (2003). Studying Large-Scale Reforms of Instructional Practice: An Example from Mathematics and Science. Educational Evaluation and Policy Analysis, 25 (1), pp. 1-29. Marz, V. (2014). It takes two to tango. Structuring actors and acting structures in the implementation of educational innovations. Thèse de doctorat en sciences pédagogiques et psychologiques non publiée. Katholieke Universiteit Leuven, Leuven. Payne, A. A., Gottfredson, D. C. & Gottfredson, G. D. (2006). School Predictors of Intensity of Implementation of School-Based Prevention Programs: Results from a National Study. Prevention Science, 7 (2), 225-237. DOI: 10.1007/s11121-006-0029-2 Resnicow, K., Davis, M., Smith, M., Lazarus-Yaroch, A., Baranowski, T., Baranowski, J., Doyle, C. & Wang, D. T. (1998). How best to measure implementation of school health curricula: a comparison of three measures. Healt Education Research: Theory & Practice, 13 (2), 239 - 250. Rowan, B. & Miller, R. J.(2007). Organizational Strategies for Promoting Instructional Change: Implementation Dynamics in Schools Working With Comprehensive School Reform Providers. American Educational Research Journal, 44 (2), pp. 252–297 DOI: 10.3102/0002831207302498 Tyack, D., & Tobin, W. (1994). The ‘grammar’ of schooling: Why has it been so hard to change? American Educational Research Journal, 31 (3), pp. 453–479.

Author Information

Sébastien Dellisse (presenting / submitting)

UCL

Psychology and Education Sciences

Louvain-la-Neuve

Xavier Dumay

Université Catholique de Louvain

Louvain-la-Neuve

Benoit Galand

UCL, Belgium

Vincent Dupriez