Research Question

Impact analyses and evaluations of professional development programmes are mainly scheduled during or at the end of a particular programme. They aim at and provide results regarding immediate and short-term effects. However, apart from and beyond that, an analysis of sustainable effects is crucial. To address this issue, this contribution deals with the central question: What is the sustainable impact of professional development programmes? Theoretical models and empirical findings are introduced. In particular, this contribution provides research results regarding an Austrian professional development programmes’ impact. Here, the factors which foster or hinder the sustainability of impact are in the focus. Finally, implications for professional development programmes’ implementation and research are discussed.

Theoretical Framework

The question of how to promote mathematics teachers’ professional development has been discussed in various papers (e.g. Krainer and Zehetmeier 2013; Loucks-Horsley et al. 1996; Sowder 2007; Zehetmeier 2015; Zehetmeier and Krainer 2011). In this context, the question of impact is of particular relevance. Evaluations and impact analyses of professional development programmes are mostly conducted during or at the end of a project and exclusively provide results regarding short-term effects. These findings are highly relevant for critical reflection of the terminated project and necessary for the conception of similar projects in the future (Fullan 2006). However, apart from and beyond that, an analysis of sustainable effects is crucial (Loucks-Horsley et al. 1996). Despite its central importance for both teachers and teacher educators, research on sustainable impact is generally lacking within teacher education disciplines (Datnow 2006; Rogers 2003). This kind of sustainability analysis is often missing because of a lack of material, financial and personal resources (McLaughlin and Mitra 2001; Hargreaves 2002).

The expected impacts of professional development programmes are not only focused on short-term effects that occur during or at the end of the project, but also on long-term effects that emerge (even after some years) after the project’s termination. Effects that are both short-term and long-term can be considered to be sustainable. Sustainability may refer to both system and/or individual level. Sustainability can be defined as the lasting continuation of achieved benefits and effects of a project or initiative beyond its termination (DEZA 2002).

Empirical evidence concerning the impact of professional development programmes points to the finding that “prior large-scale improvement efforts (…) have rarely produced lasting changes in either teachers’ instructional practices or the organization of schools” (Cobb and Smith 2008, p. 232). Thus, it seems reasonable to focus on factors which might foster the broad effects and scale-up of professional development programmes’ innovations. Cobb and Smith (2008) highlight networks, shared vision and mutual accountability as key factors.

Teacher networks are described, for example, as groups of colleagues who provide social support in developing demanding instructional practices; this affords time built into the school schedule for collaboration among teachers and access to colleagues who have already developed relatively accomplished instructional practices.

Moreover, a shared vision of high quality instruction fosters the scale-up of impact: this includes a shared vision concerning the question of instructional goals (what pupils should know and be able to do) and the question of how pupils’ development of these forms of knowing can be supported.

Another key factor which fosters the scale-up of innovations and impact in teacher education is mutual accountability. This means, for example, that if school leaders hold teachers accountable for developing high-quality instructional practices, then – in turn – school leaders are mutually accountable to teachers for supporting teachers’ learning.

Within an Austrian teacher professional development programme, several case studies were conducted, with the aim to research the sustainability of the programme’s impact. The case study presented here was based on data from various sources and time periods to gain validity by “convergence of evidence” (Yin 2003, p. 100): data collection contained documents (e.g. teachers’ project reports, which were written during and at the end of teachers’ participation in the project) and archival records (e.g. author’s field-notes, which originate from author’s activities as teacher educator in the project). Moreover, interviews were conducted during, at the end, and one year after the programme’s termination (with participating teachers, teachers’ colleagues, principals, and project facilitators and teacher educators). Data analysis included both inductive and deductive elements to analyse both the impact and the respective fostering (or hindering) factors. The interviews were semi-structured, since they were based on the analysis of existing data (document analysis), which identified various levels of short-term impact which occurred during and/or at the end of teachers’ participation. The interviews were designed accordingly (a) to gather data concerning the sustainability and scale-up of impact and (b) to reveal other types of impact which were not already coded. Data was analysed by qualitative content analysis (Mayring 2003) in order to identify common topics, elaborate emerging categories, and gain deeper insight into teachers’ professional growth over time. The case studies’ results were validated by means of member checking.

Case study A teacher who participated in a professional development programme had the goal to promote open learning settings by implementing new teaching approaches in her mathematics classes. Document analysis showed that she aimed at enhancing pupils’ inquiry-based learning opportunities. During her participation in the programme, she changed her teaching practices and implemented innovative teaching approaches to enhance her pupils’ self-directed and independent learning. Interview data clearly shows that this impact was sustained: the changes in teaching practices stayed effective even after the termination of the programme. Core fostering factors were the school principal’s support and a high level of mutual appreciation within the school staff, and pupils’ benefit. This impact was sustainable, due to her direct advantage (by getting information on her classroom performance) and the support of the school’s principal (who was convinced that reflections and self-evaluations are important steps on the journey to school quality). This impact was also fostered by teacher colleagues’ joint reflection and communication. Moreover, the school’s principal showed great interest in and provided support for the teachers’ activities. He participated in the school’s mathematics study group and shared his perspective with the teachers. The factors that fostered the sustainability of this case study’s impacts are mirrored by the theoretical framework (see above): The programme enabled networking (Cobb & Smith, 2008) by community building, mutual appreciation and joint reflection. A particular factor was the principals’ content knowledge (ibid.). Teachers’ colleagues provided communication and social support in developing and reflecting instructional practices (ibid.). Moreover, a shared vision (ibid.) of values and standards regarding high quality mathematics instruction was established. By gathering further data in the future, the professional development programme’s sustainable impact will be researched continuously, with the aim to track changes over time.

Cobb, P., & Smith, T. (2008). The challenge of scale: designing schools and districts as learning organizations for instructional improvement in mathematics. In K. Krainer & T. Wood (Eds.), International handbook of mathematics teacher education (Vol. 3, pp. 231–254). Rotterdam: Sense Publishers. Datnow, A. (2005). The sustainability of comprehensive school reform models in changing district and state contexts. Educational Administration Quarterly, 41(1), 121-153. DEZA – Direktion für Entwicklungshilfe und Zusammenarbeit (2002). Glossar deutsch [German glossary]. Bern, Switzerland: DEZA. Fullan, M. (2006). The future of educational change: system thinkers in action. Journal of Educational Change, 7, 113–122. Hargreaves, A. (2002). Sustainability of educational change: the role of social geographies. Journal of Educational Change, 3, 189–214. Krainer, K., & Zehetmeier, S. (2013). Inquiry-based learning for pupils, teachers, researchers, and representatives of educational administration and policy: reflections on a nation-wide initiative fostering educational innovations. ZDM Mathematics Education, 45(6), 875–886. Loucks-Horsley, S., Stiles, K., & Hewson, P. (1996). Principles of effective professional development for mathematics and science education: a synthesis of standards. NISE Brief, 1(1), 1–6. Mayring, P. (2003). Qualitative Inhaltsanalyse [Qualitative content analysis]. Weinheim, Germany: Beltz. McLaughlin, M., & Mitra, D. (2001). Theory-based change and change-based theory: going deeper, going broader. Journal of Educational Change, 2, 301–323. Rogers, E. (2003). Diffusion of innovations. New York: Free Press. Sowder, J. (2007). The mathematical education and development of teachers. In F. Lester (Ed.), Second handbook of research on mathematics teaching and learning (pp. 157–223). Greenwich, CT: NCTM. Yin, R. (2003). Case study research: Design and methods (3rd Edition). Thousand Oaks, CA: Sage Publications. Zehetmeier, S. (2015). Sustaining and scaling up the impact of professional development programmes. ZDM - The International Journal on Mathematics Education, 47(1), 117-128. Zehetmeier, S., & Krainer, K. (2011). Ways of promoting the sustainability of mathematics teachers’ professional development. ZDM Mathematics Education, 43(6/7), 875–887.