Review and Update of Methodology for Evaluation of the Effects of the Professional Development Programs for Teachers

Author(s):

Dragica Pavlovic Babic(submitting)Sonja Banjac(presenting)Aleksandar Baucal Tinde Kovač Cerović

Conference:

ECER 2017

Network:

01. Continuing Professional Development: Learning for Individuals, Leaders, and Organisations

Format:

Paper

Session Information

01 SES 05 C, Evaluation and Impact of Professional Development Programmes

Paper Session

Time:

2017-08-23

13:30-15:00

Room:

K3.13

Chair:

Stefan Zehetmeier

Contribution

Professional development programs for teachers (PPDT) are a key mechanism for modernising and improving quality of teaching and consequently student achievement (Guskey, 2000; Yoon, Duncan, Lee, Scarloss, & Shapley, 2007). Organization and participation in these programs yield significant time and human resource investment which raises the question of their effectiveness. Evaluation of the PPDT effectiveness has been acknowledged as a priority by a number of countries which dealt with this issue systematically (Zafeirakou, 2002).

In Serbia, on the initiative of the Ministry of Education, the Institute of Psychology team worked on the development of methodological framework for evaluation of the effects of PPDT. The proposed solution (Pavlović Babić, Kovač Cerović, Banjac, Cenerić & Baucal, 2016) was based on identified shortcomings of existing practice, the experience of other European countries and the review of research papers on methodology for PPDT effects evaluation. Additionally, it was incorporated into existing legal regulations, organization of PPDT and system quality assurance in the Serbian educational system. This framework included self-evaluations and teacher evaluations, evaluations at the school level, external evaluation of school quality, national and international assessments, and special evaluative studies of effects of individual PPDT.

The methodology aimed to provide objective, reliable and valid information about the effects of PPDT on teachers’ competencies. However, it had an underlying assumption that had to be examined further. Namely, it assumed that teacher improve certain competencies after attending PPDT and that the effects depend on quality of the programs. Nevertheless, there could be a number of unwanted scenarios between teachers’ decision which PPDT they need and implementation. Those potential obstacles can influence the occurrence of the expected program outcomes and can bring down PPDT to mere points gathering. Therefore in present study we sought to investigate three specific issues in PPDT system – planning individual professional development, perception of usefulness of different professional development types and necessary assistance in implementation.

Legal frameworks regulating PPDT in Serbian education system impose teachers’ autonomy in planning. The system allows teachers to choose advancing in the areas they consider they need enhancement based on self-evaluations of competencies and school developmental plan. In this study we wanted to investigate what are the inputs teachers take into account when planning their professional development.

Additionally, we wanted to explore how much teachers participate in different types of professional development recognized by the legal regulations and how useful it has been for their practice, according to their experience. Previous research (Garet, Porter, Desimone, Birman, & Yoon, 2001; Hattie, 2009) have shown that programs which allow teachers to participate actively, to reflect, analyse and discuss their practice are more effective. In addition to that, effective seminars were also characterised by being longer and more intensive, using more group for study forms instead of traditional lectures and they also included more teachers from the same schools.

Finally, we wanted to investigate whether teachers perceive they need help implementing what they have learnt within the PPTD and from whom do they expect it. We didn’t have any data on support teachers get in this step of PPTD and previous research (Garet et al., 2001; Hattie, 2009) has shown that programs have more effect if there is help provided after the trainings.

The findings about described aspects of PPTD would help us to validate and, if necessary, revise the proposed methodology, taking into consideration teachers’ perspective and experience. Mainly, the new data will allow us to adjust the methodology framework to existing issues making it, thus, able to detect and report them.

Method

The present study was conducted with representative sample of schools in Serbia. Namely, we randomly selected 49 municipalities and contacted one gymnasium, one vocational school and three primary schools per it. The overall sample comprised of 3499 teachers, specifically 2177 primary school teacher (from 109 schools), 592 gymnasium teachers (28 gymnasiums) and 730 vocational school teachers (41 vocational schools). Almost 60% of teachers in our sample had 8-25 years of experience in teaching, and 91,9% of all participated teachers attended some forms of PPTD in the last two years. Teachers were filling in the questionnaire that gathered information about different aspects of professional development and we decided to analayze those questions concerning individual PPTD planning, usefulness of different types of programs and assistence in implementing the programs. Due to aim to explore the existing issues, the questionnaire was extensive, that is it combined closed and open-end questions. The investigated themes were covered by eight questions in total. Question about usefulness of programs was in a form of four-point Likert scale and the one about specific support teachers expect out of different institutions was open-ended question. All of the rest were multiple choice questions and, even though answers were provided, participants were also allowed to write their own response if they did not agree with any of the given ones. A letter explaining the aims and rationale of the study was sent to head-teachers in each school. Participation was voluntary and participants were guaranteed anonymity. Questionnaire was administered online and in order to control filling in, each school had its unique password.

Expected Outcomes

The present research showed that most of the teachers in our sample plan their professional development taking into account student achievement (49,1%), innovations in their fields (46,2) and self-evaluation of their competencies (44,3%). This finding is in line with the proposed methodology which included student achievement as one of the elements, as well as with TALIS study which showed that for teachers in Serbia student achievement is the most important indicator of their performance (OECD, 2014). We also asked teachers how much of the PPTD plan has been achieved. Those that reported that less than half of the plan has not been managed attributed it mostly to the lack of finance (77%). These results suggest that teachers do not always participate in programs they need. In teachers experience the type of PPTD which has contributed most to their practice was exchange of experience and knowledge with colleagues and reading literature. The rationale for this result could be that exchange with colleagues from the school allows teachers to be active, ask direct questions and to discuss their practice which are the characteristics of effective PPDT (Garet et al., 2001; Hattie, 2009). Finally, 73% of teachers said that they decide what to implement on their own. Nonetheless, although 39% of our participants reported that they do not need help, 37% say they expect help from their colleagues or school psychologist (35%). The obtained results supported the inclusion of several elements in the proposed framework, such as evaluations at the school level and national and international student assessments. Furthermore it indicated that modifications are needed in the design and information obtained through self-evaluations and teacher evaluations, external evaluation and special evaluative studies of effects of individual PPDT.

References

Desimone, L. M. (2009). Improving impact studies of teachers’ professional development: Toward better conceptualizations and measures. Educational researcher, 38(3), 181-199. Guskey, T. R. (2000). Evaluating professional development. Corwin Press. Hattie, J. A. C. (2009). Visible learning: A synthesis of 800+ meta-analyses on achievement. Abingdon: Routledge. OECD (2014), Talis 2013 Results: An International Perspective on Teaching and Learning, OECD Publishing. http://dx.doi.org/10.1787/9789264196261-en Pavlović Babić, D., Kovač Cerović, T., Banjac, S., Cenerić, I., Baucal, I. (2016.) The proposal of Methodology for Evaluation of the Effects of the Professional Development Programs for Teachers. ECER 2016, Leading Education: The Distinct Contributions of Educational Research and Researchers. Dublin 22-26 August. Yoon, K. S., Duncan, T., Lee, S. W. Y., Scarloss, B., & Shapley, K. (2007). Reviewing the evidence on how teacher professional development affects student achievement (Issues and Answers Report, REL 2007 No. 033). Washington, DC: U.S. Department of Education, Regional Educational Laboratory Southwest. Zafeirakou, A. (2002). In-service training of teachers in the European Union: exploring central issues. World Bank, Washington DC.

Author Information

Dragica Pavlovic Babic (submitting)

University of Belgrade

Belgrade

Sonja Banjac (presenting)

University of Belgrade, Serbia

Aleksandar Baucal

University of Belgrade

Department of Psychology

Belgrade

Tinde Kovač Cerović

Faculty of Philosophy

Department of Psychology

Beograd

Search the ECER Programme

Search for keywords and phrases in "Text Search"
Restrict in which part of the abstracts to search in "Where to search"
Search for authors and in the respective field.
For planning your conference attendance you may want to use the conference app, which will be issued some weeks before the conference
If you are a session chair, best look up your chairing duties in the conference system (Conftool) or the app.