Can School External Evaluation to promote school improvement? School teachers’ opinions

Author(s):

Carla Figueiredo(presenting / submitting)Carlinda Leite Preciosa Fernandes

Conference:

ECER 2016

Network:

11. Educational Improvement and Quality Assurance

Format:

Paper

Session Information

11 SES 04 A, Paper Session

Paper Session

Time:

2016-08-24

09:00-10:30

Room:

OB-E2.14

Chair:

Irina Maslo

Contribution

School improvement has been a major target of educational policies in European countries over the last 20 years, leading to the implementation of various measures (Faubert, 2009). The process of school external evaluation (SEE) is one example of these measures.

A research conducted by the OECD concluded that countries develop school external evaluation with the aim of promoting school improvement and accountability which «… relates to access to education (equity) and education performance (quality and efficiency» (Faubert, 2009: 7).

The literature presents considerations on the relationship between SEE and school improvement. Overall, there seems to be a close link between evaluation initiatives and improvement measures implemented in schools. A common assumption is that SEE gathers important data concerning schools’ functioning and produces valuable knowledge to understand how schools work. The processes of external evaluation can identify strong features in schools functioning which can be seen as good practices to be encouraged and disseminated, and most importantly for the improvement, it can provide insights on aspects that are in need of change and issues needing solution (Reezigt & Creemers, 2005; Campbell & Levin, 2009; Coe, 2009). The knowledge produced by SEE grants schools awareness and can provide options to surpass them and thrive.

Based on this, researchers defend that school evaluation has the potential to inform change and lead to improvement (Ehren & Visscher, 2008; Campbell & Levin, 2009; Coe, 2009; Schildkamp et al, 2012; Lindahl & Beach, 2013; Gaertner, Wurster & Pant, 2014).

Despite the potential recognised in SEE, research (Perryman, 2010; Gaertner, Wurster & Pant, 2014) also shows that the relationship between school evaluation and school improvement is not always so linear and school improvement is sometimes ambiguous. In fact, as Ehren and Visscher (2008) and Ehren et al (2015) concluded, there are many factors that influence this relationship, amongst which is the issue of how the processes of school external evaluation are conducted. For instance, the nature of the relationship between the evaluators and the evaluated, the kind of evaluation model and its adequacy to the context and aims of evaluation (Skedsmo, 2011); the need to acknowledge the characteristics of context and of each school (Ehren & Visscher, 2006; Coe, 2009); the need to ensure the implementation of actions that are not imposed, but are the result of negotiation and collaboration between parties (Bolívar, 1999; 2012; Brown, 2004; Reezigt & Creemers, 2005). Regarding this, Altrichter & Kemethofer (2015) highlight the importance of the feedback provided to schools as a result of SEE. The authors affirm that the lack of feedback can limit the positive impact and the degree in which SEE can be help schools to improve. This calls for a change in the SEE processes and how they are conducted.

Bearing all of this in mind, this presentation aims to gather the opinion from the ones being evaluated, namely teachers, in order to: i) understand the relation between the processes school external evaluation and school improvement and ii) identify the factors that could ensure SEE as a promotor of improvement. Based on data from two countries with a tradition of school external evaluation, Portugal and England, this presentation addresses the questions: 1) Can School External Evaluation to promote school improvement, according to teachers?; and 2) What aspects of the SEE process should be reinforced to help schools, in teachers’ opinion?

Method

The methodological approach followed a qualitative orientation, based on qualitative data collection and analysis techniques. Data was gathered through semistructured (Bogdan & Biklen, 1994) and focus group (Greenbaum, 1998) interviews with teachers from basic and secondary education schools, from two countries with a tradition on SEE, Portugal end England. The two countries were selected due to being two eastern European countries, with somewhat similar evaluation processes with the aim of improving education, and for being countries with a well implemented SEE system. Studying these two countries could help to develop a deeper understanding on the processes of evaluation, how they are developed in different socioeconomic and political contexts, and their impacts on schools. The data collection techniques used were chosen for allowing the interviewees to freely express their opinions, while also allowing the interviewer to focus on specific subjects. The interviews followed a script which covered a range of aspects related to school evaluation regarding the opinions and conception on SEE processes, on how they are conducted and their impacts and effects. Based on the intention of gathering opinions on the processes, the main criterion for the selection of the participants was to consider teachers who had experienced at least one school evaluation moment. In the case of Portugal, and considering that most of the schools were evaluated only twice, another very important criterion was to consider schools that had improved from the first to the second SEE, and schools that had aggravated, according to the SEE results. This could help to identify relationships between SEE and improvement. A total of 27 teachers were interviewed through focus group interviews (7 groups), in Portugal, and 13 semistructured individual interviews were conducted in England. The analysis was performed through content analysis (Krippendorf, 2003), using the Nvivo 10 software.

Expected Outcomes

The analysis revealed a consistency in the discourses of teachers from both countries. So, to answer to the first research question, 1) Can School External Evaluation to promote school improvement, according to teachers?, teachers reveal that SEE can help schools to improve and is, in fact, doing so. By providing a source of useful information and promoting space for rethinking practices, schools are improving their functioning. But, while recognising this, they also feel that SEE could be more helpful to schools and are very assertive on what should be done to overcome this. To answer the second question, 2) What aspects of the SEE process should be reinforced to help schools, in teachers’ opinion?, the most common argument expressed is: a constructive and formative approach. Teachers agree that the informative aspect of school evaluation is important, but it alone is not enough to promote the schools’ improvement. They claim that evaluators possess valuable knowledge that could help schools which are struggling. And a more formative approach in which evaluators collaborate with schools is much more helpful one. Teachers also claim that the processes should be more comprehensive, considering schools as a whole, instead of focusing on one aspect. They criticise the attention payed to school results as the sole indicator of school quality, as it limits school education to numbers and provides little insights on how schools can improve. Teachers’ claims are in line with the results from previous research (Skedsmo, 2011; Ehren & Visscher, 2006; Coe, 2009; Brown, 2004; Reezigt & Creemers, 2005; Altrichter & Kemethofer, 2015), meaning that any nation that is either starting to implement school evaluation processes or already has one, could benefit from (re)thinking about how it is being conducted and how it could be better developed to reach the goal of school improvement.

References

Altrichter, H. & Kemethofer, D. (2015). Does accountability pressure through school inspections promote school improvement?. School Effectiveness and School Improvement, 26(1), 32-56. doi: 10.1080/09243453.2014.927369. Bolívar, A. (1999) Como melhorar as Escolas – estratégias e dinâmicas de melhoria das práticas educativas. Porto: Edições ASA. Bolívar, A. (2012). Melhorar os Processos e os Resultados Educativos: O que nos ensina a investigação. Vila Nova de Gaia: Fundação Manuel Leão. Brown, G. (2004) Teachers' conceptions of assessment: implications for policy and professional development. Assessment in Education: Principles, Policy e Practice, 11(3), 301-318. doi: 10.1080/0969594042000304609. Campbell, C. & Levin, B. (2009) Using data to support educational improvement. Educational Assessment Evaluation and Accountability, 21(1), 47-65. doi: 10.1007/s11092-008-9063-x. Coe, R. (2009). School Improvement: Reality and Illusion. British Journal of Educational Studies, 57(4), 363-379. doi: 10.1111/j.1467-8527.2009.00444.x. Ehren et al (2015). Comparing effects and side effects of different school inspection systems across Europe. Comparative Education, 51(3), 375-400, DOI: 10.1080/03050068.2015.1045769 Ehren, M. & Visscher, A. (2006). Towards a theory on the impact of school inspections. British Journal of Educational Studies, 54(1), 51-72. doi: 10.1111/j.1467-8527.2006.00333.x. Ehren, M. & Visscher, A. (2008). The Relationships Between School Inspections, School Characteristics And School Improvement. British Journal of Educational Studies, 56(2), 205-227. doi: 10.1111/j.1467-8527.2008.00400.x. Faubert, V. (2009). School Evaluation: Current Practices in OECD Countries and a Literature Review. OECD Education Working Papers: OECD Publishing, 42, p. 1-59. Gaertner, H.; Wurster, S. & Pant, H. (2014). The effect of school inspections on school improvement. School Effectiveness and School Improvement, 25(4), 489-508. doi: 10.1080/09243453.2013.811089. Lindahl, R. & Beach, R. (2013). The role of evaluation in the school improvement process. Planning and Changing, 44(1/2), 56-72. Perryman, J. (2010). Improvement after Inspection. Improving Schools, 13(2), 182–196. doi: 10.1177/1365480210369878. Reezigt, G. & Creemers, B. (2005). A comprehensive framework for effective school improvement. School Effectiveness and School Improvement 16(4), 407-424. doi: 10.1080/09243450500235200. Schildkamp, K. et al (2012). The use of school self-evaluation results in the Netherlands and Flanders. British Educational Research Journal, 38(1), 125-152. doi: 10.1080/01411926.2010.528556. Skedsmo, G. (2011). Formulation and realisation of evaluation policy: inconsistencies and problematic issues. Educational Assessment Evaluation and Accountability, 23, 5-20. doi: 10.1007/s11092-010-9110-2

Author Information

Carla Figueiredo (presenting / submitting)

Faculty of Psychology and Educational Sciences of the University of Porto

Educational Sciences

Porto

Carlinda Leite

Faculty of Psychology and Educational Sciences of the University of Porto, Portugal

Preciosa Fernandes