The impacts of School Evaluation Processes – Schools’ Perspectives on School Evaluation
Author(s):
Conference:
ECER 2014
Format:
Paper (Copy for Joint Session)

Session Information

09 SES 14 JS, Checking Effectiveness by Evaluating Schools

Paper Session Joint Session NW 09 and NW 11

Time:
2014-09-05
15:30-17:00
Room:
B231 Sala de Aulas
Chair:
Martin Goy

Contribution

School evaluation processes have been established in, and generalized to, the majority of European countries, assuming diverse characteristics and following different orientations (Faubert, 2009).  Some countries have developed their own models for school evaluation, while others have adopted, and adapted, evaluation frameworks already established.  Also, European countries have opted for different modalities for their school evaluation processes. Some opted for external school evaluation processes, others have focused on internal school evaluation processes (self-evaluation), and others choose to develop both modalities, aiming for a complementarity between them (Faubert, 2009).

The spreading of school evaluation is directly linked with current theoretical discourses and international political recommendations appealing for the implementation of such processes, which are drivenby: 1) the concerns and the need to increase the quality of the educational service provided in Europe, and consequently, to improve European educational systems (Figueroa, 2008; Grek, et al, 2009; Commission of the European Communities, 2001; OECD, 2011; Commission of the European Communities, 2007; Alaíz, 2003); and 2) the belief that evaluation can have significant impacts on schools functioning, being a valuable tool for diagnosis, for identifying key aspects – both positive and/or negative –, and for designing a development strategy (Reezigt & Creemers, 2005; Plowright, 2007; Sun, Creemers & Hong, 2007; Campbell & Levin, 2009; Coe, 2009; Hofman, Dijkstra & Hofman, 2009; Campbell & Levin, 2009; Coe, 2009). It is also important to stress that the concerns with educational quality are the result of a globalized and competitive world, in constant evolution, which demands better responses to the increasing challenges (Commission of the European Communities, 2001; European Union Council, 2009). The above referred arguments justify the increasing establishment of school evaluation processes in European nations, in the last decade.

Regardless of the school evaluation modality in question, it is always a process that interferes with schools’ natural daily routine (Finnigan & Gross, 2007; Ehren & Visscher, 2008). This interference has direct impacts, positive or negative, in all levels of school work, from the general management and leadership, to the classroom practices. Such impacts cannot be left unexplored, especially considering the pressure addressed to schools to become better, to reach better outcomes, and to autonomously improve their functioning. Therefore, to effectively understand the impacts of school evaluation processes in school life, it is necessary to take into consideration the opinions of the main actors involved in such processes: teachers, headmasters, and other school population members.

Focusing the attention on what school agents experience during evaluations, and on their perspectives on school evaluation processes and impacts on schools’ functioning, it is possible to reach a, somewhat, meta-evaluation of the evaluation process itself. That is, it becomes possible to identify main strengths, flaws and outcomes, and improve school evaluation in order for it to meet its goals. European countries with a school evaluation culture already established, such as Portugal, England, Scotland, Denmark, among others, constitute appropriate examples for deepening this knowledge.

Following this line of thought, this presentation aims to shed some light on the subject of school evaluation. Specifically, this presentation aims to: 1) understand the perspectives of school agents on the subject of school evaluation (both external and self-evaluation); 2) identify the main positive outcomes of school evaluation processes (both external and self-evaluation), from the point of view of school agents; 3) identify the main negative outcomes of school evaluation processes (both external and self-evaluation), from the point of view of school agents.

Based on data gathered in two Portuguese schools, this presentation will present the main impacts, strengths and flaws of school evaluation processes, as identified by the schools’ headmasters, self-evaluation teams and teachers.

Method

Aiming to reach an understanding on the impacts of school evaluation processes in schools’ daily routines and the educational service provided, this research followed a mixed approach for data gathering and analysis, using both qualitative and quantitative techniques. The choice for a mixed approach is justified by: 1) the goal of reaching a comprehensive understanding of key school actors’ perspectives on school evaluation, such as school headmasters and school self-evaluation teams, which demanded a qualitative approach to data collection; and 2) the need to assess if the perceptions expressed by key actors are generalized to teachers, as main educative actors. For the qualitative data gathering techniques, the choice fell upon semi-structured interviews (Bogdan & Biklen, 1994) to be conducted to the school headmasters of two Portuguese schools, and focus group interviews (Greenbaum, 1998) with the self-evaluation teams of the two schools. Both the semi-structured interview and the focus group interview allow the interviewees to freely express their opinions, while allowing the interviewer to focus on the subjects in study. Also, the focus group, being a technique in which a group of people discuss a certain subject, can reveal key aspects, both by the extent of agreement, as well as for the disagreements they raise. For this reason, the focus group interview is a good asset in this research, enriching the data collection. In what concerns to the quantitative technique, the choice fell upon a survey questionnaire (Ghiglione & Matalon, 1993) to be applied to a representative sample of teachers from both schools. The questionnaire will be the result of an adaptation of a more extend questionnaire built, validated and applied in a research project focusing school evaluation in Portugal. The adaptation will focus the items specially dedicated to the impacts of evaluation. In order to meet the nature of the data collected, two data analysis techniques will be used: content analysis (Krippendorf, 2003), using the Nvivo 10 software, for the qualitative data; and statistical analysis, using the SPSS software, for the quantitative data. The content analysis allows the identification of main trends in the interviewees’ discourse, essential to build knowledge and understanding. The statistical analysis enables to assess the degree of agreement, or disagreement, of a larger sample size, concerning several key aspects. This allows to confirm, or deny, the main perceptions and trends from the interviews.

Expected Outcomes

By the end of the data analysis process, it should be possible to answer to the research objectives expressed above, and to draw some conclusions regarding school evaluation processes and impacts on schools’ functioning. The analysis should reveal, amongst other aspects, the following: - The main perceptions on school external evaluation processes and how they are conducted, from the point of view of schools; - The main perceptions on school self-evaluation processes and how they are conducted, from the point of view of schools; - The relation between the development of school external evaluation processes and the development of school self-evaluation processes; - The main positive aspects of school evaluation processes, identified by school members, related to the processes and how they are conducted; - The main negative aspects of school evaluation processes, identified by school members, related to the processes and how they are conducted; - The main positive outcomes and impacts of school evaluation processes, identified by school members; - The main negative outcomes and impacts of school evaluation processes, identified by school members; Based on all the ideas and opinions from the school actors, and the conclusions drawn from the analysis, it will also be possible to point some improvement suggestions concerning the evaluation processes.

References

Alaiz, V. et al (2003) Auto-avaliação de Escolas: Pensar e praticar, Porto: Edições ASA Campbell, C. & Levin, B. (2009) Using data to support educational improvement, Educational Assessment Evaluation and Accountability, 21, pp. 47-65 Coe, R. (2009) School Improvement: Reality And Illusion, British Journal of Educational Studies, 57(4), pp. 363-379 Commission of the European Communities (2007) Schools for the 21st Century – Commission Staff Working Paper, http://ec.europa.eu/education/school21/consultdoc_en.pdf Ehren, M. & Visscher, A. (2008) The Relationships Between School Inspections, School Characteristics And School Improvement, British Journal of Educational Studies, 56(2), pp. 205-227 European Parliament and Council (2001) Recommendation of the European Parliament and of the Council of 12 February 2001 on European cooperation in quality evaluation in school education, Official Journal of the European Communities L60, http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=OJ:L:2001:060:0051:0053:EN:PDF Faubert, V. (2009), “School Evaluation: Current Practices in OECD Countries and a Literature Review”, OECD Education Working Papers, No. 42, OECD Publishing. Figueroa, D. (2008) “Modelo de autoevaluación institucional global con apoyo externo en un centro educativo particular de Lima. Un estudio de caso, Educación 17(33), pp. 63-80 Finnigan, K. & Gross, B. (2007) Do Accountability Policy Sanctions Influence Teacher Motivation? Lessons From Chicago’s Low-Performing Schools, American Educational Research Journal, 44(3), pp. 594 –629 Ghiglione, R. & Matalon, B. (1993). O inquérito: teoria e prática. Oeiras: Celta Editora; Greenbaum, T. (1998) The handbook for focus group research. Thousand Oaks: Sage Publications; Grek, S. et al (2009) North by northwest: quality assurance and evaluation processes in European education, Journal of Education Policy, 24(2), pp. 121-133 Hofman, R., Dijkstra, N. & Hofman A., (2009) "School self-evaluation and student achievement”. School Effectiveness and School Improvement, 20(1), pp.47–68 Krippendorf, K. (2003). Content analysis: An introduction to its methodology. Beverly Hills: Sage OECD (2011) Evaluation and Assessment Frameworks for Improving School Outcomes: common policy challenges. OECD Education and Training Policy Note, http://www.oecd.org/edu/school/46927511.pdf OECD. Paris Plowright, D. (2007) Self-evaluation and Ofsted Inspection: Developing an Integrative Model of School Improvement, Educational, Management, Administration & Leadership, 35(3), pp. 373-393 Reezigt, G. & Creemers, B. (2005) A comprehensive framework for effective school improvement, School Effectiveness and School Improvement, 16(4), pp. 407-424 Sun, H., Creemers, B. & Jong, R. (2007) Contextual factors and effective school improvement, School Effectiveness and School Improvement, 18(1), pp. 93-122

Author Information

Carla Figueiredo (presenting / submitting)
Faculty of Psychology and Educational Sciences of the University of Porto, Portugal
Faculty of Psychology and Educational Sciences of the University of Porto, Portugal
Faculty of Psychology and Educational Sciences of the University of Porto, Portugal

Update Modus of this Database

The current conference programme can be browsed in the conference management system (conftool) and, closer to the conference, in the conference app.
This database will be updated with the conference data after ECER. 

Search the ECER Programme

  • Search for keywords and phrases in "Text Search"
  • Restrict in which part of the abstracts to search in "Where to search"
  • Search for authors and in the respective field.
  • For planning your conference attendance, please use the conference app, which will be issued some weeks before the conference and the conference agenda provided in conftool.
  • If you are a session chair, best look up your chairing duties in the conference system (Conftool) or the app.