Impact Evaluation of ISO 9000 and EFQM on the planning culture in pre-university educational centres: a comparative Study

Author(s):

Monica Fontana(presenting / submitting)Elvira Congosto Luna Rafael Carballo

Conference:

ECER 2017

Network:

11. Educational Improvement and Quality Assurance

Format:

Paper

Session Information

11 SES 09 B, Can We Trust Assessments to Improve Schools' Quality?

Paper Session

Time:

2017-08-24

13:30-15:00

Room:

W2.05

Chair:

Buratin Khampirat

Contribution

The purpose of schools has always been to offer quality education, especially in recent years. The concept of quality needs to be defined by each organisation according to the characteristics of their education project, mission and vision. Therefore, it is a continuously evolving concept, which must be prepared, shared and internalised by the members comprising the education community (López Cabanes and Ruiz Gimeno, 2004).

What are the factors that help an organisation provide quality education? Many studies have been carried out on this topic. As pointed out by Tiana Ferrer (1999), the OECD already established, back in 1991, a series of factors, including the necessary existence and joint definition of clear standards and goals, which entails strategic and continuous planning in the organisation, taking into account the results of the institutional evaluation conducted previously.

To address this quality requirement and improve the services provided, the implementation of quality management standards and systems, especially ISO and EFQM, has been one of the objectives of schools in the last few years.

One of the key dimensions, both in ISO and EFQM Standards, is the school management system that contemplates a planning culture, understood as a tool for diagnosis, analysis, reflection and collective decision-making and policies for teacher support and recognition. Authors such as Bernhardt (2013) and Davies and Ellison (2013), consider education planning a factor of organisation effectiveness and improvement. Continuing with Drucker (1980) "...planning does not mean knowing what decision I'm going to make tomorrow, but rather what decision I need to make today to achieve what I want tomorrow". A decision which must be made by designing strategies that enable its achievement. Therefore, it is an institutional management key, as it strives to improve education quality. In relation to policies for teacher support and recognition, these are crucial to generate positive feelings towards their work and greater engagement with the organisation (Villa, Troncoso and Díez, 2015).

There has been a lot of research on the implementation of Quality Management Systems to improve schools (López Alfaro, 2010; Cantón Mayo, I. & Arias Gago, A. R., 2009; Ramírez & Lorenzo, E., 2009: Martínez Mediano & Riopérez Losada, 2005; Quong, T & Walker, A., 1996; Bradley, L. H. & Kloppenborg, T. J., 1994) and also on the partial results achieved by the ISO 9000 Standards and the EFQM Model. However, there is a gap in the available scientific literature regarding the consolidation of results over time and the culture of quality created once the system has been internalised by everyone involved in the institution (Detert, Schroeder & Mauriel, 2000), especially in the field of education. Further research is needed to be able to appreciate the true impact, understood as substantial changes in the organisations that are sustainable over time, whether in staff attitude, work style, climate, school culture, planning, evaluation or development of the organisations (Fernández Díaz, 2012).

This study aims to compare the two models, since one of the main differences between the ISO Standards and the EFQM Model are the rules and regulations, which can lead to differences in terms of impact on generating a planning culture.

The purpose of this paper is to analyse the difference between two most frequently used quality management approaches implemented by firms ISO 9000 and EFQM (Bayo-Moriones, Merino Díaz, Escamilla de León y Selvam 2011; Heras, Casadesús and Marimon, 2011; Marimon, Heras y Casadesús, 2009; Arrizabalaga and Londeta, 2007; Senlle and Gutierrez, 2005), in terms of the impact on adoption of planning culture in pre-university schools.

Method

We conducted a non-experimental, exploratory, ex post fact study, with the participation of 179 Spanish schools, public, private with state subsidies and private, located in the Autonomous Communities of Andalusia, Castilla y León, Madrid and Valencia, which have been applying the ISO 9001 Quality Management System or the EFQM Model for at least three years. We ruled out schools that had implemented both systems. The analysis was conducted in the same period and applying the same methodology, but in a new way, using a sample of teachers and headmasters from different schools that had implemented one of the two systems. We conducted a survey of 2487 teachers and 414 headmasters or the person in charge of Quality in the school. Initially, we designed and validated an evaluation model defining the main dimensions to be analysed in order to determine the impact. These dimensions were then divided into subdimensions which, in turn, were then split into indicators that were translated into the items that were included in the questionnaire completed by the education community participating in the study. One of the dimensions was, precisely, School Management System, in which we distinguished between "Planning Culture" and "Support and Recognition Policies". In this dimension, the questionnaires had twenty-six items on a Likert scale from 1 to 5: 18 on "Planning Culture" and 8 on "Support Policies". Data Analysis Statistical data processing was conducted with the software package SPSS 24. We carried out both a technical analysis of the tool (Reliability with Cronbach's alpha) (Maydeu-Olivares & McArdle, 2005; Pardo & Ruiz, 2002), and a descriptive study of the answers (in both subdimensions). These analyses were complemented with a differential study, for each item (applying the Student's t-test) (Hair et al., 2005; Stevens, 2009), of the effect caused according to the Quality Management System used by the school: ISO or EFQM. We also analysed the differences in other characteristics, such as: type of school, Autonomous Community, number of years the Quality Management System has been implemented, position, size of teaching staff and number of students. In this paper we only highlight the differences in the long-term results in the dimension of planning culture and support and recognition policies that the two models adopted. It should be noted that this paper is a part of a wider research project financed by the Spanish Ministry for Education and Science under the system of R&D grants.

Expected Outcomes

The analyses carried out showed an excellent level of reliability for the whole of the management elements (α = 0.959) with homogeneity levels no lower than 0.35 and mostly around 0.6 (maximum HI = 0.752). In general, the schools surveyed believed implementing the QMS (regardless of the system used) had led to considerable improvement in the school planning model, in the way it organised and planned all activities. The things highlighted included, obviously, the minutes taken at all meetings and the agreements reached (typical of this type of QMS), as well as improvement of the general planning system and the school's Annual Programming. In this sense, the schools using the EFQM system expressed the highest score with the changes in general planning at the school, in the Annual Programming and evaluation of extracurricular activities. Those using the ISO system noted above all the impact on preparing reports of all the activities conducted. In contrast, participants did not consider relevant the impact that these systems have had on support and recognition for teachers in their work, or on incentives in the development of improvement and innovation programmes. Although the level of impact is not very high in general, the opinion of those who have used the ISO system is particularly low in all items, while those who used the EFQM was medium. These results appear to underline the differences among the target schools between the two systems: ISO is more focused on quality assurance and EFQM more on improving operations.

References

BAYO-MORIONES, A., MERINO, J., ESCAMILLA, S. & SELVAM, R. M. (2011). The impact of ISO 9000 and EFQM on the use of flexible work practices. International Journal of Production Economics, 130, (1) 33–42. BERNHARDT, V. (2013). Data analysis for continuous school improvement. New York: Routledge. Retrieved from http://books.google.es/books?hl=es&lr=&id=STtnAQAAQBAJ&oi=fnd&pg=PP1&dq=school+improvement+and+strategic+planning&ots=TEeaYnioJW&sig=nBjmVTDhE018T3G_2MnXcdjoGlc DAVIES, B. & ELLISON, L. (2013). The new strategic direction and development of the school: Key frameworks for school improvement planning. New York: Routledge. Retrieved from http://books.google.es/books?hl=es&lr=&id=XsvB0B6QCQkC&oi=fnd&pg=PP1&dq=school+improvement+and+strategic+planning&ots=LJ1NF-djg&sig=AG_vfC18wDGPV__hJseLAirjiPo DETERT, J. R., SCHROEDER, R. G. y MAURIEL J. J. (2000). A Framework for Linking Culture and Improvement Initiatives in Organizations. Academic Management Review, 25 (4), 850-863. DRUCKER, P. (1980). Managing in Turbulent Times. London: Heinemann. FERNÁNDEZ DÍAZ, M. J. (2013). Evaluación del impacto para un cambio sostenible en las organizaciones educativas. Revista Española de Pedagogía, 254, 119-140. HERAS-SAIZARBITORIA, I; CASADESÚS, M.; y MARIMÓN, F. (2011). The impact of ISO 9001 standard and the EFQM model: The view of the assessors. Total Quality Management, 22(2), 197-218. LÓPEZ ALFARO, P. (2010). Factores de calidad educativa, Estudios Pedagógicos, XXXVI, Nº 1, 147-158. MARTÍNEZ-MEDIANO, C. y ARRIBAS, J. A. (2016). Valoración de la Aplicación de los Sistemas de Gestión de la Calidad ISO 9001 en los Centros Educativos a través de un Cuestionario, Meta: Avaliação, Rio de Janeiro, v. 8, n. 23, p. 326-358. MARTÍNEZ MEDIANO, C. Y RIOPÉREZ LOSADA, N. (2005). El Modelo de excelencia de la EFQM y su aplicación para la mejora de los centros educativos. Educación XXI: Revista de la Facultad de Educación, 8, 35-66. MAYDEU-OLIVARES, A. & MCARDLE, J. J. (2005). Contemporary Psychometrics. A Ferstschrift to Roderick P. Mc Donald. Mahwah, NJ: Lawrence Erlbaum. OCDE (2001). Schooling for tomorrow: Trends and scenarios. Paris: CERI-OECD. PARDO, A. y RUIZ, M.A. (2002). SPSS 11: Guía para el Análisis de Datos. Madrid: McGraw-Hill QUONG, T. y WALKER, A. (1996). TQM and school restructuring: A case study. School Organisation, 16(2), 219-231. STEVENS, J.P. (2009). Applied Multivariate Statistics for Social Sciences (5th Edition). Routledge Academic. TIANA FERRER, A. (1999). La evaluación y la calidad: dos cuestiones de discusión. Ensaio. Avaliação e Políticas Públicas em Educação, vol. 7, nº 22, pp. 25-46. VILLA, A., TRONCOSO, P. E. y DÍEZ, F. (2015). Estructura latente y fiabilidad de las dimensiones que explican el impacto de los sistemas de gestión de calidad en los centros educativos. Revista de Investigación Educativa, 33(1), 65-82.

Author Information

Monica Fontana (presenting / submitting)

Universidad Complutense de Madrid

Madrid

Elvira Congosto Luna

Facultad de Educación de la Universidad Complutense de Madrid

Métodos de Investigación y Diagnóstico en Educación

Pozuelo de Alarcón (Madrid)

Rafael Carballo

UCM