Impact of ISO 9001 STANDARDS in educational centres. Comparative study between teachers and management team.

Author(s):

Jesús Miguel Rodríguez-Mantilla(presenting / submitting)Francisco José Fernández-Cruz María José Fernández-Díaz

Conference:

ECER 2017

Network:

11. Educational Improvement and Quality Assurance

Format:

Paper

Session Information

11 SES 04, Standardization Tests vs. Other Assessments

Paper Session

Time:

2017-08-23

09:00-10:30

Room:

W2.10

Chair:

Jana Poláchová Vaštatková

Contribution

In many countries the implementation of Quality Management Systems (QMS) (ISO STANDARDS, Total Quality Management Models, and other alternatives) in educational organisations has increased (Chen, Chen, & Chen, 2013; Duque, 2013; Mehta, Verma, & Seth, 2013). Various administrations and the organisations promoting these models defend the need and relevance of the management systems associated with these models in order to set up continuous improvement plans. However not everyone is a major advocate of these models and management systems. Detractors also strongly criticise and challenge their relevance in improving organisations, especially in the education sector (Doherty, 2008).

This criticism has gradually decreased with time due to the positive experiences in many educational institutions. Nonetheless, the supposed improvement generated by these systems requires proof and evidence. Until now barely any references could be found in literature and the limited evidence from studies has not provided identical results on the usefulness of quality management systems in schools (Stensaker, Langfeldt, Harvey, Huisman & Westerheijden, 2011). While some studies seem to show considerable improvements in education (Cantón Mayo & Arias Gago, 2009; Ramírez García & Lorenzo, 2009), other papers indicate their effects are irrelevant or even detrimental for education centres.

Without going into the requirements of implementing a QMS, the improvement that these systems seek to promote is not based on immediate results, but rather medium to long-term sustainable changes. This transformation cannot only take place at the start of the implementation, not even during the first years, but rather it should be consolidated over time, as a consequence of the progressive implementation of the system. These are the medium-long term effects. In this sense, it is necessary to differentiate immediate results from those generated over time as a consequence of implementing the system and improvement plans, through interaction with other factors in the same context, centre or organisation. In this paper, it has been termed the latter “impact”, to differentiate it from the immediate results referred to above (Fernández-Díaz, 2013).

There is no doubt that the arguments and procedures required for the implementation of these QMS appear to indicate their efficacy, as has been expressed by those who, with a greater or lesser level of confidence, decided at one point to set up a management system. The correct implementation of QMS in schools has an impact in different areas. Through the specialized literature and the documentation of the different QMS (Rodríguez-Mantilla & Fernández-Díaz, 2013; Fernández-Cruz, Rodríguez-Mantilla & fernández-Díaz, 2015), large dimensions can be identified where the potential impact of QMS is to be assessed, such as: the vertical and horizontal communication system (among teachers, management team, families, etc.); the management system of the school; the quality of the interpersonal relationships between teachers, students, families and the support and acknowledgments received by teachers for their work; the teaching-learning process; the satisfaction of students, families, teachers, etc.; the relations maintained by the school with other schools and institutions, etc. However, schools are organizations where different agents interact, so it is questionable whether the improvement brought about by the introduction of a QMS is perceived in a similar way by the different groups of people that make up the school.

Given this situation, the main aim of this paper is to compare the assessments that teachers and management team have on the impact of the implementation of a QMS (ISO 9001 STANDARDS) in schools with over 3 years of implementation.

Method

The design used for this study was non-experimental, exploratory, along the lines of ex post facto studies. To select the study sample, the school was required to have implemented ISO 9001 STANDARDS for at least 3 years. Thus, the sample was composed of a total of 2,185 subjects (1,884 teachers and 301 members of the Management Team -Head Masters, Heads of Department, Area Coordinators and Quality Managers-) belonging to a total of 85 schools in 4 Autonomous Communities of Spain (Madrid, Andalusia, Valencia and Castilla y León). To measure the impact the ISO 9001 STANDARDS may have had on schools and analyse the differences in perception between teachers and members of the management team, we used an ad hoc questionnaire where subjects answered 93 items on a 0 to 4 Likert-type scale. The tool evaluated 7 major dimensions: Communication System (9 items, scale 0-36), Management System (19 items, scale 0-76), Support and Recognition (8 items, scale 0-32), School Climate (17 items, scale 0-68), Teaching Process (21 items, scale 0-84), Satisfaction (4 items, scale 0-16) and school's External Relations (15 items, scale 0-60). The tool's reliability analysis showed excellent results (Cronbach's alpha > .93 overall and in each dimension). To secure subject participation, we arranged an initial interview with the school Head Masters, in which we explained the study objective and procedure. To administer the questionnaires, which took place during the first semester of 2016, members of the research team visited the schools on the appointed dates, ensuring complete anonymity and confidentiality of the assessments and results. To analyse data, we used the software package SPSS 24. First, we conducted a descriptive analysis of each dimension evaluated. Next, we established differentials in the dimensions between teachers and members of the management team (using a t-test for independent samples, ANOVA and Scheffé, with a significance level of .01 and calculating the effect size with Cohen's d -Cohen, 1992-).

Expected Outcomes

Descriptive studies indicate that ISO 9001 STANDARDS show a medium impact on the dimensions of Communication, Management, Teaching Process and External Relations. However, we found a medium-low impact on the dimensions Support and Recognition, Climate, and Satisfaction. Differential studies showed significantly higher values (p<.01) among management team members compared to teachers in all dimensions, except External Relations, where no differences were found. Specifically, all members of the management team (Head Masters, Heads of Department, Area Coordinators and Quality Managers) placed a significantly higher value than teachers on the impact which ISO 9001 STANDARDS have had on the Management System and Satisfaction (with a moderate effect, Cohen's d = .43). In the case of management, results may be due to a greater knowledge and use of school management processes by the management team, an area less understood by teachers. On the other hand, only Head Masters, Heads of Department and Coordinators presented significantly higher values than teachers for school Communication System, Support and Recognition (moderate effect, Cohen's d = .40), Climate and Teaching Process (small effect, Cohen's d =.32). In these dimensions, teachers and Quality Managers had similar assessments. Therefore, the management team considered that, particularly, School Communication and Support and Recognition of teachers have improved to a greater extent thanks to the implementation of ISO 9001 STANDARDS, versus the perception of teachers and Quality Managers (who are usually also teachers at the school). Therefore, we can state that the ISO 9001 STANDARDS have a medium impact on schools with, in general, a higher assessment from the management team members.

References

Cantón Mayo, I., & Arias Gago, A. R. (2009). Metáforas y teorías implícitas de los directores escolares. Educación XXI, 28(1), 227-248. Cohen, J. (1992). Statistical power analysis. Current directions in psychological science, 1(3), 98-101. Fernández-Cruz, F. J., Rodríguez-Mantilla, J. M., & Fernández-Díaz, M. J. (2015). Evaluación del impacto de la aplicación del modelo de excelencia europeo (EFQM) en el clima esoolar y en la satisfacción de sus miembros. In Investigar con y para la sociedad (pp. 1179-1190). Chen, Y., Chen, J., & Chen, C. (2013). Teaching quality in higher education: An introductory review on a process-oriented teaching-quality model. Total Quality Management & Business Excellence, 25(1-2), 36-56. Doherty, G. D. (2008). On Quality in Education. Quality Assurance in Education, 16(3), 255-265. López Alfaro, P. (2010). El componente liderazgo en la validación de un modelo de gestión escolar hacia la calidad. Educação e Pesquisa, 36(3), 779-794. Duque, L. (2013). A framework for analysing higher education performance: students' satisfaction, perceived learning outcomes, and dropout intentions. Total Quality Management & Business Excellence, 25(1-2), 1-21. Fernández-Díaz, M. J. (2013). Evaluación del impacto para un cambio sostenible en las organizaciones educativas. Revista Española de Pedagogía, 254, 45-65. Mehta, N., Verma, P., & Seth, N. (2013). Total quality management implementation in engineering education in India: an interpretive structural modelling approach. Total Quality Management & Business Excellence, 25(1-2), 124-140. Ramírez García, A., & Lorenzo, E. (2009). Calidad y Evaluación de los Centros Educativos no Universitarios en Andalucía: Una adaptación de la EFQM. Revista Iberoamericana de Evaluación Educativa, 2(2), 22-45. Rodríguez-Mantilla, J. M. & Fernández-Díaz, M. J. (2013). Validación de un instrumento de medida para evaluar el impacto de los Sistemas de Gestión Calidad en centros educativos. In Investigación e Innovación Educativa al Servicio de Instituciones y Comunidades Globales, Plurales y Diversas: Actas del XVI Congreso Nacional/II Internacional Modelos de Investigación Educativa de la Asociación Interuniversitaria de Investigación Pedagógica (AIDIPE). Alicante, 4-6 de septiembre, 2013 (pp. 1621-1629). Stensaker, B., Langfeldt, L., Harvey, L., Huisman, J., & Westerheijden, D. (2011). An in-depth study on the impact of external quality assurance. Assessment & Evaluation in Higher Education, 36(4), 465-478.

Author Information

Jesús Miguel Rodríguez-Mantilla (presenting / submitting)

Complutense University of Madrid, Spain

Francisco José Fernández-Cruz

Francisco de Vitoria University of Madrid

María José Fernández-Díaz