Session Information
WERA SES 02 B, World-Wide Perspectives on Management and Leadership in Education
Paper Session
Contribution
The progressive application of intervention plans and programs in a very diverse range of organizations over the last several decades has brought about an increasingly frequent presence of publications making reference to the evaluation of objectives, effects, achievements, or impacts as outcomes of such intervention (Fernández Díaz, Rodríguez Mantilla & Fontana, 2014). Clearly, professionals have a great interest in determining just how efficient and effective those plans or programs are so that they can then assess the extent to which the effort spent actually produces the expected objectives or results, their magnitude, and even the factors determining their attainment.
For that reason, research is needed to find out the true impact, in which ‘impact’ is understood as being the set of relevant substantial changes produced over time following a specific action or intervention, and consolidated as sustainable changes in organizations, whether that be in the attitude of the personnel, in the way of working, in the climate, in the culture of the institution, in the planning, evaluation and in general in the development of the organizations (Fernández Díaz, 2013).
However, despite the benefits and improvements that certain intervention programs may have, there is little evidence of their impact on the organization (Rodríguez Ponce et al., 2011; De Vries, 2005; Stensaker et al., 2011). It therefore becomes necessary to introduce the culture of impact evaluation, i.e., of assessing the mid- and long-term effects that crystallize as sustainable changes, thereby modifying or enhancing their operation and their results. We estimate that there are three main conditions for evaluating impact in organizations understood in that way (Fernández Díaz, 2013; Valdéz, 2008):
- At least two measurements must be taken to be able to compare them and assess the change accordingly.
- Evaluation of impact involves measuring the changes observed in the application of an intervention.
- A minimum number of years of implementing the program at the organization is required to be able to analyze its impact.
Starting from those three premises, this paper proposes a model of evaluation that can be defined as:
- A multidimensional model: the dimensions that configure an educational organization must be identified and defined for an intervention to have an effect on it.
- A model with its own methodology of evaluative processes, defined as:
- Longitudinal study, although not always feasible, since processes and outcomes must be assessed over time at different moments.
- Measuring and collecting information that involves obtaining data from the very start of the intervention and across time to be able to compare the results. The information gathering techniques must be both quantitative and qualitative (Lambert, G & Ouedraogo, 2008).
- Analytic techniques: the information must be analyzed (with quantitative and qualitative techniques) to get a deep understanding of the impact, the magnitude of the change, and its associated factors as an outcome of implementing the program.
- Decision-making: along with knowledge of the achievement of the objectives of the processes and results, the analysis must help make decisions for improving the system and let these programs be applied to other organizations in the conditions most suited to their effectiveness.
The paper will justify the pertinence and consistency of the model and will present its application to a specific field, namely, the Quality Management System (QMS) at educational institutions.
Method
Expected Outcomes
References
De Vries, W. (2005). Calidad, eficiencia y evaluación de la educación superior. Madrid: Netbliblo. Fernández Díaz, M. J. (2013). Evaluación del impacto para un cambio sostenible en las organizaciones educativas. Revista Española de Pedagogía, 254, 45-65. Fernández Díaz, M. J., Rodríguez Mantilla, J. M. & Fontana, M. (2014). Impact of implementation of quality management systems on internal communications and external relations at schools. Total Quality Management & Business Excellence, DOI: 10.1080/14783363.2014.954365. See http://www.tandfonline.com/doi/abs/10.1080/14783363.2014.954365#.VMfIX0B0zDc (accessed on 13-1-2015) Lambert, G. y Ouedraogo, N.(2008) Empirical investigation of ISO 9001 quality management systems impact on organizational learning and process performance. Total Quality Management, 19:10, pp. 1071-1085. Rodríguez-Ponce, E., Pedraja-Rejas, L., Araneda-Guirriman, C., González-Plitt, M & Rodríguez-Ponce, J. (2011). El impacto del sistema de aseguramiento de la calidad en el servicio entregado por las universidades privadas en Chile. Ingeniare. Revista chilena de ingeniería, 19 (3), 409-419. Stensaker, B., Langfeldt, L., Harvey, L, Huisman, J, & Westerheijden, D. F. (2011). An in-depth study on the impact of external quality assurance. Assessment & Evaluation in Higher Education, 36 (4), 465- 478. Valdés, M. (2008) La evaluación de impacto de proyectos sociales. See http://scholar.google.es/scholar?q=La+evaluaci%C3%B3n+de+impacto+de+proyectos+sociales+de+Valdes&btnG=&hl=es&as_sdt=0%2C5 (accessed on 18-1-2015)
Search the ECER Programme
- Search for keywords and phrases in "Text Search"
- Restrict in which part of the abstracts to search in "Where to search"
- Search for authors and in the respective field.
- For planning your conference attendance you may want to use the conference app, which will be issued some weeks before the conference
- If you are a session chair, best look up your chairing duties in the conference system (Conftool) or the app.