Session Information
11 SES 09 A, Quality of School Education
Paper/Ignite Talk Session
Contribution
In Europe, the use of school inspections to assess and hold schools accountable for goals related to student achievement and educational quality is well established. Some educational stakeholders also consider an inspection as a tool for improvement of quality and improvement of students’ learning and achievement (Ehren et al., 2013). To stimulate school improvement, inspectors assess schools in accordance with a set of preconceived standards and give feedback on the schools’ strengths and weaknesses during the school visits and in inspection reports (Ehren, 2016). Overall results of inspection research about whether or not school inspections contribute to school development purposes are far from conclusive (Husfeldt, 2011).
Given that feedback has a strong influence on teachers and the improvement of their teaching practice, and that providing feedback does not take place in most schools on a consistent basis (OECD, 2013), it is important to understand how school inspections can drive school improvement through the feedback that they provide to schools during inspection visits and/or in inspection reports. The following research questions were posed:
(1) How are teachers’ affective responses related to their cognitive responses in the context of a school inspection?
(2) Do affective responses mediate the relationship between teachers’ cognitive responses and their feedback acceptance?
The purpose of this study was to identify determinants of the acceptance of school inspection feedback at the teacher level. On the basis of a previous study (Quintelier et al., 2019), we included three primary categories of teachers’ affective responses – joy, anger, and sadness – as possible predictors of teachers’ feedback acceptance. In turn, we studied how these affective responses were influenced by different cognitive responses: inspectors’ expertise, inspectors’ trustworthiness, procedural justice, distributive justice, feedback constructiveness, feedback clarity, and feedback relevance. We hypothesised that teachers’ affective responses mediate the relationship between teachers’ cognitive responses and feedback acceptance.
The results of our literature review suggest that both cognitive and affective responses to feedback seem to be essential for altering teachers’ attitudes, perceptions, and behaviour.
Ehren et al. (2013) indicate that school inspection feedback is only expected to emerge as an improvement tool for schools when it is understood, accepted, and eventually used by actors in schools.
This is in line with the feedback process model of Ilgen et al. (1979) who suggests that an individual’s reaction to feedback depends mostly on feedback acceptance, stating this as the core mechanism of the feedback process. In this study, we define feedback acceptance as “perceptions about the accuracy of the inspection feedback received” (based on Anseel & Lievens, 2009).
Feedback research in general distinguishes cognitive responses and affective responses as significant factors for feedback acceptance (e.g. Brett and Atwater 2001; Leung et al. 2001).
Three main groups of cognitive responses (thoughts) have been widely discussed as significant factors influencing feedback acceptance in organisational psychology: (a) source credibility (source’s expertise and trustworthiness), (b) perceived violations of procedural and distributive justice (organisational justice), and (c) feedback characteristics (feedback sign, constructiveness, clarity, relevance; e.g., Brett & Atwater 2001; Greller & Herold 1975; Ilgen et al., 1979; Leung et al., 2001).
To analyse the affective responses of teachers (recipients’ emotional reactions to the feedback event) this study as well as recent studies in the educational setting (Bahia et al., 2013; Chen, 2016) uses the classification of emotions based on the research of Parrott (2001). While most researchers of emotions (e.g., Sander, 2013; Sutton & Wheatley, 2003) distinguish between positive (happiness and joy) and negative emotions (fear, anger, and sadness), Parrott (2001) divided six primary emotions – that is, love, joy, surprise, anger, sadness, and fear – into secondary and tertiary emotions.
Method
A survey of teachers’ perceptions regarding the above-mentioned aspects has been conducted. Using a path analysis, we tested the existence and strength of the relationships presented in the theoretical framework. The study population included every Flemish primary school inspected between January and November 2018. Data was collected from a total sample of 80 schools, encompassing 687 teachers, using self-report questionnaires. To ensure the content validity of the questionnaire, we examined the extent to which the theoretical framework regarding individuals’ feedback acceptance in organisational psychology and school inspection research was sufficiently elaborated upon in the survey (Cohen et al., 2011; Taherdoost, 2016). A pilot version of the questionnaire was handed to four content experts, resulting in a new questionnaire. The feedback gained from discussions with three teachers to examine the difficulty level and to understand whether the cognitive processes that the respondents were going through were in line with what the questionnaire intended to measure led to adaptations to the final questionnaire. The construct validity of the scales was tested by conducting a confirmatory factor analysis using software package lavaan in R (Rosseel, 2012). The fit indices that were taken into account to evaluate the validity of the instrument were the comparative fit index, the Tucker–Lewis index, and the root-mean-square error of approximation. We analysed the data by calculating the descriptive statistics and correlations for all variables, then intraclass correlations were calculated as well as the variances between and within schools. Finally, structural equation modelling was used to answer our research questions. This allowed for modelling the direct and indirect relationships between this study’s constructs. On the basis of our theoretical framework, we built a path model with the three affective responses as mediators between teachers’ cognitive responses and teachers’ feedback acceptance. Given the fact that we were analysing teachers within schools, the nested structure of the data was taken into account by the MLR estimator. Modification indices were examined to further optimise the model.
Expected Outcomes
Our results indicate that the majority of Flemish teachers take a positive stance towards school inspectors and the inspection process, and that the Flemish educational context is characterised by high acceptance ratings of the inspection feedback received. Positive perceptions of inspectors’ trustworthiness and distributive justice are of importance for reducing teachers’ emotions of anger and sadness regarding the inspection outcome. Teachers who trust the school inspector’s intentions and believe that the inspection outcome is a fair result of the inspection process, will express less emotions of anger and sadness. The importance of organisational justice is also shown in its direct relationship with feedback acceptance. Regardless of the emotions experienced, teachers who report higher levels of procedural justice are more likely to perceive the inspection feedback as accurate than teachers who report lower levels of justice. This is in line with the research of Leung et al. (2001), who found that feedback acceptance increased when recipients perceived feedback as correct. Teachers who reported higher levels of organisational justice are more likely to perceive the inspection feedback as accurate compared with teachers who reported lower levels of justice. Teachers who reported higher levels of anger are less likely to accept the inspection feedback. Our findings also confirm that feedback relevance is a critical success factor for feedback acceptance (Ehren & Visscher, 2008). Our results demonstrate that inspection feedback that is perceived to be relevant will likely lead to higher levels of feedback acceptance among teachers but will also engender higher levels of anger and sadness. In order to shape inspection systems with maximal impact, it is important that sufficient care is taken to enhance teachers’ perceptions of the psychometric quality and transparency of school inspections.
References
Anseel, F., & Lievens, F. (2006). Certainty as a moderator of feedback reactions? A test of the strength of the self-verification motive. Journal of Occupational and Organizational Psychology, 79(4), 533– 551. https://doi.org/10.1348/096317905X71462 Bahia, S., Freire, I., Amaral, A., & Estrela, M. T. (2013). The emotional dimension of teaching in a group of Portuguese teachers. Teachers and Teaching: Theory and Practice, 19(3), 275–292. https://doi. org/10.1080/13540602.2012.754160 Brett, J. F., & Atwater, L. E. (2001). 360° feedback: accuracy, reactions, and perceptions of usefulness. Journal of Applied Psychology, 86(5), 930–942. Chen, J. (2016). Understanding teacher emotions: The development of a teacher emotion inventory. Teaching and Teacher Education, 55, 68–77. https://doi.org/10.1016/j.tate.2016.01.001 Cohen, L., Manion, L., & Morrison, K. (2011). Research methods in education (7th ed.). Routledge. Ehren, M. C. M. (2016). Introducing school inspections. In M. C. M. Ehren (Ed.), Methods and modalities of effective school inspections (pp. 1–16). Springer International Publishing. https://doi.org/10.1007/ 978-3-319-31003-9_1 Ehren, M. C. M., & Visscher, A. J. (2008). The relationships between school inspections, school characteristics and school improvement. British Journal of Educational Studies, 56(2), 205–227. https://doi. org/10.1111/j.1467-8527.2008.00400.x Greller, M. M., & Herold, D. M. (1975). Sources of feedback: A preliminary investigation. Organizational Behavior and Human Performance, 13(2), 244–256. https://doi.org/10.1016/0030- 5073(75)90048-3 Husfeldt, V. (2011). Wirkungen und Wirksamkeit der exzernen Schulevaluation. Überblick zum Stand der Forschung. Zeitschrift für Erziehungswissenschaft. doi:10.1007/s11618-011-0204-5. Ilgen, D. R., Fisher, C. D., & Taylor, M. S. (1979). Consequences of individual feedback on behavior in organizations. Journal of Applied Psychology, 64(4), 349–371. https://doi.org/10.1037/0021-9010.64.4.349. Leung, K., Su, S., & Morris, M. W. (2001). When is criticism not constructive? The roles of fairness perceptions and dispositional attributions in employee acceptance of critical supervisory feedback. Human Relations, 54(9), 1155–1187. https://doi.org/10.1177/0018726701549002. OECD. (2013). Synergies for better learning. An international perspective on evaluation and assessment. In OECD Reviews of Evaluation and Assessment in Education. Paris: OECD Publishing. Rosseel, Y. (2012). lavaan: An R package for structural equation modelling. Journal of Statistical Software, 48(2), 1–36. https://doi.org/10.18637/jss.v048.i02 Taherdoost, H. (2016). Validity and reliability of the research instrument; how to test the validation of a questionnaire/survey in a research. International Journal of Academic Research in Management, 5(3), 28–36.
Search the ECER Programme
- Search for keywords and phrases in "Text Search"
- Restrict in which part of the abstracts to search in "Where to search"
- Search for authors and in the respective field.
- For planning your conference attendance you may want to use the conference app, which will be issued some weeks before the conference
- If you are a session chair, best look up your chairing duties in the conference system (Conftool) or the app.