Session Information
99 ERC SES 07 F, Innovations and Insights in Educational Measurement and Evaluation
Paper Session
Contribution
In Italy, from 2018 the national assessment of learning in the area of reading comprehension will provide individual descriptive feedback on the level of proficiency obtained in standardized tests for each student at the end of the school year in lower and upper secondary schools. This shift from a feedback of data exclusively at the level of the system (national, school, class) to a feedback also at the level of the individual student represents a change of perspective in which the student assumes a new position, which is even more central because the evaluation of the education system does not only use its results as information on the "well-being" of the system, but refers directly to the student the level achieved in reading comprehension, in relation to his/her school grade and in specific areas of knowledge and skills, in relation to national standards.
With the present exploratory study, we aim to reflect on the possible impact of such descriptive feedback on teaching and school action in the context of an impact evaluation (Stern et al. 2016; Stame 2016). The reference paradigm of social constructivism, within which this research is located, highlights the need to understand the complexity of this phenomenon through the direct experience of the people involved in the feedback process, specifically students and teachers. The aim of this qualitative study (Berg, 2001) is to investigate what, if any, effects descriptive feedback from national standardized tests can have on reading comprehension in the school context, specifically in the classroom (Winstone, Mathlin e Nash, 2019; Brooks, Carroll, Gillies e Hattie, 2019; Molloy Bound e Henderson, 2020). This aim was broken down into two main research questions: 1) How do students perceive the descriptive feedback from national standardized tests that they receive at the end of the school year on their reading comprehension? 2) How can descriptive feedback from national standardized tests in reading comprehension, from the experience of those involved (students and teachers), contribute to an educational impact on teaching and learning processes?
Method
To answer the research questions, a constructivist approach will be used to interpret the phenomenon of interest from the experiences and perceptions of the participants through thematic analysis of interviews. This method makes it possible to identify and analyze recurring themes in the qualitative data collected to construct meanings from individual narratives. The qualitative data collection process therefore involved individual semi-structured interviews (Denzin & Lincoln, 2000) with teachers and students in their final year of secondary school. The interviews were conducted online. The participants in the research were about 15 teachers and about 10 secondary school students, as they were directly involved in the teaching action related to the national standardized tests. The data collected were analyzed using the thematic analysis model (Braun and Clarke, 2021; Attride-Stirling, 2001; Pagani, 2020), according to an approach that made it possible to arrive at general explanatory themes of the phenomenon
Expected Outcomes
The analysis of the interviews identified some specific themes relating to: perceptions of the national standardized tests; teachers' awareness; the opportunities for descriptive feedback from the national test; students' perceptions; suggestions for improvement; pedagogical value. In particular, the data suggests that the national standardized test of text comprehension is poorly integrated into the educational context and is perceived primarily as a regulatory requirement. The individual feedback currently provided at the end of the school year does not influence school performance, and teachers are often unaware of the content of the level descriptions, focusing more on the numerical class and school averages discussed in data reporting meetings. However, the potential of the feedback to inform pedagogical action within the classroom context is recognized. In particular, students appreciate the clarity of the descriptive levels and find them useful in understanding their own competence in text comprehensio
References
Attride-Stirling, J. (2001). Thematic Networks: An Analytic Tool for Qualitative Research. Qualitative Research, 1, 385-405. http://dx.doi.org/10.1177/146879410100100307 Berg, B. L. (2001). Qualitative research methods for the social sciences (4th ed). Allyn and Bacon, Boston. Braun, V. and Clarke, V. (2021) Thematic Analysis: A Practical Guide. Sage, London. Brooks, C., Burton, R., Van Der Kleij, F., Carroll, A., & Hattie, J. (2021). Towards student-centred feedback practices: evaluating the impact of a professional learning intervention in primary schools. Assessment in Education: Principles, Policy & Practice, 28(5–6), 633–656 Denzin, N. K., & Lincoln, Y. S. (Eds.). (2000). Handbook of qualitative research (2nd ed). Sage Publications. Looney, J. W. (2011), “Integrating Formative and Summative Assessment: Progress Toward a Seamless System?”, OECD Education Working Papers, No. 58, OECD Publishing. Molloy, E., Boud, D., & Henderson, M. (2020). Developing a learning-centred framework for feedback literacy. As
Update Modus of this Database
The current conference programme can be browsed in the conference management system (conftool) and, closer to the conference, in the conference app.
This database will be updated with the conference data after ECER.
Search the ECER Programme
- Search for keywords and phrases in "Text Search"
- Restrict in which part of the abstracts to search in "Where to search"
- Search for authors and in the respective field.
- For planning your conference attendance, please use the conference app, which will be issued some weeks before the conference and the conference agenda provided in conftool.
- If you are a session chair, best look up your chairing duties in the conference system (Conftool) or the app.