Session Information
09 SES 16 B, Economic Competencies / Impact of Assessment Reforms
Paper Session
Contribution
In the last two decades educational research has recognized (and supported) the large diffusion of a new assessment culture with a strong emphasis on the integration of assessment and instruction and the required alignment of teaching, learning, and assessment (Fullan 2007; Darling-Hammond & Bransford, 2015; Gardner et al., 2010; Townsend & Bates, 2007).
The interest reserved to students’ learning assessment as well the rapid expansion of high stakes, external, and standardized assessment invested the majority of countries worldwide. Therefore, classroom assessment practices have been exposed to deep changes. First of all, following the theoretical perspective of Assessment for Learning (AfL) and the socio-cultural views of learning (Baird, 2011; Black & Wiliam, 1998; 2004; McMillan, 2001), classroom assessment practices shifted from an objective measure of the acquisition of knowledge towards a view of assessment as a social practice (Bennett, 2011) that provides continual information in support of student learning (Filsecker & Kerres, 2012; Kingston & Nash, 2011; Lund, 2008). Secondly, large-scale assessment has been progressively used as a policy instrument to measure educational quality and as a key aspect in the education governance.
Finally, teachers’ data use has become crucial «to make many types of decisions from school improvement to classroom and instructional decision-making» (Mandinach & Gummer, 2016: 1). Moreover, several research studies (Boardman & Woodroof, 2004; Brookhart, 2011; McMillan et al., 2002) have already demonstrated that teachers do not use assessment data during their decision-making and their teaching practice. If on the one hand, teachers recognize the importance of using evidence and data and not only personal opinions, preferences, and intuitions, on the other hand, they should be able to manage several sources of data, including data gathered through large-scale assessment.
As Black and Wiliam (1998; 2004) argue, it is impossible implementing reform measures concerning assessment without considering contextual and cultural embeddedness of assessment school practice. To understand how to promote substantive and effective changes it is necessary exploring school contexts. Moving from this assumption the present paper focuses on what conceptions the Italian teachers have of assessment. More specifically:
- What conceptions primary and middle school teachers have of assessment?
- How do teachers practice assessment in the classroom?
- How do they use different kinds of data for decision-making?
Although the consolidated international scientific debate, the Italian teachers continue to perceive large-scale assessment as improper: the national survey Teacher Questionnaire administrated to Italian and Math teachers, since 2013, by INVALSI (Italian National Institute for School Evaluation and Assessment of Learning) confirmed how national large-scale assessment results are not considered (and used) in a functional way to support school system and teaching practice.
This paper aims to focus on what conceptions teachers have of assessment (both at the classroom and large-scale level) while deep changes occur in the Italian school system. Results of the qualitative analysis allow shedding light not only on teachers’ assessment conceptions, but also on the effects that conceptions have on assessment (Brown, 2004), teaching practice, use of data, and decision-making.
The paper also tries to identify:
- What challenges must be addressed to contribute to teacher education for assessment;
- What improvement actions can be suggested and shared in order to implement a comprehensive and coherent school system.
Method
This study is qualitative in nature. In order to assure a number of methodological possibilities within the interpretative paradigm, a phenomenological methodology was used to explore teachers’ conceptions of assessment. All participants were informed of the purpose of the study, assured of anonymity and confidentiality and voluntarily consented to participate. Semi-structured interviews were used. Altas.ti software has been helpful in structuring, saving and copying the raw material. Descriptive codes were compared with each participant. The interviews have been coded following two steps: • Analysis of each interview to identify emerging issues; • Comparison between interviews in order to find possible common themes. Specifically, a comparison has been made respect to different variables (e.g., teachers’ age of service; teaching subject matter). Results are based on interviews with 70 teachers (selected from a population of 5 schools in the XXXX school district). Only teachers of Math and Italian have been considered because they represent the target population of the INVALSI large-scale program. The interview, composed by 15 questions, was divided into four sections: • Respondents’ biographical sketch. This section concerned socio-demographic variables. It gathered information on teachers’ education path and their professional careers (sex, age, years of experience, teaching experience); • Assessment conceptions and representations. Questions in this section gathered information about teachers’ conceptions of assessment; • Assessment and teaching practice. This third section was about the relationship between teaching practice and assessment: How is the relationship between teaching and assessment? How do they assess during the daily teaching work?; • Use of data. This last section aimed to analyse if, and how, teachers use data, gathered through both classroom assessment and large-scale assessment, for their decision-making. The data analysis followed a three steps process: open coding, axial coding, and selective coding. The analysis revealed four main dimensions: • Teacher education and teacher learning needs (courses attended, usefulness, transferability); • Conceptions of assessment (what and why teachers assess, which criteria they use); • Assessment practice (assessment design, process, and strategies); • Data-use (what kind of data are used and for which purposes, distinction among large-scale assessment, classroom assessment and school system evaluation). A cross-checking procedure of independently-coded data was used by the author and two research assistants. Cohen’s Kappa index has been performed as a measure of the agreement between researchers. The Kappa value at all (.75) and the K value of the coding categories (among .61 and .75) indicate a good level of agreement and an adequate level of inter-coder reliability.
Expected Outcomes
Although collected information is sensitive, some inferences about teachers’ points of view about assessment in the Italian school context can be made. There are no substantial differences in teachers’ conceptions of assessment. No differences have been highlighted between teachers. Gender, age, subject matter, and professional experience do not affect answers. Teachers are not able to provide a definition of assessment that goes beyond the mere dimension of measure of learning achieved by their students. They need to guarantee a more fair assessment but are not able to use adequate tools and instruments. Interviewed teachers appear very worried about the large-scale assessment program: Italian and Maths teachers confirm to perceive the pressure of these tests in their teaching practice. Teachers do not see the real value of this large-scale assessment and are scared about the idea of using students’ results for teachers’ performance appraisal. For this reason, most of them admit teaching to the test and cheating, although they recognize these are malpractices Teachers demonstrate conceptions partially influenced by a traditional view of assessment. While the national testing policy initiatives have been reframed following equity and social justice criteria and taking up the position that tests are fundamental in order to improve students’ learning outcomes, these results confirm how the Italian teachers are “in the darkness” about testing and reporting purposes, and are not sufficiently informed about the rationale of this kind of assessment. It is like if teachers do not see clear connections between the testing program, high stakes, and the classroom assessment practice. Such a small case study cannot be generalized to the whole Italian system. However, this study presents a unique opportunity to share and compare challenges that must be addressed to better understand teachers’ conceptions along schools levels in the context of school reform in Italy.
References
Baird, J. (2011). Does the learning happen inside the black box?. Assessment in Education: Principles, policy and practice, 18(4), 343-345. Black, P., & Wiliam, D. (1998). Inside the black box. London: Kings College. Black, P., & Wiliam, D. (2004). The formative purpose: Assessment must first promote learning. In M. Wilson (Ed.), Towards coherence between classroom assessment and accountability (pp. 20-50). Chicago, IL: NSSE & University of Chicago Press. Boardman, A. G., Woodruff, A.L. (2004). Teacher change and “high stakes” assessment: what happen to professional development?. Teaching and Teacher Education, 20(6), 545-557. Brookhart, S. M. (2011). Educational assessment knowledge and skills for teachers. Educational Measurement: Issues and Practices, 30(1), 3-12. Brown, G.T.L. (2004). Teachers’ conceptions of assessment: Implications for policy and professional development. Assessment in Education, 11(3), 301-318. Darling-Hammond, L., Bransford, J. (2015). Preparing Teachers for a Changing World. New York: Jossey Press. Filsecker, M., & Kerres, M. (2012). Repositioning formative assessment from an educational assessment perspective: A response to Dunn & Mulvenon (2009). Practical Assessment, Research and Evaluation, 17(16), 1-9. Fullan, M. (2007). The new meaning of educational change. New York: Teachers College Press. Gardner, J., Harlen, W., Hayward, L., & Stobart, G. with Montgomery, M. (2010). Developing Teacher Assessment. London: McGraw Hill. Kingston, N., & Nash, B. (2011). Formative assessment: A meta-analysis and a call for research. Educational Measurement: Issues and Practice, 30(4), 28-37. Lund, A. (2008). Assessment made visible: Individual and collective practices. Mind, Culture, and Activity, 15(1), 35-51. Mandinach, E.B., Gummer, E.S. (2016). Data literacy for teachers: Making it count in teacher preparation and practice. New York, NY: Teachers College Press. McMillan, J.H. (2001). Essential assessment concepts for teachers and administrators. Thousand Oaks, CA: Corwin Publishing Company. McMillan, J.H., Myran, S., & Workman, D. (2002). Elementary teachers’ classroom assessment and grading practices. The Journal of Educational Research, 95(4), 203-213. Townsend, T., Bates, R. (2007). Handbook of Teacher Education. Dordrecht: Springer.
Search the ECER Programme
- Search for keywords and phrases in "Text Search"
- Restrict in which part of the abstracts to search in "Where to search"
- Search for authors and in the respective field.
- For planning your conference attendance you may want to use the conference app, which will be issued some weeks before the conference
- If you are a session chair, best look up your chairing duties in the conference system (Conftool) or the app.