Session Information
99 ERC SES 04 E, Assessment, Evaluation, Testing and Measurement
Paper Session
Contribution
This ethnographic study aims to explore the development of one school’s assessment culture, focusing on how teachers’ beliefs related to students’ learning and assessment as well as their classroom practices have been changing while school is introducing the assessment innovation - the System of Assessing Student’s Individual Progress (the SASIP). The introduction of the SASIP was conditioned by the attempt to improve the overall assessment practices in schools following the recommendation of the Ministry of Education. The school’s administration has responded to the researcher’s proposal sent through Vilnius municipality Education office to all city schools inviting them to work jointly on the improvement of school’s assessment practices. The readiness of this school to implement change was noticeable in the meetings with administration and convinced the researcher to select this particular school out of five other candidate schools. Since both, the school representatives and the researcher, shared the believe in the capacity of the community to participate actively in the project that is meant to improve their daily practices (Kemmis et al., 2014), the Participatory action research approach has framed the process of introducing the SASIP in the school.
Nowadays, schools are required to implement competence-based curriculum. While the vision of outcomes of such curriculum seems compelling, its successful implementation depends on how teaching, learning, and assessment are transformed (Marope et al., 2017). Due to its judgemental nature the assessment gains the power to influence both, teaching and learning practices, and either support or diminish the outcomes of competence-based curriculum. Any attempts to rethink assessment and its social meaning confront hidden teachers’ beliefs (Shepard, 2000), the unconsciously accepted unwritten rules rooted in school’s cultural practices.
M. Birenbaum (2014) discusses school culture through assessment lenses and distinguishes between grading oriented testing culture (TC) adopted by schools that consider teaching and assessment as separate activities and prioritize accountability function while in learning oriented assessment culture (AC), “the assessment becomes the glue that holds pieces of learning process together” (Graue, 1993). AC is conceptualized through complex system framework and includes two interrelated learning systems, namely classroom learning and teacher professional learning.
Constructivist approach defines learning as active process of meaning making, the new knowledge is built on the basis of the existing foundations, as J. Dewey (1910, p. 39) puts it ’facts…do not constitute, even when combined, reflective thought…they must be arranged with reference to one another and with reference to the facts on which they depend for proof’. The studies on effect sizes related to learning outcomes provide convincing evidence that formative assessment (FA) strategies like goal setting, self-reported grades, providing effective feedback, scaffolding, questioning, peer tutoring are the prerequisites of learning (Black &Wiliam, 1998; Hattie &Timperley, 2007).
Teachers’ professional learning contributes to the development of AC in schools. The most effective professional development is placed into the context of particular school and relates to day-to-day classroom challenges (Birenbaum, 2014), thus by cultivating professional learning communities within schools it is possible to improve student learning through changing teaching and classroom practices (Harris &Jones, 2010).
Despite the convincing arguments in favour of AC, the changes in assessment practice have been difficult to sustain (Gardner et all, 2008), e.g. in Lithuania, the External school evaluation reports indicate that schools do not make good use of the FA and fail to address different learning needs in order to ensure every students’ progress (NŠA, 2019). Thus, the study of how the school develops the AC that takes into consideration the peculiarities of the local context could contribute to the research on school improvement.
Method
Critical theory provides grounds for the study that is in line with M. Apple (2012, p. 18) ideas on the relationship of power and culture and his call to examine ’the day to day interactions and regularities of the hidden curriculum that tacitly taught important norms and values’. It also justifies the selection of PAR strategy to frame the project activities of introducing assessment innovation in school in a way that helps people understand and transform their practices (Kemmis et al., 2014). The aim of the critical research is related to the exploration of social inequalities, the nature of social structures, power, culture, and human agency (Carspecken, 1996). When assessment is seen only as the estimation of someone’s skills or knowledge, it implies power relations that contribute to social and cultural reproduction in education. The attempt to challenge this view has become central to this research and determined its epistemology. The culture is approached through the ethnographic analysis of participants’ experiences. This approach is helpful when reconstructing what has been going on in school in terms of assessment based on the first-person viewpoint of participants. Moreover, the viewpoint of the researcher is also significant for the reconstruction because both participants and researcher behave, in terms of their beliefs and desires, emotions and purposes “(Agar,2013, p. 45). When the culture is explained in relation to the critical theory, the validity claims must meet certain standards to avoid bias, as well as to be rooted in the democratic principles that are necessary to explore the concept of truth (Carspecken, 1996). I have been spending 18 month in the research school. I act as a consultant and a critical friend. We have been jointly working with staff on the development and implementation of the SASIP. I have organized tree training sessions, had some interviews with volunteer teachers, received a permission from some teachers to observe their lessons and participate in the weekly staff meetings where they discuss students’ progress. The data generated from participant observation and interviews have been transcribed and analysed in compliance with Carspecken’s recommended five critical ethnography stages. Up to this date, Stage One (Compiling the primary record) and Stage Two (Preliminary reconstructive analysis) have been in progress. I will further continue Stages One and Two and will start Stages Three (Dialogic data generation) Four (Describing system relations) Five (System relations as explanations of findings) in the upcoming months.
Expected Outcomes
The data about everyday interactions between and among students, teachers, and administrative staff collected through observation and interviews in 12 months period while school was implementing PAR to introduce assessment innovation will lead to better understanding about the scope and nature of transformations needed for developing learning oriented assessment culture in schools. The comprehensive account of school’s experiences will provide ’a conceptual framework that facilitates understanding of social action at both empirical and theoretical level’ (Pole &Morrison, 2003, p.4). The length of study, the combination of different methods for collecting and analysing data, the richness of data sources are the triangulation strategies used in this study. The aim of triangulation is to increase the “ecological validity that represents the degree to which the behaviours observed and recorded in a study reflect the behaviours that actually occur in natural settings” (M. Agar, 2013, p. 26) and produce findings that can be generalized to other schools of similar context as the research school. The empowering nature of the PAR will contribute to teachers’ agency and school’s capacity to adapt to the new demands that schools are meeting. The cooperation between schools and academics has to continue in order to inform education policy and make schools better places for students.
References
Agar, M. (2013). The Lively Science. Remodeling Human Social Research. Minneapolis: Mill City Press. Apple, M. W. (2012). Education and Power. New York: Routledge. Birenbaum, M., Kimron, H., & Shilton, H. (2011). Nested contexts that shape assessment for learning: School-based professional learning community and classroom culture. Studies in Educational Evaluation, 37, 35–48. Birenbaum, M. (2014). Conceptualizing assessment culture in schools. In C. Wyatt-Smith, V. Klenowski, & P. Colbert (Eds.). Designing Assessment for Quality Learning, 1, 285-302. Black, P., & Wiliam, D. (1998). Inside the black box: raising standards through classroom assessment. The Phi Delta Kappan, 80(2), 139-148. Carspecken, P. F. (1996). Critical ethnography in educational research. A theoretical and practical guide. New York: Routledge. Dewey, J. (1910). How we think. Boston, New York, Chicago: D.C. Heath & Co Publishers. Gardner, J., Harlen, W., Hayward, L., Stobart, G.(2008). Changing Assessment Practice: Process, Principles and Practice. Retriewed from https://www.researchgate.net/publication/323245019_Changing_Assessment_Practice_Process_Principles_and_Practice/stats Graue ,M. E. (1993). Integrating Theory and Practice Through Instructional Assessment. Educational Assessment, 1:4, 283-309, DOI: 10.1207/s15326977ea0104_1. Harris A., & Jones, M. (2010). Professional learning communities and system improvement. Improving Schools, 13(2), 172–181. doi: 10.1177/1365480210376487 Hattie, J., & Timperley, H. (2007). The power of feedback. Review of Educational research, 1, 81–112. Kemmis, S., McTaggart, R., Nixon, R., (2014). The action research planner. Doing critical participatory action research. Springer. Marope, M., Griffin, P., & Gallagher, C. (2017). Transforming teaching, learning and assessment. To support competence-based curricula. UNESCO: International Bureau of Education. Nacionalinė švietimo agentūra. (2019). Bendrojo udgymo mokyklų išorinio rizikos vertinimo rezultatai. Retrieved from https://www.nsa.smm.lt/wp-content/uploads/2020/10/9_isivertinimas_ugdymas.pdf Pole, C., & Morrison, M. (2003). Ethnography for education. Glasgow: Open University Press. Shepard, L. A. (2000). The Role of Assessment in a Learning Culture. Educational Researcher, 29(7), 4–14. doi:10.3102/0013189X029007004. Wiliam, D. (2011). Embedded formative assessment. Bloomington: Solution Tree Press.
Search the ECER Programme
- Search for keywords and phrases in "Text Search"
- Restrict in which part of the abstracts to search in "Where to search"
- Search for authors and in the respective field.
- For planning your conference attendance you may want to use the conference app, which will be issued some weeks before the conference
- If you are a session chair, best look up your chairing duties in the conference system (Conftool) or the app.