This paper examines the change in teachers’ attitudes as a systemic reform package for the primary grades in schools is rolled out nationally. Alongside a significant investment in professional development and a new curriculum, a requirement to use criteria-based assessment practices is being introduced. This marks an important cultural shift away from previous assessment practices. These consisted of teachers’ subjective, norm-based interpretations of state standards that did not permit reliable or transferable appraisal of individualised learning to record and follow students’ progress objectively. Indeed, it is a stated aim of the country being researched, Kazakhstan, to set up a national database of its school students to not only track the introduction of per capita funding but also to encourage a big data approach that monitors students’ progress throughout their academic careers (Shayakmetova, 2017). The primary aim of this research, therefore, is to chart how large-scale national assessment reform is experienced over time by the teachers effecting it and to formulate theory that may explain its course. This leads to the following research question being posed:

What are the current and predicted attitudes of teachers stemming from the introduction of nationally imposed new models of assessment that move from a norm-based to criteria-based approach?

Examining this research question informs on the in-vivo events of systemic reform in schools in Kazakhstan but also applies elsewhere. This research informs on any educational system where a central drive in policy for measurable, student-centred learning-outcomes and the creation of big data is being enacted by stipulating teachers attune their assessment practices to reflect prescribed national expectations of attainment (Rust, 2002). Policies that promote a learning-outcomes system through teacher guidelines and other means have already been seen in Europe, Australia and the USA to negatively impact on the expectations of teachers’ practices and teachers’ liberty to apply professional judgement in interpreting state standards against curriculum aims (Sadler, 2005). To ameliorate this, a more recent trend has been to build on the learning outcomes based approach by adopting a Criteria-Based Assessment (CBA) approach that aligns with and acts alongside state standards (OECD, 2013). Underpinning the effectiveness of CBA is that teachers undergo professional development that enhances their understanding of the purposes of assessment. Alongside this, their capacity must expand to use a variety of formative and summative methods thereby equipping them to reliably assess; and, to combine the meeting of state standards with subject and grade-specific criteria (Koh, 2011).

Despite explicit connections between curricula and desirable outcomes, the unavoidable filter of teachers’ interpretations of criteria will always likely include a norm-based lens as students’ performances are compared one to another (Wiliam, 1993). Suppressing the natural urge to compare rather than objectively appraise may be a particularly difficult transition for post-Soviet nations; since there is no intermediary stepping stone of an outcomes-based approach (Lok, McNaught & Young, 2016). Furthermore, the required change in a teacher’s assessment mindset is positioned against the longstanding background (since 1937 according to Bolotov et al., 2016) of a purely norm-based classroom assessment approach. Therefore, since extant literature has no focus on the direct move from a norm-based to a criteria-based classroom assessment model, an exploratory method of enquiry is appropriate to record and then theorise how previous class-level learning goals are replaced by individualised targets. In particular, the likely transfer of emphasis from the teacher being the sole responsible agent for students attaining standards (acting as ‘controller’) to a shared responsibility between the learner and the teacher (acting as ‘conductor’)  is, as yet, an under examined area (Black & Wiliam, 2009). 

A mixed methods qualitative research design was used to address the research questions. This approach was chosen to garner detail through interviews and focus groups that supplemented findings from a qualitative survey question. A qualitative methodology was chosen to collect open data that would permit inductive interpretation of participants’ contributions for theory-generation (Willig, 2008). The theory would then be open to later examination in future, planned phases of research by means of quantitative methods and additional large-scale non-numerical data collection. The themes upon which an emergent theory is proposed along with the hypothesised theory itself are presented here. The research was structured in terms of the number of years’ experience that participants had with the new assessment practices. Therefore, the primary variable under examination was the grade that teachers taught since, starting with grade 1 in September 2015, the new assessment practices were being rolled out over time, grade-by-grade. This led to teachers of grade 3, at the time of data collection, being about to enter their third year with the new assessment practices having already used such in grades 1 and 2. Teachers of grade 4 have as yet no experience with the new assessment practices and those teaching grades 1 and 2 have, respectively, a small amount and just over a year. Open coding using thematic analysis (Braun & Clarke, 2005) of the entire dataset was applied to reveal themes related to the chronology of implementation. Thereby, a theory was inductively proposed that relates growing familiarity and increasing direct experience of new assessment practices to teachers’ attitudes towards them. Data contributions came from teachers of primary grades in six schools around Kazakhstan. These totalled 68 participants from seven interviews and 17 focus groups alongside 87 survey responses to a single open question asking teachers to describe the purposes of assessment. The six schools form a subset of the 30 schools currently piloting the new State Mandatory Standard for Primary Education. Different types of schools from various regions were represented by a rural and an urban location in each location. Research instruments were written in English and translated into Kazakh and Russian by native speakers with an educational research background. Participants had a free choice of language in their participation. All data were anonymous other than the region (southern, western or central Kazakhstan) and location (rural or urban) of the mainstream school at which the participant worked.

Results indicate that there is compliance to the new assessment regime but overwriting years of habits and traditional reports to parents and others has taken an enormous amount of cognitive restructuring, time, patience and effort. Teachers’ conversion to the benefits of a criterion-based system are emerging – primarily based on the positive effect it has on students followed by familiarity and a new sense of creative professional empowerment to be permitted to employ a range of novel assessment practices. Constructs of assessment appear to have evolved over time in teachers with the most experience of CBA through their reports of more complex purposes of assessment. These have shifted from knowledge-checking, means of motivation, effort-recognition and tools of discipline to terminology that makes reference to individual ability, uneven preparedness for school, checking-learning, informed lesson-planning, independent learning and differentiated tasks. However, a significant amount of discourse, including those with over two years of experience applying CBA continued to contain norm-based references and class-level remarks. Approval was given to the close coupling of the curriculum to CBA but how this related to the state standards very rarely appeared in the data. Returning to teachers adopting assessment practices to take on either the role of ‘controller’ versus ‘conductor’, further aspects of concern were that classroom management was more difficult without punitive marking and asking children to self and peer review also meant a relinquishment of control – often framed and justified as a concern of quality assurance. Hence, although more work remains, based on analysis of the results to date, the proffered theory from this research is likely to develop this role choice as a signal of teachers’ adjustment and fundamental understanding of the transition from a norm-based model of control over a class to a criteria-based approach of facilitating and orchestrating individuals’ learning.

Black, P. & Wiliam, D. (2009). Developing the theory of formative assessment. Educational Assessment Evaluation and Accountability, 21(5). https://doi.org/10.1007/s11092-008-9068-5 Bolotov, V., Valdman, I., Kovaleva, G., & Pinskaya M. (2015). Russian Quality Assessment System in Education, Russian Education & Society, 57(7), 531-571, https://doi.org/10.1080/10609393.2015.1096145 Braun, V. & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3 (2): 777-101. http://dx.doi.org/10.1191/1478088706qp063oa Koh, K. H. (2011). Improving teachers’ assessment literacy through professional development, Teaching Education, 22(3), 255-276. https://doi.org/10.1080/10476210.2011.593164 Lok, B., McNaught, C., & Young, K. (2016). Criterion-referenced and norm-referenced assessments: compatibility and complementarity, Assessment & Evaluation in Higher Education, 41(3), 450-465. https://doi.org/10.1080/02602938.2015.1022136 OECD (2013), Synergies for Better Learning: An International Perspective on Evaluation and Assessment, OECD Publishing, Paris. http://dx.doi.org/10.1787/9789264190658-en Rust, C. (2002). The Impact of Assessment on Student Learning.How Can the Research Literature Practically Help to Inform the Development of Departmental Assessment Strategies and Learner-Centred Assessment Practices? , Active Learning in Higher Education, 3(2), 145-158. https://doi.org/10.1177/1469787402003002004 Sadler, D. R. (2005). Interpretations of criteria-based assessment and grading in higher education. Assessment and Evaluation in Higher Education, 30, 175–194. https://doi.org/10.1080/0260293042000264262 Shayakmetova, Z. (2017). Digital trends in education transform approaches to teaching and management. Published in NATION on 6 OCTOBER 2017. Available online at https://astanatimes.com/2017/10/digital-trends-in-education-transform-approaches-to-teaching-and-management/ Wiliam, D. (1993) Validity, dependability and reliability in National Curriculum assessment, The Curriculum Journal, 4(3), 335-350. https://doi.org/10.1080/0958517930040303 Willig, C. (2008). Introducing qualitative research methods in psychology: Adventures in theory and method. 2nd ed. Maidenhead: McGrawHill/ Open University Press.