Internationally there has been a shift in educational policy towards evidence-based decision-making within schools (Mandinach, 2012; Schildkamp, Karbautzki, & Vanhoof, 2014; Scottish Government, 2017) to the extent that many Western democracies have implemented laws and policy reforms focused on education and schooling. These reforms are designed to put the child at the centre of the education system and places demands on teachers to use multiple sources of evidence that they have available to them to guide their professional judgments and decisions. Notable examples of these law and policy reforms range from the No Child Left Behind Act (2002) - now superseded by the Every Student Succeeds Act (2015) - in the United States, the European Union's Strategic Framework for European Cooperation in Education and Training (ET 2020), the National Education Plan in France and the Education (Scotland) Act (2016) in Scotland. In recent years, however, there has been an intensification in this trend towards datafication in education. Teachers and school leaders around the world are now encouraged (to varying degrees) to base their decisions on data (Merk, Poindl, Wurster, & Bohl, 2020). Indeed, many European governments have followed suit by enacting educational policies that mandate the increased use of educationally relevant data by teachers and school leaders to drive improvement in educational outcomes for children and young people (Mandinach & Jimerson, 2016). These policies imply that teachers and school leaders ought to be cognisant of, and conversant with the multiple lines of evidence available to them whilst placing the onus on them to be able to make effective professional judgements about students learning and be able to act on those judgements in a manner that facilitates improvements in performance outcomes.

Research suggests that many teachers feel threatened by the concept of data-informed decision making and ill-prepared to engage in a sustained way with educationally relevant data (Dunn, Airola, Lo, and Garrison, 2013a). Research evidence also suggests that many teachers do not systematically use data-informed judgements within their daily practice or if they do, they only trust the data that confirms their intuition rather than using all of the available evidence available to them to shape their professional judgements (Dunn, Airold & Garrison, 2013b; Schildkamp & Ehren, 2013).  These findings indicate that many in-service teachers may lack the skills, motivation, or positive attitude towards the use of educational data to support their professional judgements. What is less well understood is how Initial Teacher Education (ITE) student teachers (also referred to as pre-service teachers) are prepared to use the wide range of educationally relevant data available to them and what factors might affect their use of data within their professional judgement.

This study aims to explore how final year ITE students handle, analyse and make meaning from educational data as part of their professional judgment and decision-making processes. This aim is operationalised by the following research questions (1) To what extent can final year ITE students analyse and interpret educationally relevant data as part of their reflective practice? (2) Does final year ITE students’ ability to analyse educationally relevant data differ compared to their programme of study.

This research reports findings from the quantitative phase of an exploratory, sequential, mixed methods investigation into how final year ITE students use multiple forms of evidence to make meaning and formulate professional judgements about teaching. The final year ITE students from three teacher education programmes – Professional Graduate Diploma in Education (Secondary) [PGDE (S) n=136], Professional Graduate Diploma in Education (Primary) [PGDE (P) n=95] and the Bachelor of Arts (Honours) in Primary Education [BA4 n=104] - within one university division of education were asked to complete a paper-and-pencil data analysis and interpretation activity designed to explore how final year ITE students analyse and make meaning from tracking and monitoring data as well as how they make professional judgments about practice from that data. The data analysis and interpretation task contained three sections where section one focused on classroom level tracking and monitor data. Sections two focused on school level data and section three focused on school to national level data. The data analysis and interpretation activity scripts were sorted into the three educational programmes, then graded independently by two researchers and cross checked for concordance. All grades were then entered into an Excel spreadsheet and then transferred to SPSS for downstream descriptive and inferential statistical analysis to compare the findings for each programme of study against each other using a Friedman ANOVA, Mann-Whitney test, or a t-Test dependent on the variance of the sample.

In terms of the final year ITE students’ ability to analyse, interpret and make meaning from educationally relevant data, our finding suggest that BA4 students mean ability score is significantly lower than that for PGDE (S) [mean ± SD - BA4 39.3% ± 10.1% v PGDE (S) 47.9% ± 10.3%]. There is a strong statistical difference (Mann-Whitney U-Test p<0.0001) between the PGDE (S) and BA4 mean scores and the PGDE (P) and BA4 mean score. However, there was no significant difference between the PGDE (S) and PGDE (P) mean scores. Looking at the data relating to item Q1d Reflecting upon the data in Table 2 [table on the activity worksheet], if this were your class, what does this data suggest about (i) pupils’ attainment? (ii) Your teaching? Only 2.9% of PGDE (S), 4.2% of PGDE (P), and 1.9% of BA4 students could give four points about pupil attainment from the data. However, 19.9% of PGDE (S) students, 20.0% of PGDE (P) and 9.6% of BA4 students could give 3 points and 22.8% of PGDE (S), 32.6% of PGDE (P), and 37.5% of BA4 students could give one point. Worryingly, 8.1% of PGDE (S), 1.1 % of PGDE (P) and 35.6% of BA4 students could not give any points. These findings suggest that final year ITE students struggled to reflect on what the assessment data might indicate with regards to teaching practice. This research indicates that more support is required to help final year ITE students make meaningfully interpretations from assessment data to support better pedagogical decision-making

Dunn, K. E., Airola, D. T., Lo, W. J., & Garrison, M. (2013a). Becoming data driven: The influence of teachers’ sense of efficacy on concerns related to data-driven decision making. The Journal of Experimental Education, 81 (2), 222-241. Dunn, K. E., Airola, D. T., & Garrison, M. (2013b). Concerns, knowledge, and efficacy: An application of the teacher change model to data driven decision-making professional development. Creative Education, 4 (10), 673. Mandinach, E. B (2012). A Perfect Time for Data Use: Using Data-Driven Decision Making to Inform Practice, Educational Psychologist, 47 (2), 71-85. Mandinach, E. B., & Jimerson, J. B. (2016). Teachers learning how to use data: A synthesis of the issues and what is known. Teaching and Teacher Education, 60, 452-457. Merk, S., Poindl, S., Wurster, S., & Bohl, T. (2020). Fostering aspects of pre-service teachers’ data literacy: Results of a randomized controlled trial. Teaching and Teacher Education, 91, 103043. Schildkamp, K., & Ehren, M. (2013). From “Intuition”-to “Data”-based Decision Making in Dutch Secondary Schools? In Data-based decision making in Education (pp. 49-67). Springer, Dordrecht. Schildkamp, K., Karbautzki, L., & Vanhoof, J. (2014). Exploring data use practices around Europe: Identifying enablers and barriers. Studies in educational evaluation, 42, 15-24. Scottish Government (2017) National Improvement Framework and improvement plan for Scottish education. (Available online) https://www.gov.scot/publications/2017-national-improvement-framework-improvement-plan/ [Last Accessed 26th Jan 2023]