A 2022 PISA report (De Bortoli et al., 2023) highlighted an ‘unprecedented drop in performance’ since 2018, with average mathematics scores falling by almost 15 points—equivalent to about three‑quarters of a year of learning. Although Australian performance in 2022 remained stable compared with 2018, mathematics outcomes have continued to drive policy reforms and system‑wide initiatives across the country.

In 2021, the Department for Education, South Australia, provided an online one-to-one mathematics tutoring program commencing with a pilot program for Year 6 and Year 8 students. In 2022 the online one-to-one program was extended to include students from Years 6, 7, 8 and 9 with an additional online group tutoring program trialled with high performing Year 7 students in groups with up to three students.

Other Australian states also implemented similar initiatives that targeted schooling disruptions due to COVID-19 with tutoring programs designed as ‘COVID catch-up’ for students negatively impacted during this time. Internationally, various programs have been established to support students, such as the Global Education Coalition supporting up to one million students with online and offline opportunities to support continuity of learning (UNESCO, 2021). The Department for Education in England established one-to-one and small group tutoring programs to support vulnerable students (DfE, 2020) while the French Ministry of Education allocated additional teaching hours after school to support students (OECD, 2020). South Australian students had minimal disruptions to their education in 2020.

The Learning+ pilot program was designed to provide no-cost online tutoring for all students with students from Years 6 and 8 in the pilot program chosen from a representative sample of schools. In 2022 this one-to-one online program was extended to include students from Years 6, 7, 8 and 9. Altogether 277 qualified teachers have tutored 2890 students from 137 schools across South Australia.

Four key principles of design were considered for the online mathematics tutoring initiative: Strategic Design, Structural Design, Technical Design and Systematic Development (adapted from Burkhardt & Swan, 2017). Strategic Design focuses on ensuring that the design complements the existing system and that it can be embedded in the existing system while meeting the needs of all stakeholders. Structural Design ensures that the structure, tools and resources are designed to support teaching and to target student learning. The Technical Design components relate to creative design and research-based decision-making. Finally, Systematic Development utilizes feedback and evaluation systems to guide ongoing improvement.

In 2018, The Office of the Chief Scientist released an Occasional Paper outlining key findings for school-wide improvement in mathematics outcomes. One suggestion for leading improvement in mathematics is a focus on learning, understanding and developing skills rather than a focus on ability, performance and competition (Smith, Ladewig & Prinsley, 2018). The Learning+ initiative focused on developing and consolidating conceptual understanding of key mathematical topics from the Number Strand, designed to improving student engagement and results.

While the Learning+ program was based on Educational Design Research which included resources to support mathematics teaching and learning the researchers also considered how the available learning time could be maximised for each student. Resources included research-based one-to-one number assessment interviews, resource books and ongoing professional development for tutors, and the creation of Professional Learning Communities (PLCs) with PLC leaders appointed to work with groups of 10 teachers.

Group tuition for up to three high performing students was introduced during Term 3, 2022. This presentation focuses on assessment results for the for 76 high performing Year 7 South Australian students who participated in the small group online tutoring sessions. In particular, this presentation examines the results for student confidence when completing mathematics assessment tasks.

Educational Design Research involves iterations of design seeking to improve practice and includes researchers and teachers taking an active role in initiating the project and in the design process (van den Akker, Gravemeijer, McKenney & Nieveen, 2006). The purpose of the Learning+ initiative was to improve student outcomes in mathematics while being supported by qualified teachers taking on roles as tutors. The associated research project directly informed the provision of professional learning, development of assessment instruments and teaching resources, and teaching advice based on the Number and Algebra strand for the Australian Curriculum: Mathematics (ACARA, 2014). The initial year of the Learning+ program included Year 6 and Year 8 students from a representative sample of SA schools with one-third of students deemed to be at, below and above expected level. In the second year of the program students from Years 6, 7, 8 and 9 participated. Students were offered one-to-one 2 half-hour online tutoring sessions per week for 10 weeks. To establish each student’s needs, the tutors conducted a one-to-one interview using one of the assessment instruments aligned with the Australian Curriculum: Number and Algebra Strand (ACARA, 2014). An additional Learning+ initiative was introduced during Term 3, 2022 involving group tutoring for up to three high performing students per group, as determined by PAT-M results. As online group tutoring did not allow the tutor the time to conduct the full number interview with each student, a written Number Screening Test version of the interview was developed. The Number Screening Test was completed independently online. For 24 of the 35 questions students were also asked to rate their degree of confidence in their answer using a Likert scale. At the beginning of the group tutoring sessions students independently completed the Number Screening Test. They completed a second Number Screening Test at the end of the tutoring sessions. The pre and post Number Screening Test results were analysed to identify student progress after participating in the 10-week group tutoring initiative. The analysis used paired pre and post Number Screening Test data for 76 students who participated in the group program. The analysis focusses on student scores, student confidence in their answers, and finally the combination of both answer accuracy and student confidence. Finally, an inferred student understanding measure using the combined confidence and accuracy data was applied to indicate presence of understanding, knowledge gaps and misconceptions for the various questions.

The Learning+ initiative aimed to support tutors with professional learning and resources to maximise student engagement in tutoring sessions, and to develop students' mathematical understanding of number and algebra topics. Strategic design considerations included the alignment of assessment instruments and tutoring activities with the Australian Curriculum: Mathematics (ACARA, 2014) to complement rather than replace students’ mathematics learning at school. The PLC structure was modified to target accessibility and support for tutors in a collegial environment with other tutors, a lead tutor and the researchers. The structural design components were developed, tested and revised. For example, analysis of the interview record sheets provided valuable insights into how these records were being used and resulted in adjusting the record sheet and development of targeted professional learning about conducting, recording and analysing interview results. All aspects of the system design process were ongoing. Most students in the high-performing cohort significantly improved their mathematics skills across all topics and became more confident about their mathematics performance after participating in the group tutoring program. Student confidence about the accuracy of their answers also increased following participation in the program. Initially the average response for confidence in their answers sat between ‘Not sure’ and ‘Probably right’ while by the end of the program average confidence in answers sat between the ‘Probably right’ and ‘Definitely right’ category. Collecting student confidence data as well as mathematics content knowledge provides educators with multifaceted diagnostic information. While inclusion of student confidence questions within a test extends the time taken to complete that assessment and increases the volume and complexity of the data, it can provide useful diagnostic information to underpin planning and implementation of teaching. This research project used empirical evidence and research-based approaches to continually refine and redesign a large-scale online tutoring initiative, which was a complex and dynamic process.

Australian Curriculum, Assessment and Reporting Authority (ACARA). (2014). The Australian Curriculum: Mathematics. Retrieved from https://www.australiancurriculum.edu.au/f-10-curriculum/mathematics/ Burkhardt, H., & Swan, M. (2017). Design and development for large-scale improvement. In Kaiser, G (Ed). Proceedings of the 13th International Congress on Mathematical Education (pp. 177-200). Springer. Department for Education, England. (2020). National tutoring programme launched to support pupils in schools across in England. Retrieved from https://nationaltutoring.org.uk/news/national-tutoring-programme-launched-to-support-pupils-in-schools-in-england/ De Bortoli, L., Underwood, C., & Thomson, S. (2023). PISA 2022. Reporting Australia’s results. Volume I: Student performance and equity in education. Australian Council for Educational Research. https://doi.org/10.37517/978-1-74286-725-0. Earp, J. (2023). PISA 2022 – Good and bad news for Australia in global student assessment. Teacher Magazine. Australian Council for Educational Research. Retrieved from https://www.teachermagazine.com/au_en/articles/pisa-2022-good-and-bad-news-for-australia-in-global-student-assessment Global Education Coalition [UNESCO]. (2021). Supporting learning recovery one year into COVID-19: the Global Education Coalition in action. Retrieved from https://unesdoc.unesco.org/ark:/48223/pf0000376061 Organisation for Economic Cooperation and Development (OECD). (2020). The impact of covid-19 on student equity and inclusion. Retrieved from https://read.oecd-ilibrary.org/view/?ref=434_434914-59wd7ekj29&title=The-impact-of-COVID-19-on-student-equity-and-inclusion Smith, P., Ladewig, M., & Prinsley, R. (2018). Improving the mathematics performance of chief scientist Australia’s students. Australian Government: Office of the Chief Scientist. Retrieved from https://www.chiefscientist.gov.au/sites/default/files/Improving-the-mathematics- van den Akker, J., Gravemeijer, K., McKenney, S., and Nieveen, N. (2006). Educational Design Research. Routledge.