Session Information
16 SES 01 A, Chatbots and Robotics
Paper Session
Contribution
The digitalisation of education offers transformative potential, enriching teaching practices and broadening instructional possibilities within schools. However, this shift also introduces a set of complex challenges that impact both pedagogy and curriculum. Within this evolving digital landscape, which includes domains such as artificial intelligence, data management, cloud computing, and sustainable technologies (González-Pérez & Ramírez-Montoya, 2022), teachers are faced with complex considerations, including classroom management, assessment, ethical concerns, and the integration of digital technologies. A key area of focus within digital transformation is the development of computational thinking through programming and educational robotics, targeting 21st-century skills such as collaboration, critical- and ethical thinking (González-Pérez & Ramírez-Montoya, 2022; Ye et al., 2022).
In response to these educational imperatives, programming has been integrated into the school curricula of several countries. While some countries have introduced programming as a separate subject or as a subsection of Mathematics, others like Finland and Norway have embraced a cross-curricular approach, incorporating programming into diverse subjects such as Art and Design, Music, and Science, in addition to Mathematics.
Based on the interdisciplinary landscape of programming education, its research intersects with various academic disciplines and pedagogical approaches. In order to shed light on how these interdisciplinary perspectives are brought together in current K-12 programming research, as well as to gauge the scope and quality of evidence syntheses that have been undertaken previously, as well as identify research gaps, a meta scoping review (Booth et al., 2022) was undertaken. The overarching research question guiding this study is:
What is the nature and scope of evidence synthesis on programming and robotics in primary and secondary education?
To provide a comprehensive answer to this main research question, the study is broken down into the following sub-questions:
- What types of evidence syntheses have been conducted, and in what years were they published?
- In which journals are these evidence syntheses published?
- What is the geographical distribution of the authors, their affiliations and disciplines?
- To what extent is there collaboration in the systematic reviews on programming in primary and secondary education?
- What is the quality of K-12 programming evidence synthesis?
- What overlap and gaps exist among research questions across evidence syntheses in terms of topic or subject area?
- What benefits and challenges of programming in K-12 education have been identified?
By conducting this meta scoping review, the study aims to lay a foundational groundwork for future primary and secondary research in the domain of programming education.
Method
In order to answer the research questions, a meta scoping review was conducted (Booth et al., 2022), using explicit and predefined criteria (Gough et al., 2012; Zawacki-Richter et al., 2020), with the reporting guided by the PRISMA guidelines (Page et al., 2021). A scoping meta-review is a type of tertiary review (Kitchenham et al., 2009), which synthesises secondary research such as systematic reviews and meta-analyses. The review was undertaken based on previous tertiary reviews (Bond et al., 2024; Buntins et al., 2023), with the first search conducted in April 2023, and subsequent searches conducted until 17 January 2024 to ensure the inclusion of extant literature. The platforms and databases searched were the Web of Science, Scopus, EBSCOHost (including ERIC), and Progress, with the OpenAlex platform (Priem et al., 2022) also searched via evidence synthesis software EPPI Reviewer (Thomas et al., 2023). A search string was developed based on two previous tertiary reviews (Bond et al., 2024; Buntins et al., 2023), focusing on programming, computational thinking and robotics in K-12, as well as variations of evidence synthesis (Sutton et al., 2019). The search strategy yielded 4,369 items, which were exported as a .txt or .ris file and imported into EPPI Reviewer. Following the automatic removal of 485 duplicates, two reviewers screened the same 200 items on title and abstract (2 x 100), applying the inclusion/exclusion criteria, to ensure inter-rater reliability. After achieving perfect agreement, the remaining 3,684 items were screened on title and abstract. Studies were included if they explored programming or computational thinking in K-12, were a journal article published after 2010 in English, and were a form of evidence synthesis, leaving 195 items to screen on full text. To ensure continued inter-rater reliability, a further 10 items were double screened at this stage, and again the reviewers were in complete agreement. After screening the remaining items, 121 evidence syntheses were identified for data extraction and synthesis within EPPI Reviewer. For the purposes of this paper, however, only the 50 reviews pertaining to programming and robotics will be included.
Expected Outcomes
This meta scoping review explores the evolution, distribution, and quality of evidence syntheses in programming education research from 2011 to 2023, focusing on primary and secondary education. Whilst analysis is currently ongoing, systematic reviews and meta-analyses have been dominant, with a gradual increase in the number and range of syntheses being conducted since 2021. The 50 journal articles in the corpus were published in 37 unique journals, reflecting a wide interest in not only the topic, but in synthesis methods also. Geographically, authors span five continents, with most authors hailing from Europe (42%) and Asia (38%), suggesting worldwide engagement in this research area. However, there was a notable lack of representation from Africa and Oceania. Collaboration patterns showed a heavy preference for domestic partnerships (64.8% of co-authored articles), with only 18% of research published by international research teams. The quality of studies also varied, with a preponderance of medium and low-quality evidence, with very few higher quality studies, highlighting the need for more rigorous and transparent approaches to evidence synthesis, echoing findings in the wider field of education (Bond et al., 2024; Buntins et al., 2023). Thematic analysis revealed a focus on sub-themes such as skill development, teaching methods, and pedagogical goals. However, gaps were evident, particularly in subjects like Mathematics, on the ethical considerations of AI and robotics, and the role of teachers in programming education. The benefits of programming and robotics education emerged as significant, enhancing cognitive development, creativity, and interdisciplinary learning. Challenges included resource constraints, curriculum integration, teacher training needs, cognitive load concerns, and the need for more parental involvement in robot-assisted learning. While programming education research is extensive and diverse, areas identified for future exploration, particularly in underrepresented regions, include ethical issues in technology use, and more inclusive pedagogical strategies.
References
Bond, M., Khosravi, H., De Laat, M., Bergdahl, N., Negrea, V., Oxley, E., Pham, P., Chong, S.W., & Siemens, G. (2024). A meta systematic review of Artificial Intelligence in Higher Education: A call for increased ethics, collaboration, and rigour. International Journal of Educational Technology in Higher Education, 21. https://doi.org/10.1186/s41239-023-00436-z Booth, A., Sutton A., Clowes, M., Martyn-St James, M. (2022). Systematic Approaches to a Successful Literature Review. SAGE Buntins, K, Bedenlier, S., Marín, V., Händel, M., & Bond, M. (2023). Methodological approaches to evidence synthesis in educational technology. A tertiary systematic mapping review. MedienPädagogik Research Syntheses, 54, 167–191. https://doi.org/10.21240/mpaed/54/2023.12.20.X González, M.Á., Rodríguez-Sedano, F.J., Llamas, C.F., Gonçalves, J., Lima, J., & García-Peñalvo, F.J. (2020). Fostering STEAM through challenge‐based learning, robotics, and physical devices: A systematic mapping literature review. Computer Applications in Engineering Education, 29, 46 - 65. Gough, D., Oliver, S., & Thomas, J. (Eds.). (2012). An introduction to systematic reviews. SAGE. Kitchenham, B., Pearl Brereton, O., Budgen, D., Turner, M., Bailey, J., & Linkman, S. (2009). Systematic literature reviews in software engineering – A systematic literature review. Information and Software Technology, 51(1), 7–15. https://doi.org/10.1016/j.infsof.2008.09.009 Page, M. J., McKenzie, J. E., Bossuyt, P. M., . . . Moher, D. (2021). The PRISMA 2020 statement: An updated guideline for reporting systematic reviews. BMJ, 372. https://doi.org/10.1136/bmj.n71 Priem, J., Piwowar, H., & Orr, R. (2022). OpenAlex: A fully-open index of scholarly works, authors, venues, institutions, and concepts. ArXiv. https://arxiv.org/abs/2205.01833 Sutton, A., Clowes, M., Preston, L., & Booth, A. (2019). Meeting the review family: Exploring review types and associated information retrieval requirements. Health Information and Libraries Journal, 36(3), 202–222. https://doi.org/10.1111/hir.12276 Thomas, J., Graziosi, S., Brunton, J., Ghouze, Z., O'Driscoll, P., Bond, M., & Koryakina, A. (2023). EPPI Reviewer: advanced software for systematic reviews, maps and evidence synthesis. EPPI Centre Software. UCL Social Research Institute. London. https://eppi.ioe.ac.uk/cms/Default.aspx?alias=eppi.ioe.ac.uk/cms/er4 Ye, J., Lai, X., & Wong, G. K.‑W. (2022). The transfer effects of computational thinking: A systematic review with meta‐analysis and qualitative synthesis. Journal of Computer Assisted Learning, 38(6), 1620–1638. https://doi.org/10.1111/jcal.12723 Zawacki-Richter, O., Kerres, M., Bedenlier, S., Bond, M., & Buntins, K. (Eds.). (2020). Systematic Reviews in Educational Research. Springer. https://doi.org/10.1007/978-3-658-27602-7
Search the ECER Programme
- Search for keywords and phrases in "Text Search"
- Restrict in which part of the abstracts to search in "Where to search"
- Search for authors and in the respective field.
- For planning your conference attendance you may want to use the conference app, which will be issued some weeks before the conference
- If you are a session chair, best look up your chairing duties in the conference system (Conftool) or the app.