Session Information
11 SES 16 A, Educational Technologies and Quality Assurance
Paper Session
Contribution
AI, or artificial intelligence, refers to computing systems that can perform tasks similar to those of humans, such as adapting, learning, and using data for complex processing (Popenici & Kerr, 2017). There are various branches and sub-branches of AI, but for feedback purposes, the most relevant ones are natural language processing (NLP), educational data mining (EDM), and learning analytics (LA) (Gardner et al., 2021). NLP is beneficial for feedback because it can analyze linguistic components of students’ written work and provide feedback on writing quality, syntactic complexity, and grammatical errors. EDM allows for data-supported feedback through data visualization and can also provide verbal feedback using NLP or manual input from instructors. LA uses student activity data to provide personalized feedback through an interactive dashboard. Feedback can be either semi-automatic or fully automatic, depending on the system used (Wongvorachan et al., 2022). AI has been incorporated into NLP, EDM, and LA, leading to the development of complex systems that can provide students with timely and individualized feedback. As a result, both their performance and learning process can be improved. It has been demonstrated that AI-based feedback systems are more effective and efficient than more conventional forms of feedback.
It has been demonstrated that incorporating AI into feedback improves student motivation and engagement, which results in higher learning outcomes (Alazmi & AlZoubi, 2020). Moreover, it has been revealed that AI-based feedback systems are economical and scale to large classrooms, making them appropriate for use in both traditional and online learning environments (Chang et al., 2020). Thus, the integration of AI in feedback is not only improving the learning experience of students but also transforming the traditional methods of feedback in education. According to Zawacki-Richter et al. (2019), the incorporation of AI in K-12 education has seen significant growth in recent years. Crompton and Song (2021) also note that the use of AI offers numerous possibilities for enhancing teaching and learning. One way AI is being utilized is in the automatic grading of essays, as reported by Yang et al. (2019). Additionally, AI can provide swift feedback to students, as stated by Benotti et al. (2018), and can adjust instruction to meet the unique needs of each student, as highlighted by Arnett (2016).
A systematic review of the application of AI and robotics in K-12 education was carried out by Hrastinski et al. (2019), with an emphasis on the relationship between teachers and students. However, the scope of the study was limited to papers from one international symposium and solely on robotics, rather than AI more broadly. Furthermore, it did not examine the potential of AI in enhancing feedback practices in K-12 education. Zafari et al.'s (2022) and Crompton et al.’s (2022) studies examined the current state of AI integration in K-12 education, with a focus on its general use, not just its use for feedback.
The aim of this study is to address the call by scholars (Banihashem et al., 2022) to investigate the role of NLP, EDM, and LA in enhancing feedback practices in K-12 education. This paper will provide researchers and educators with a deeper insight into the application of NLP, EDM, and LA for feedback purposes. In this regard, the present systematic review seeks to answer the following research questions:
1. What are the primary reasons behind the utilization of NLP, EDM, and LA in feedback studies in K-12 education?
2. What types of data are utilized in studies on NLP, EDM, and LA to provide feedback in K-12 education?
3. What NLP, EDM, and LA tools and techniques are employed by studies to facilitate feedback in K-12 education?
Method
We followed the PRISMA framework (identification, screening, eligibility and inclusion) to conduct the systematic review. Search strategy We searched the most relevant terms and synonyms which are overlapping the concepts that the present study focused on by identifying the prior systematics reviews (e.g., Banihashem et al., 2022; Zafari et al., 2022) on K-12 education, artificial intelligence or feedback practices. The search string included the following terms: ("learning analytic*" OR "educational data mining" OR "artificial intelligence" OR "natural language processing") AND (feedback OR "formative assessment" OR feedforward) AND ("K-12 student*" OR "K-12" OR "K-12 education" OR "primary school*" OR "primary education" OR “kindergarten*" OR “pre-primary” OR “middle school*" OR "secondary education" OR "secondary school*" OR “high school*” OR “1st grade” OR “2nd grade” OR “3rd grade” OR “4th grade” OR “5th grade” OR “6th grade” OR “7th grade” OR “8th grade” OR “9th grade” OR “10th grade” OR “11th grade” OR “12th grade” OR "grade 1" OR “grade 2" OR "grade 3" OR "grade 4" OR "grade 5" OR "grade 6" OR "grade 7" OR "grade 8" OR “grade 9" OR "grade 10" OR "grade 11" OR "grade 12"). Web of Science (WOS), ERIC, and IEEE databases were chosen considering their reputation and inclusion of numerous research studies on the topics that were addressed in the current study. Criteria for inclusion The following criteria were used to determine which articles were included: (a) journal articles published between 2013 and 2023; (b) articles written in English language, (c) peer-reviewed journals to ensure quality, and (d) empirical studies. However, conference proceedings were excluded. Identification of relevant publication During the initial screening phase in the selected databases (WOS [n=72], ERIC [n=55], and IEEE [n=443]), a total of 570 were identified. After eliminating duplicates (n=22) and non-peer-reviewed articles (n=10), a pool of publications (n=538) remained. In the second phase, the titles and abstracts were screened against our inclusion criteria, and 459 papers did not meet the criteria, 79 papers were further evaluated through full-text screening. The final pool of papers was then used for quality appraisal. Quality appraisal We adopted quality appraisal criteria from Theelen et al. (2019), based on the work of Savin-Baden and Major (2007) for evaluating qualitative studies and NICE (2012) for evaluating quantitative studies.
Expected Outcomes
Preliminary Findings and Expected Outcomes The Rayyan program is utilized for reviewing research papers. It's a free online tool for scientists conducting systematic reviews and similar projects. Initially, we used it as a blind version for accuracy purposes. We independently evaluated the papers by marking "include" or writing reasons for “exclusion”. After both authors finished evaluating the articles, we switched to the unblinded version and resolved conflicts. We disagreed on five papers, with one author wanting to include while the other excluded. We ultimately included only three of these five studies. At least one of us used the word "maybe" in evaluating 34 papers, so we also evaluated each paper as a team. We used the label "maybe" because the abstract was not clear on which AI techniques used. After our team review, we chose 16 of them for further evaluation because the full text can aid in labeling the technique. Out of the articles where the technique was identifiable, 42 utilized LA, 16 used NLP, and 5 employed EDM. We employ Nvivo to conduct content analyses relevant to our research questions. We will also conclude our review by highlighting key challenges and opportunities for future research.
References
Alazmi, B., & AlZoubi, A. (2020). The role of artificial intelligence in education. Journal of Education and e-Learning Research, 7(2), 19-30. Arnett, T. (2016). Teaching in the machine age: How innovation can make bad teachers good and good teachers better. Christensen Institute. Banihashem, S. K., Noroozi, O., van Ginkel, S., Macfadyen, L. P., & Biemans, H. J. (2022). A systematic review of the role of learning analytics in enhancing feedback practices in higher education. Educational Research Review, 37. Benotti, L., Martinez, M.C., & Schapachnik, F. (2018). A tool for introducing computer science with automatic formative assessment. IEEE Transactions on Learning Technologies, 11(2), 179–192. Chang, K.E., Huang, Y.M., & Chen, W.H. (2020). A review of AI-based feedback systems for education. JETDE, 3(1), 1-14. Crompton, H., Jones, M.V., & Burke, D. (2022). Affordances and challenges of artificial intelligence in K-12 education: a systematic review. JRTE, 1-21. Crompton, H., & Song, D. (2021). The potential of artificial intelligence in higher education. Revista Virtual Universidad Católica Del Norte, 62, 1–4. Gardner, J., O’Leary, M., & Yuan, L. (2021). Artificial intelligence in educational assessment: Breakthrough? Or buncombe and ballyhoo?. Journal of Computer Assisted Learning, 37(5), 1207–1216. Hrastinski, S., Olofsson, A.D., Arkenback, C., Ekström, S., Ericsson, E., Fransson, G., Jaldemark, J., Ryberg, T., Öberg, L-M., Fuentes, A., Gustafsson, U., Humble, N., Mozelius, P., Sundgren, M., & Utterberg, M. (2019). Critical imaginaries and reflections on artificial intelligence and robots in post digital K-12 education. Postdigital Science and Education, 1(2), 427–445. Popenici, S.A.D. & Kerr, S. (2017). Exploring the impact of artificial intelligence on teaching and learning in higher education. RPTEL, 12(22). Theelen, H., Van den Beemt, A., & den Brok, P. (2019). Classroom simulations in teacher education to support preservice teachers’ interpersonal competence: A systematic literature review. Computers & Education, 129, 14-26. Wongvorachan,T., Lai, K.W, Bulut, O. Tsai, Y. & Chen, G. (2022). Artificial Intelligence: Transforming the Future of Feedback in Education. Journal of Applied Testing Technology, 23(1), 1-22. Yang, Y., Xia, L., & Zhao, Q. (2019). An automated grader for Chinese essay combining shallow and deep semantic attributes. IEEE Access 7. Zafari, M., Bazargani, J.S.,Sadeghi-Niaraki, A., & Choi, S.M. (2022). Artificial intelligence applications in K-12 education: A systematic literature review. IEEE Access, 10. Zawacki-Richter, O., Marín, V.I., Bond, M., & Gouverneur, F. (2019). Systematic review of research on artificial intelligence applications in higher education – Where are the educators? IJETHE, 16(1), 1–27.
Search the ECER Programme
- Search for keywords and phrases in "Text Search"
- Restrict in which part of the abstracts to search in "Where to search"
- Search for authors and in the respective field.
- For planning your conference attendance you may want to use the conference app, which will be issued some weeks before the conference
- If you are a session chair, best look up your chairing duties in the conference system (Conftool) or the app.