Learning outcomes prediction has become an important factor affecting the position of a University on the educational markets. Significant research has been dedicated to predicting  students’ dropout rate, retention, GPA and other significant indices. An often used approach is based on the analysis of the historic data. The future GPA and possible dropout can be predicted on the basis of previous GPAs and dropouts. Still there exists a problem of predicting learning outcomes in situations with limited historic data, which occurs for example, when a student starts to learn a new subject. Predicting students’ advancement in a course is a matter of professors’ interest, as it allows to timely provide scaffolding for those who experience difficulties in learning at the courses and thus decrease the course failure rate. Improving the quality of the learning process requires new non-disruptive tools, allowing us to predict academic outcomes based on data, available on the learning platforms, which can be extracted, processed and analysed. Thus our work is aimed at answering the research question:

-How can students’ academic results be predicted non-disruptively and what digital footprints can be used for building the prediction model?

To answer the research question we aimed to reach the following objectives:

-Analyse the literature in the field.

-Build a conceptual model of predicting academic outcomes in distance learning.

-Collect the data reflecting students’ digital footprints.

-Predict students’ performance on the course based on their footprints.

-Validate the conceptual model based on their actual performance.

The theoretical framework of our research is based on the latest publications in the area of academic outcomes prediction. With the emergence of technology enhanced learning (TEL), a broad array of data that may be used in predicting academic outcomes has become available for research. Such data may include demographic profile, family status, job, gender, nationality (Thiele). Among the learning data traditionally grades for the previous term are used (Hellas). In some cases making sound predictions requires additional data obtained from psychological and sociological surveys. In Musso’s (Musso) research over 20 psychological and sociological parameters, predicting academic results were used. The research showed that the most essential GPA predictors are the student’s motivation, isolation, information processing and attention reaction time. 

With the development of elearning platforms researchers received a number of ways to analyze their users behaviour with data obtained from logs of the users’ actions. A number of relevant factors, serving as academic outcomes predictors may be extracted from the literature (Hellas, Khanna). Among them are:

-Text complexity

-The student’s time in the system

-Successful tests  passing

-Participation in forums

-Completion of tasks

-Portfolio management

-Course navigation

-Chat messages

-Use of resources

-Interaction with other students

Context: Our research is part of a European project on Learning Analytics undertaken by Moscow City University, University of Catlow III of Spain and Armenian Brusov University, aimed at finding effective tools for predicting and improving academic outcomes. For predicting course learning results we used presentations and speech scripts of  Moscow City University professors and students, enrolled in 2020-2021 in the International Baccalaureate program. In total 83 texts, consisting of 9929 words were analysed. Those texts were extracted from MS Teams, which served in the course as the communication environment where professors and students could plan and conduct video conferences, exchange messages, post and view presentations.

Modern teaching and learning offers vast opportunities for digital footprints analysis (Siemens 2011). Such evidence of students learning as logs of their actions on e-learning platforms, their products, created throughout the course, their use of social networks etc can be used to make assumptions about their success or failure in achieving learning goals. This allows researchers to avoid incorporating disruptive procedures, such as tests, surveys or observations into learning. Analysis of digital footprints, left in the students' works, is an effective way to trace the learners’ progress without disrupting the process itself. Our model of predicting students’ result of learning in a course is based on an assumption that creating his product a student uses the framework given to him by his professor and adds to it his own understanding and interpretation of the subject. His interpretation may not completely coincide with that of the professor, but it can be measured by the means of semantic analysis. At the first step of our analysis we formed professors’ and students’ text corpora, consisting of all the words, included into professors’ and students’ presentations and scripts which we used for further analysis. All the learning products, extracted from the MS Teams, were converted into the text format and cleaned from stop words, not conveying any subject specific meaning. All the corpora items were lemmatized, i.e. reduced to the minimal meaningful form. The lemmatization was fulfilled with pymorphy2 method, for which used Python https://pymorphy2.readthedocs.io/en/stable/ library which supports Russian and English languages.We used NLTK (http://www.nltk.org/nltk_data/ ) dictionary to delete words of low significance from the corpora. Thus professors and students' word corpora, consisting of high significance words, were formed.

Comparing students’ and professors’ corpora we were looking for common words, observed in both those corpora used in the students’ products. Having found those words, we built our model on a hypothesis that successful graduation from a course depends on the way the student comprehends the concepts, conveyed in the professor’s lectures and uses the key words from the professor’s corpus in his own products. The professor’s corpus consisted of 427 words used by the professor to explain the key concepts of their subject. In the professor’s lecture the words occur with a frequency of 1 to 18 times. The professor’s corpus was compared with the corpora of students studying “A Foreign Language for Professional Goals” subject. Students’ corpora sizes varied from 53 to 150 words. When predicting the result of a student's training in the course, we used an assessment of the similarity of the teacher's text and the students' texts, calculated by the formula: Ss=1/|Lg(Nt*Nks)| For evaluating the effectiveness of this algorithm, the similarity score was compared with the actual scores received by students. The correlation between predicted and actual course results is >0.9 which testifies to high effectiveness of the described approach. We have implemented and tested a model that helps to predict students’ results in a course based on the analysis of their digital footprints that they leave throughout their learning. The approach, based on comparing students’ and professors’ corpora, allows to predict the students’ success or failure in the course. Online tool, available for professors, helps them to plan interventions and prevent students from falling behind from their class. This approach helps to significantly improve students' performance and decrease the dropout rate.

Aguiar, E., Ambrose, G. Engagement vs performance: Using electronic portfolios to predict first semester engineering student persistence. // Journal of Learning Analytics, 1(3), 2014. Al-Barrak, M.A., Al-Razgan, M. Predicting students final gpa using decision trees: a case study. // International Journal of Information and Education Technology, 6(7):528, 2016. Grivokostopoulou, F., Perikos, I., Hatzilygeroudis, I. Utilizing semantic web technologies and data mining techniques to analyze students learning and predict final performance. // 2014 International Conference on Teaching, Assessment and Learning (TALE), pages 488–494. IEEE, 2014. Hellas, A., Ihantola, I., ‘‘Predicting academic performance: A systematic literature review,’’ // Proc. Companion 23rd Annu. ACM Conf. Innov. Technol. Comput. Sci. Edu., 2018, pp. 175–199. Khanna, L., Singh S. "Educational data mining and its role in determining factors affecting students academic performance: A systematic review", Information Processing (IICIP) 2016 1st India International Conference on, pp. 1-7, 2016. Murtaugh, P. A., Burs, L. D., & Schuster, J. (1999). Predicting the retention of university students // Research in Higher Education, 40(3), 355-371. Musso, M., Hernández C., Cascallar, C., "Predicting key educational outcomes in academic trajectories: A machine-learning approach"// Higher Educ., vol. 80, no. 5, pp. 1-20, Nov. 2020. Perez, C. Different Tests, Same Flaws: Examining the SAT I, SAT II, and ACT. // Journal of College Admission, 177, 20-25. 2002. Shahiri, M., Husain W. "A review on predicting student’s performance using data mining techniques"// Procedia Comput. Sci., vol. 72, pp. 414-422, Dec. 2015. Siemens, G., Long, P. (2011). Penetrating the fog: Analytics in learning and education. // EDUCAUSE Review, 46(5), 31–40 Tapia-Leon, M., A. Carrera Rivera, A. "Representation of Latin American University Syllabuses in a Semantic Network," in 2nd International Conference on Information Systems and Computer Science (INCISCOS), 2017, pp. 295--301. Thiele, T., Singleton, A., Pope, D. & Stanistreet, D. (2015) Predicting students’ academic performance based on school and socio-demographic characteristics, Studies in Higher Education, 27, 1– 23. Wymore, A. Wayne, A Mathematical Theory of Systems Engineering // The Elements, John Wiley & Sons: New York, NY, 1967.