Engagement is important for student success in online learning, as it is positively related to academic achievement and negatively associated with student dropout (Lee & Choi, 2011; Steele & Fullagar, 2009). In the context of many online courses, the key instructional methods are either attending synchronous classes, watching asynchronous instructional videos, or both. In online classes student synchronous and asynchronous engagement are important for their performance (Authors, 2024). This makes fostering student engagement in synchronous and asynchronous activities crucial for their success.

Engagement is a dynamic and malleable construct, meaning that it changes over time and is potentially responsive to interventions (Fredricks et al., 2004). Research shows that student engagement can be supported in different ways in online learning, such as providing students with analytics on their learning (Karaoglan Yilmaz & Yilmaz, 2022; Yang et al., 2018), procedural support (Staikopoulos et al., 2015), and metacognitive guidance on efficient learning strategies (Karaoglan Yilmaz & Yilmaz, 2022). Providing learning analytics to learners may be beneficial for students’ engagement, as it makes learners more aware of their progress and shows areas for improvement (Brown et al., 2023).  Providing students with information about potential learning resources (procedural support) can help students manage learning resources effectively and stay on track (Staikopoulos et al., 2015). Furthermore, providing information about effective learning strategies (metacognitive support) increases engagement by developing student awareness and control over their learning (Karaoglan Yilmaz & Yilmaz, 2022). To ensure students’ attention to these recommendations, it is crucial to emphasise the value and relevance of the strategies for their academic results (Brown et al., 2023; Jovanović et al., 2017). 

Research suggests that the effectiveness of engagement support may be affected by students' prior academic achievement levels (Wise, 2014). However, the interplay between prior academic achievement, support, and engagement remains poorly understood. On the one hand, students with lower prior achievement may benefit more from learning analytics and metacognitive guidance, as they often struggle to monitor and control their learning and they tend to adopt less effective and passive engagement strategies (Yang et al., 2018). On the other hand, high-performing students could gain more from this support, as they are generally more open to adopting new strategies and guidance that aims to improve their academic achievement (Sitzmann et al., 2009). 

To address these gaps, we conducted an experiment whereby students in the first group (the group with support) received personalized reports containing learning analytics on their webinar participation, a list of missed webinars for asynchronous viewing, and recommendations to promote synchronous engagement. However in the second group (the group without support), students did not receive any recommendations regarding engagement. The aim of the study was to examine the impact of such support on student synchronous and asynchronous engagement, as well as student achievement in online learning. We also examine how student prior achievement moderates the effect of support on engagement and performance. We investigate the following research questions:

1. How does providing support affect students’ level of engagement (overall, synchronous, and asynchronous)?

2. How does providing support affect students’ level of academic achievement?

3. How does student prior achievement moderate the effect of the support on engagement and achievement?

426 students enrolled in an online course designed for school graduation exam preparation were asked for consent to participate in this study. 272 students gave their consent to participate. The primary focus of this research was to examine student attendance in webinars, as this activity holds a pivotal role in learning throughout the course. The online course provided flexible webinar participation options for students: they could attend live webinars or watch recorded sessions, available on the LMS. We measured overall engagement by tracking the share of live and recorded webinars each student attended; synchronous engagement by tracking the share of live webinars attended by a student; asynchronous engagement by tracking the share of recorded webinars viewed by a student. Prior achievement was measured using student scores for the first mock exam (scaled 0 to 100) taken at the start of the course. Performance was measured using student scores for the United state examination (scaled 0 to 100). In February, 2024 students were randomly assigned to either the group with or without support. The balance between groups after randomisation was checked based on students' achievement and number of webinars attended during the course. Students from the group with support received a personal report from a tutor every week (for 9 weeks), while students from the group without support were not provided with personal reports. In total, 7 reports were sent to the students from the experimental group. The individual report comprises three key components: Engagement analytics: provides students with their weekly webinar attendance data; Personalized recommendations: offers a checklist of missed webinars with direct links to recordings; Success strategies: highlights the benefits of attending synchronous classes for exam performance. To assess the effect of the support on student engagement we used the non-parametric Mann–Whitney U test. We conducted linear regression analysis with the interaction between student prior achievement and the fact of receiving or not receiving support to evaluate the moderating effect of prior achievement.

The results revealed that students who received support demonstrated significantly higher overall engagement compared to those without support. Our findings also emphasized that support had a greater impact on students' synchronous engagement. The positive effect of the support might also be attributed to the emphasis placed on the utility and relevance of synchronous webinar attendance for academic achievement. Previous research has shown that highlighting the value of specific learning activities and their connection to students' goals can have a positive effect on student engagement (Eccles & Wigfield, 2020). This study didn’t reveal a significant impact of support on learner asynchronous engagement. The findings may be attributed to the factor that asynchronous engagement support included only a checklist of missed webinars to watch without the information about the value and relevance of support for students, which could have prevented students from adopting the recommendations (Brown et al., 2023). The findings show that the support is more beneficial for low-performing students. This may be because low performing students tend to struggle with self-monitoring and often adopt less effective engagement strategies which may have also been the case in our study (Li et al., 2017). While high-performing students did benefit from the support to some degree, the effect was less pronounced for this group of students. This study didn’t reveal a significant impact of support on academic achievement. Students who received support showed slightly higher exam scores, compared to the group without support, however, the difference is not statistically significant. This slight improvement in scores of students who received support may indicate that the intervention had a minor effect on academic achievement, which was not substantial enough to reach statistical significance. Another possible explanation is that the support primarily targeted students’ engagement rather than directly aiming to improve their academic achievement.

Authors, 2024 Brown, A., Basson, M., Axelsen, M., Redmond, P., & Lawrence, J. (2023). Empirical evidence to support a nudge intervention for increasing online engagement in higher education. Education Sciences, 13(2), 145. Eccles, J. S., & Wigfield, A. (2020). From expectancy-value theory to situated expectancy-value theory: A developmental, social cognitive, and sociocultural perspective on motivation. Contemporary educational psychology, 61, 101859. Fredricks, J. A., Blumenfeld, P. C., & Paris, A. H. (2004). School engagement: Potential of the concept, state of the evidence. Review of Educational Research, 74(1), 59–109. Jovanović, J., Gašević, D., Dawson, S., Pardo, A., & Mirriahi, N. (2017). Learning analytics to unveil learning strategies in a flipped classroom. The Internet and Higher Education, 33, 74–85. Karaoglan Yilmaz, F. G., & Yilmaz, R. (2022). Learning analytics intervention improves students’ engagement in online learning. Technology, Knowledge and Learning, 27(2), 449–460. Lee, Y., & Choi, J. (2011). A review of online course dropout research: Implications for practice and future research. Educational Technology Research and Development, 59(5), 593–618. Li, P., Zhou, N., Zhang, Y., Xiong, Q., Nie, R., & Fang, X. (2017). Incremental theory of intelligence moderated the relationship between prior achievement and school engagement in Chinese high school students. Frontiers in Psychology, 8, 1703. Sitzmann, T., Bell, B. S., Kraiger, K., & Kanar, A. M. (2009). A multilevel analysis of the effect of prompting self‐regulation in technology‐delivered instruction. Personnel Psychology, 62(4), 697–734. Staikopoulos, A., OKeeffe, I., Yousuf, B., Conlan, O., Walsh, E., & Wade, V. (2015). Enhancing student engagement through personalized motivations. 2015 IEEE 15th International Conference on Advanced Learning Technologies, 340–344. Steele, J. P., & Fullagar, C. J. (2009). Facilitators and outcomes of student engagement in a college setting. The Journal of Psychology, 143(1), 5–27. Wise, A. F. (2014). Designing pedagogical interventions to support student use of learning analytics. Proceedings of the Fourth International Conference on Learning Analytics And Knowledge, 203–211. Yang, T.-C., Chen, M. C., & Chen, S. Y. (2018). The influences of self-regulated learning support and prior knowledge on improving learning performance. Computers & Education, 126, 37–52.