The purpose of this study is to elucidate how incorporating time for students to visualize and perceive changes in the entire class and individual students before and after a lecture using text-mining can influence their level of participation, interest, learning outcomes, and depth of learning in large-group lectures.

Large-group lectures pose significant challenges in maintaining undergraduate students' interest and developing their subject-specific knowledge and skills. To address these challenges, active learning methods have been integrated into large-group lectures in higher education. Additionally, learning management systems (LMS) and e-portfolios have been developed as lecture support systems.

Previous research has identified key factors that contribute to the effectiveness of learning activities in large-group lectures. Jerez et al. (2021) highlighted five themes that facilitate effective large-group learning: (1) student-teacher and student-student interaction, (2) implementation of active learning strategies, (3) classroom management, (4) students' motivation and commitment, and (5) the use of online teaching resources. McCoy et al. (2018) examined Learner-Centered Education (LCE) from five perspectives: (1) world relevance, (2) competency-based learning, (3) collaboration, (4) deliberative practices, and (5) technology/multimedia. Their analysis revealed that a five-point review process of lectures led to curriculum inventory, faculty consensus, goal setting, identification of practice gaps, and instructional improvement.

Building on these findings, we designed learner-centered learning activities by incorporating real-world, authentic topics for students, pair and group activities in face-to-face large-group lectures, and utilizing technology. After the lecture, students were required to post their reflections on their learning on the LMS bulletin board within one week and comment on each other's posts. However, while pair and group discussion activities in face-to-face large-group lectures seemed to promote active learning, the depth of the content discussed was often insufficient.

To address this, we incorporated the flipped learning method into our large-group lecture format. Through the flipped learning method, in which students read and watch materials related to their expertise and the basic concepts covered in face-to-face lectures in advance, they participated more actively and engaged in deeper discussions than in previous lectures. However, not all students read the distributed materials and video materials in advance, leading to variations in the depth of discussions among pairs and groups. Additionally, the content of the reflection statements posted after the lectures did not change significantly. Fewer students wrote about how their thinking changed before and after the lecture and what they learned throughout the series of lectures, which was expected with the introduction of flipped learning. Instead, they described their learning progress in their submitted assignments.

Although evaluation comments from teachers and teaching assistants (TAs) on each student's reflections can enhance learning, it is challenging to provide comments to more than 250 students each time. Securing the budget to hire many TAs and the time required to train them to write effective evaluation comments was difficult. Therefore, it was considered necessary to develop a system to automatically provide appropriate evaluation comments to students. However, securing the cost and time required to develop such a system was not easy. This highlighted the need for alternative views and strategies in lecture design to encourage students to reflect on the lectures, connect the reflections of each lecture, and lead to deeper learning.

To sustain interest and improve the professional knowledge and skills of undergraduate students from diverse backgrounds in large-group lectures, we introduced text-mining as a novel method. The research question was whether presenting students with the results of their learning using this method would enhance their awareness of learning trends and individual outcomes, thereby sustaining their interest and encouraging reflection on their learning.

This study involved 251 second- to fourth-year undergraduate students who participated in 15 media lectures from April to July 2022. Based on the results of this trial study, a follow-up study was conducted from April to July 2023 with 554 and 2024 with 438 second- to fourth-year students to confirm the teaching methods and examine the outcomes of the improved methods. The purpose and objectives of the study were explained to the students, and only the statements of those who provided consent were included in the analysis. Undergraduate students were required to complete a pre-assignment on the basic concepts and analytical methods to be covered in the upcoming lecture. At the beginning of each lecture, students answered a quiz on their understanding of these concepts using Google Forms. During the lecture, students were tasked with analyzing media use based on the expertise and knowledge from the pre-assignment materials. For example, they were asked to analyze the differences between the layouts of print and digital newspapers or compare the reliability of TV news and online news. These tasks were discussed in pairs or groups. After the lecture, students were asked to reflect on the lecture and write their thoughts on the Learning Management System (LMS) during a five-day post-assignment period. The quiz questions for the pre- and post-assignments were identical, focusing on descriptive explanations to assess the depth of understanding for both teachers and students. A rubric with four gradations was provided to guide students on what constitutes an excellent reflection statement. The results of the pre- and post-assignment writings were text-mined and presented to the students at the beginning of each lecture as a review of the previous session to highlight the changes that had occurred. Summary sentences from the pre- and post-assignments were compared and presented using an automatic summarization function that extracts sentences with many important words and high similarity to other sentences. For this purpose, we used UserLocal AI text-mining with this function (https://textmining.userlocal.jp/).

Interest in 2022 was significant (t(169) = 10.538, p < 0.01, cohen d = 1.350) as a result of a t-test with correspondence between the starting and ending points. Interest in 2023 was also significant (t(369) = 8.328, p < 0.01, cohen d = 1.658) as a result of a t-test with correspondence between the starting and ending points. Interest in 2024 was also significant (t(246) = 7.736, p < 0.01, cohen d = 1.730) as a result of a t-test with correspondence between the starting and ending points. The results on a 5-point scale showed that students' knowledge and skills (Q1-Q3) were above 4.0, and their thinking, judgment, and expression skills (Q4-Q6) were close to 4.0. The free evaluation of the lectures in 2022,2023,and 2024 included the following feedback: 1. Summarizing overall trends in assignments was interesting. 2. Continued feedback was appreciated. 3. Knowing various opinions on the bulletin boards was beneficial. 4. The analysis sheet used in the lecture was easy to understand. 5. The opportunity to analyze was enjoyable. 6. Writing was challenging. The efforts in 2022 ,2023 and 2024 using text-mining use of text mining to visualize pre-assignments and post-assignments to students demonstrated the potential for meaningful improvements in teaching methods. This conclusion is based on the analysis of pre- and post-project description results, interest change results, learning outcome self-assessment results, and class evaluation results. Regarding the reflection and deepening of learning, students' comments highlighted the significance of using pre-assignments and enhancing analytical activities within lectures and discussions, even in large groups.

Goedhart, N. S., Blignaut‑van Westrhenen, N., Moser, C., & Zweekhorst, M. B. M. (2019). The flipped classroom: supporting a diverse group of students in their learning. Learning Environments Research 22:297–310. https://doi.org/10.1007/s10984-019-09281-2 Jerez, O., Orsini, C., Ortiz, C. & Hasbun, B. (2021). Which conditions facilitate the effectiveness of large-group learning activities? A systematic review of research in higher education. Learning: Research and Practice 7(2): 147–164. https://doi.org/10.1080/23735082.2020.1871062 Lillard, A., & Taggart, J. (2022). Reimagining Assessment in a Large Lecture: An Alternative Approach Inspired by Thomas Jefferson and Maria Montessori, College Teaching, https://doi.org/10.1080/87567555.2022.2140097 McCoy, L., Pettit, R.K., Kellar, C., & Morgan, C. (2018). Tracking Active Learning in the Medical School Curriculum: A Learning-Centered Approach. Journal of Medical Education and Curricular Development 5: 1–9. https://doi.org/10.1177/2382120518765135 van Alten, D. C.D., Phielix, C., Janssen, J., & Kester, L.(2019). Effects of flipping the classroom on learning outcomes and satisfaction: A meta-analysis. Educational Research Review 28, November. https://doi.org/10.1016/j.edurev.2019.05.003