According to the collaborative diagnostic reasoning framework (CDR; Radkowitsch et al., 2022), the process of diagnosing builds upon information-gathering referring to individual (e.g., generating hypotheses) and collaborative activities (e.g., sharing evidence) to reduce uncertainty and make professional decisions (Richters et al., 2023). Previous studies suggest that collaborative diagnosing has advantages compared to individual only, in particular when students working together are prepared (e.g., Zambrano et al., 2019; Lam & Kapur 2017). The rationale is that by activating prior knowledge and generating self-explanation in the preparatory phase, group members benefit more from the subsequent collaborative learning activities (e.g., Mende, Proske & Narciss 2020; Chen, Lyu & Su 2024).

The aim of the study is to investigate the preparatory effects (e.g., for collaboration: Lam & Kapur 2017; Mende, Proske & Narciss 2020; see also problem solving: Hartmann, van Gog & Rummel 2021) on collaborative learning in a simulation-based learning environment. We developed a simulation that presents information about the mathematical skills of simulated students via a teacher dashboard. The pre-service teachers were asked to take over the role of a teacher and to diagnose students` mathematical skills. The simulation allows to approximate real-life diagnostic problems in a risk-free environment without overwhelming learners (Chernikova, Fiedler & Jansen 2023; Fischer & Opitz 2022). Moreover, the use of a technology-enhanced simulation allows to collect and integrate multimodal process data (e.g., log files, thinking aloud protocol via audio recording) for a better understanding of the diagnostic process. The study focuses on the individual and collaborative activities (CDAs: Fischer et al., 2014), in which students engage with or without preparation during the diagnostic process and how these activities influence learning outcomes.

A total of 150 primary school pre-service teachers (87% females; Mage = 22.53, SDage = 2.95) participated in the study. In a quasi-experimental pre-test post-test design, the pre-service teachers were randomly assigned to the three experimental conditions: (1) individual only (2) collaborative only (3) individual and collaborative combined and a control group. While pre-service teachers in condition two (Ngroups= 16) worked collaboratively in small groups without a preparatory phase, participants in condition three (Ngroups= 14) entered collaboration after an individual preparatory phase. The short-term intervention (100-min.) was integrated into a teacher education course. We assessed pre-service teachers diagnostic reasoning skills with a pre-/post-test (e.g., estimation and justification of task complexity, EJTC; diagnostic judgement with teacher dashboard, DJTD). We coded the answers to the open-ended questions according to a developed coding scheme. For 16 % of all participants, data were coded by three trained raters. Overall, the coefficients indicated a fair level of interrater-reliability (AC1/AC2-t1 = .69 - .86; AC1/AC2-t2 = .47 - .94; Gwet, 2021) and the remaining answers were coded by one rater. Additionally, pre-service teachers responded to questions about digital media use in teaching, that is, their perceived knowledge (4 items: ω = .84; Guggemos & Seufert, 2021), experience (4 items: ω = .90; Hahn, Pfeifer & Kunina-Habenicht, 2022) and value beliefs (3 items: ω = .79; Quast, Rubach & Lazarides, 2021). We applied propensity score matching via the R-package MAGMA (Urban et al., 2024) using these measures as well as study subject and subject-related semester as covariates. Matching pre-service teachers on these variables reduced the original sample size to four equal subsamples (n = 29). Additionally, to consider that the small groups comprised pre-service teachers studying mathematics either as a minor or main subject, we used a variable of group composition for exact matching both collaborative conditions. This reduced the original sample size to two equal subsamples (n = 24). However, power analyses showed that sample size still was appropriate to uncover relevant effects of medium size. The matched groups were compared using ANOVA for repeated measures. Moreover, we collected process data to evaluate the individual and collaborative activities in the simulation-based learning environment. The process data consist of pre-service teachers` written reasoning reports, log file data and audio-recorded think-aloud protocols (preparatory and collaboration phase), which were coded with regard to diagnostic activities (e.g., generating hypotheses, sharing evidence, Fischer et al. 2014) and quality of reasoning.

Our analyses reveal a significant interaction effect between time and condition, F(3, 103) = 8.56, p < .001, η2 = .10, for diagnostic reasoning (EJTC), showing that pre-service teachers across all experimental groups had significant learning gains and the experimental groups outperformed the control group. Furthermore, the comparison of both collaborative conditions show no significant main effect, F(1, 46) = 1.25, p = .270, η2 = .01, as both conditions performed equally well over time, F(1, 46) = 40.27, p < .001, η2 = .32. That is, pre-service teachers in both collaborative conditions with and without preparatory phase benefit, but the advantage of individual preparation was not be founded. With regard to diagnostic reasoning (DJTD) we find no significant interaction effect, F(3, 103 = 0.13, p = .945, η2 < .01, as all conditions performed equally well over time, F(1, 103) = 5.35, p = .023, η2 = .02. However, descriptive statistics show higher scores for all experimental groups compared to the control group. Also, the comparison of both collaborative conditions shows no significant effects. For the missing preparatory effect, it might be assumed that individual preparation facilitates task-related diagnostic activities but also increases effort to collaborate and establish a shared understanding as openness for alternative perspectives is reduced. Therefore, to gain a deeper understanding of the individual and collaborative diagnostic activities, the paper will address this by using the process data in further moderator analysis. In sum, our findings highlight that the instructional support within simulations needs further theoretical consideration and empirical evidence. For future research, it seems fruitful not only to investigate instructional support during collaboration (e.g., collaboration scripts) but beforehand.

Chen, W., Lyu, Q., & Su, J. (2024). The role of individual preparation before collaboration: An exploratory study on students’ computer-supported collaborative argumentation in a primary classroom. Journal of the Learning Sciences, 33(4-5), 757–798. Chernikova, O., Fiedler, D. & Jansen, T. (2023). Facilitating diagnostic competences with simulations. German Journal of Educational Psychology (Zeitschrift für Pädagogische Psychologie) 38(1-2), 1-2. Fischer, F. & Opitz, A. (2022). Learning to Diagnose with Simulations. Examples from Teacher Education and Medical Education. Springer. Fischer, F., Kollar, I., Ufer, S., Sodian, B., Hussmann, H., Pekrun, R., et al. (2014). Scientific reasoning and argumentation: Advancing an interdisciplinary research agenda in education. Front. Learn. Res. 4, 28–45. Guggemos, J., & Seufert, S. (2021). Teaching with and teaching about technology–Evidence for professional development of in-service teachers. Computers in Human Behavior, 115, 106613. Gwet, K. L. (2021). Handbook of inter-rater reliability: The definitive guide to measuring the extent of agreement among raters – Vol. 1 Analysis of categorical ratings (5th Ed.). Advanced Analytics, LLC. Hahn, S., Pfeifer, A., & Kunina-Habenicht, O. (2022). Multiple facets of self-rated digital competencies of pre-service teachers: A pilot study on the nomological network, empirical structure, and gender differences. Frontiers in Education, 7. Hartmann, C., van Gog, T., & Rummel, N. (2021). Preparatory effects of problem solving versus studying examples prior to instruction. Instructional Science, 49(1), 1–21. Lam, R., & Kapur, M. (2017). Preparation for future collaboration: Cognitively preparing for learning from collaboration. Journal of Experimental Education, 86(4), pp. 546-559. Mende, S., Proske, A., & Narciss, S. (2021). Individual preparation for collaborative learning: Systematic review and synthesis. Educational Psychologist, 56(1), 29–53. Quast, J., Rubach, C. & Lazarides, R. (2021). Lehrkräfteeinschätzungen zu Unterrichtsqualität mit digitalen Medien: Zusammenhänge zur wahrgenommenen technischen Schulausstattung, Medienunterstützung, digitalen Kompetenzselbsteinschätzungen und Wertüberzeugungen. Zeitschrift für Bildungsforschung 11, 309-341. Radkowitsch, A., Sailer, M., Fischer, M. R., Schmidmaier, R., & Fischer, F. (2022). Diagnosing collaboratively: A theoretical model and a simulation-based learning environment. In F. Fischer & A. Opitz (Eds.), Learning to Diagnose with Simulations. Springer. Richters, C., Stadler, M., Radkowitsch, A., Schmidmaier, R., Fischer, M. R., & Fischer, F. (2023). Who is on the right track? Behavior-based prediction of diagnostic success in a collaborative diagnostic reasoning simulation. Large-Scale Assessments in Education, 11(1), 3. Zambrano, J., Kirschner, F., Sweller, J. & Kirschner, P.A. (2019). Effects of group experience and information distribution on collaborative learning. Instructional Science, 47, 531–550.