Collaborative problem-solving (CPS) skills have become an inevitable part of workforce readiness in contemporary society (Graesser et al., 2018). Numerous studies have shown that CPS is a powerful learning tool that could lead to more creative, efficient and comprehensive solutions than other approaches (Fiore, 2008). Sometimes it is the only possible way to solve complex problems. That is not surprising that the Organisation for Economic Cooperation and Development (OECD, 2019) includes the development of collaboration skills in the education development agenda for 2030. A lot of attempts were made to introduce CPS in everyday educational practice. However, the benefits of the CPS often fail to be achieved (Le et al., 2018). 

Collaborative problem-solving is usually defined as working together toward a common goal (Hesse et al., 2015). It includes interdependency between group members in joint activity and shared responsibility for the group results. 

Despite many contributions, there is a lack of instruments for measuring student-specific versions of collaborative processes during group work (Wang et al., 2009). The focus is often on the effect of this type of learning assessed through achievement data (Jansen, 2010) while the quality of the collaborative process is beyond research aims.  Usually, self-assessment tools were used for this purpose accompanied by methodological limitation of subjective assessments. In these attempts, students' perceptions and experience with CPS are not distinguished from the quality of collaboration present during group work. Also, collaboration is assessed as an individual skill separate from its nature as a joint activity. This study aims to construct an instrument for assessing the quality of collaboration between students while trying to solve a complex problem. This study is part of the larger project PEERSovers with a focus on designing an evidence-based training program for enhancing high-school students' collaborative skills. The theoretical background for constructing the instrument involves a qualitative systematic literature review of 160 articles published between 2021 and 2022 that investigated differences between productive and unproductive peer collaboration (Baucal et al., 2023). Four aspects of peer interaction were identified as a result of this analysis. The first covers cognitive exchange between group members. Research shows that productive CPS includes argumentative dialogue between team members and constructive evaluation of ideas. Also, the effort is made to move from the personal opinion toward a shared understanding of the problem. Well-known Mercer studies (for example, Mercer et al., 2019; Mercer & Dawes, 2014) pointed out that exploratory talk during group work enhances the co-construction of joint cognitive activity, fosters critical thinking skills and contributes to the overall learning experience in educational settings. The second aspect refers to the emotional aspect of group work manifested through group atmosphere, presence of conflicts and tension, group cohesion, members' sense of belonging, mutual tolerance and empathy. In unproductive groups, members are disrespected and prevented from fully participating. Often the inequality in power is present during group work. Some members dominate in the dialogue space and prevent others from contributing. The third and fourth aspects are dedicated to two domains of group regulation: task activity regulation (time management, coordination of the activity, planning group activity, task-focus approach) and relationship regulation (group norms, sharing responsibility, dividing the assignment, efficient conflict management etc.). An unproductive group is often characterised by lots of off-task behaviour. Usually, few or only one participant takes overall responsibility for group work. We tried to operationalize these four aspects as dimensions of the instrument used for evaluating a CPS.

Sample: Participants were selected from 12 secondary schools in Belgrade (6 vocational and 6 general/gymnasium schools). School counsellors, guided by the students’ preferences, formed triads of male or female students from the same class. The sample included 64 groups of three students (192 participants), of which 37 were girls and 27 were men. All students involved in the research had formal parental consent and their assent. Procedure: Students’ triads participate in CPS sessions trying to solve a single but complex real-life problem. Problem tasks used in this study were related to four community-relevant themes: (1) ecology (2) teen behaviour, (3) media, and (4) education. The assigned task for each group involved generating a written solution to the presented problem, subsequently assessed for its quality. The entire interaction during the CPS process was video-recorded video for subsequent analysis. CPS sessions were conducted on school premises during the regular school day. The average duration of a CPS session was 97 minutes (SD = 30; range = 19-167). Instruments: CPS observational grid (CPS-OG). The quality of collaboration was assessed based on video recordings of CPS sessions. Each session was rated by two independent reviewers using a 22-item observational grid. The grid was designed to capture four dimensions of productive CPS: socio-cognitive (SC - 9 items, 2 reverse-scored; e.g., Group members sought and/or provided explanations for presented ideas and suggestions); socio-emotional (SE - 4 items, 1 reverse-scored; e.g., Group members worked together, as a team); task management (TM - 5 items, 1 reverse-scored; e.g., The group planned its approach to solving the task); relationship management (RM - 5 items, 2 reverse-scored; e.g., Throughout the work, group members purposefully coordinated group and individual activities). Each item is scored on a 5-point Likert scale, ranging from 0 (not at all) to 4 (to a large extent). Data Analyses: Analyses were performed to examine the structural and reliability properties of measures designed specifically for this study. The unidimensionality of CPS-OG subscales was inspected via Exploratory Factor Analysis (EFA) and Confirmatory Factor Analysis (CFA). Internal consistency for CPS-OG dimensions was determined by calculating Cronbch’s alpha coefficient.

The results confirmed good psychometric characteristics of the CPS observational grid. Exploratory factor analysis (Principal component analysis with Oblimin rotation) resulted in four factors explaining 76% of the total variance. Correlations between factors were moderate with a maximum value of 0.44. The first factor (50% of the variances) mainly included SC variables. The second factor (15 % of the variances) corresponds to the TM dimension. The third (6%) factor represents a mix of the SE and RM variables. It includes statements about negative relationships in the group (present tension, conflicts and isolation of the members). Finally, the fourth factor (5% of the variance) covers the absence of an authoritative leader and good conflict management as aspects of the RM dimension. The correlations between the first factor and the other three are moderate (from -0.33 to 0.44). The correlations between the other factors are low. Confirmatory factor analysis (CFA) confirmed a single-factor solution for all dimensions, except the TM. Item-level intraclass correlation (ICC) for CPS observational grid (CPS-OG) indexes reached excellent values (Cicchetti, 1994), ranging from .75 to .95. Dimension-level ICC values were also excellent: .94 for SC, .90 for SE, .93 for TM, .85 for RM. Internal consistency (Cronbach's alpha) ranges from good to excellent (.921 for SC, .914 for SE, .856 for TM, .791 for RM.) The next research step will include the external validation of the instrument. We will examine the association between the dimensions of the CPS observational grid and the quality of the proposed group solution. The quality of the solution will cover several dimensions: whether the solution is realistic; an assessment of the proposal's creativity; an assessment of the degree to which the proposal is well-argued with various perspectives.

Baucal, A., Jošić, S., Ilić, I. S., Videnović, M., Ivanović, J., & Krstić, K. (2023). What makes peer collaborative problem solving productive or unproductive: A qualitative systematic review. Educational Research Review, 100567. https://doi.org/10.1016/j.edurev.2023.100567 Fiore, S. M., Graesser, A., & Greiff, S. (2018). Collaborative problem solving education for the 21st century workforce. Nature: Human Behavior, 2(6), 367–369. https://doi.org/10.1038/s41562-018-0363-y Graesser, A. C., Fiore, S. M., Greiff, S., Andrews-Todd, J., Foltz, P. W., & Hesse, F. W. (2018). Advancing the science of collaborative problem solving. Psychological science in the public interest, 19(2), 59-92. https://doi.org/10.1177/1529100618808244 Hesse, F., Care, E., Buder, J., Sassenberg, K., & Griffin, P. (2015). A framework for teachable collaborative problem solving skills. Assessment and teaching of 21st-century skills: Methods and approach, 37-56. DOI: 10.1007/978-94-017-9395-7_2 Le, H., Janssen, J., & Wubbels, T. (2018). Collaborative learning practices: teacher and student perceived obstacles to effective student collaboration. Cambridge Journal of Education, 48(1), 103-122. https://doi.org/10.1080/0305764X.2016.1259389 Mercer, N. & Dawes, L (2014). The study of talk between teachers and students, from the 1970s until the 2010s. Oxford Review of Education, 40 (4) (2014), pp. 430-445. https://doi.org/10.1080/03054985.2014.934087 Mercer, N., Hennessy, S., & Warwick, P. (2019). Dialogue, thinking together and digital technology in the classroom: Some educational implications of a continuing line of inquiry. International Journal of Educational Research, 97, 187–199. https://doi.org/10.1016/j.ijer.2017.08.007 OECD. (2019). An OECD Learning Framework 2030 (pp. 23-35). Springer International Publishing. Wang, L., MacCann, C., Zhuang, X., Liu, O. L., & Roberts, R. D. (2009). Assessing teamwork and collaboration in high school students: A multimethod approach. Canadian Journal of School Psychology, 24(2), 108-124. DOI: 10.1177/0829573509335470