Collaborative problem solving (CPS) is considered an important skill for academic and workplace success. The term refers to the process involving a number of people working together as equals to solve a problem, thus encompassing the complex processes of problem solving and collaboration. Many initiatives and researchers, globally, have focused on the development of computer-based tasks for the assessment of young people’s CPS competency. For instance, the Assessment and Teaching of 21st Century Skills (ATC21S) project (Griffin & Care, 2015) and the Programme for International Student Assessment (PISA) (OECD, 2017) have recently developed frameworks of CPS, which informed the assessment of such skills.  For the purposes of PISA CPS assessment, CPS competency was defined as “the capacity of an individual to effectively engage in a process whereby two or more agents attempt to solve a problem by sharing the understanding and effort required to come to a solution and pooling their knowledge, skills and efforts to reach that solution” (OECD, 2017, p. 134).

Following the development of the CPS frameworks, the focus of the current literature is on the assessment of CPS competence. Many research groups around the globe focus on how to best assess the construct and are in the process of creating tasks and investigating data analytic techniques for CPS assessments (e.g., Cukurova et al., 2018; von Davier, Zhu, & Kyllonen, 2017). Published computer-based assessments of CPS competence include designs  in which students navigate through tasks as prescribed by multiple-choice pathway options from computer-simulated agents (OECD, 2017) and tasks in which students use free form chat to interact with other humans (Griffin and Care, 2015; Scoular and Care, 2019).

Despite the recent developments in the assessment of CPS competency, there are no widely accepted CPS curricula or standards in school systems at this point (Scoular & Care, 2019), and as argued by Graesser et al. (2018), we are nearly at ground zero in terms of identifying pedagogical approaches to improving CPS skills. Inquiry-based teaching approaches have been widely researched for their relationship with students’ achievement as well as dispositions in various. In science, Jerrim et al.  (2019) found little evidence for the positive association of inquiry-based teaching with students’ performance. On a theoretical basis, problem-based learning, case-based learning, team-based learning, and studio-based learning have been suggested as potentially effective methods for the development of CPS skills (Graesser et al., 2018). On an empirical basis, though, there is a lack of evidence in favour of the aforementioned approaches. 

Student characteristics, such as interpersonal skills, attitudes, emotions, personality factors and motivation have been identified to affect problem-solving success in small scale studies about collaborative learning (Gomez, Wu, & Passerini, 2010; Morgeson, Reider, & Campion, 2005). Knowledge of a particular domain as well as everyday knowledge, can influence students’ capacity to collaborate to solve a problem (OECD, 2017) with research evidence supporting that problem-solving strategies rely, to some extent, on domain knowledge (Mayer, 1992).

This study aims to contribute to knowledge about the relationship between inquiry-based teaching practices and the development of students’ CPS competency. Since CPS is not being explicitly taught as a school subject, it is expected that it is embedded in classroom practice in other traditional subjects such as mathematics and science (OECD, 2017). Hence, students’ perceptions of the inquiry-based teaching they experienced in the subject of science are used to answer the following research questions (RQ):

RQ1: How is students’ CPS competency associated with students’ perceptions of inquiry-based teaching in science?

RQ2: Are there specific components of inquiry-based teaching associated with students’ CPS competency?

Data from the 2015 cycle of PISA for England that capture students’ competence in reading, science and mathematics is used. PISA 2015 focused on science as the major domain and included the assessment of CPS competency for the first time as an innovative domain. Additionally, students completed a background questionnaire about their attitudes and experiences; English participants further completed an ICT-familiarity questionnaire. A total of 5194 students from 206 schools participated in England. The key outcome measure of interest is students’ CPS competency. Based upon students’ responses to the test questions, PISA estimated their competency in each subject area using an item-response theory (IRT) model. This produces a range of possible values (10 plausible values) drawn for each student in each subject area. These plausible values each have a mean of around 500 points and a standard deviation of around 100 points. Analysis was conducted using Stata version 16 (StataCorp, 2019) and the ‘REPEST’ package (Avvisati & Keslair, 2014). The main explanatory variable of interest for this study is the inquiry-based teaching scale. Students responded to nine statements about how often certain activities occur when learning science topics at school. Items were scored so as higher values correspond to higher levels of inquiry-based teaching practices. To account for potential confounders, the following student-level variables were also considered as independent variables in the analysis. - Student demographics (gender, socio-economic status, immigrant status, language spoken at home) - Student attitudes (test anxiety, achievement motivation, attitudes towards collaboration) - ICT usage (ICT usage at school and for leisure) - Mathematics, science and reading literacy scores. Using the data and variables listed above, the associations between students’ CPS competency and inquiry-based teaching practices were investigated by estimating a series of multiple linear regression models. Models were run in the following order to illustrate how parameter estimates changed with the addition of extra variables: - Model 1: only basic demographic characteristics. - Model 2: Model 1 plus inquiry-based teaching practices. - Model 3: Model 2 plus student attitudes and ICT usage. - Model 4: Model 3 plus scores in the PISA mathematics, science and reading test. To answer RQ1, Model 4 including the full set of explanatory variables was the preferred model. To answer RQ2, Model 5 included all explanatory variables as Model 4 except for the inquiry-based teaching variable, which was replaced by students’ responses to the nine items comprising the scale.

This research set out to provide new evidence on the effectiveness of inquiry-based teaching practices and specific components of inquiry-based teaching. The results indicate that students’ perception of inquiry-based teaching has a very weak relationship with students’ CPS competency after accounting their performance in science, mathematics and reading – the only positive effect is observed for the inquiry-based teaching component of doing an investigation to test ideas. A possible explanation for the results might be that the outcome measure used (PISA CPS scores) failed to capture real-world skills which are what inquiry-based teaching aims to develop in students. Despite its novelty and theoretical considerations, PISA’s approach to large-scale assessment of students’ CPS competency has been argued to be limited from at least three perspectives: (i) not considering CPS from the perspective of groups, but rather focusing on the assessment of individuals’ competencies; (ii) presenting CPS competency in a general manner removing all contexts from the operationalisation of the construct, and (iii) using computer-simulated agents in the assessment tasks, which raises concerns about ecological validity (Cukurova et al., 2018; Graesser et al., 2018; Scoular and Care, 2019). It is important that these findings are considered in light of the limitations of this study: (i) the inquiry-based teaching scale is derived from student self-reports, (ii) the hierarchical structure of the data was not taken into account; hence all students were treated as one group, and (iii) results do not capture cause and effect relationships. Further analysis will focus on the use of multilevel models to account for students nested in schools.

Avvisati, F., & Keslair, F. (2014). REPEST: Stata module to run estimations with weighted replicate samples and plausible values (Version revised 05 Jun 2019). Retrieved from https://ideas.repec.org/c/boc/bocode/s457918.html Cukurova, M., Luckin, R., Millán, E., & Mavrikis, M. (2018). The NISPI framework: Analysing collaborative problem-solving from students’ physical interactions. Computers & Education, 116, 93–109. https://doi.org/10.1016/j.compedu.2017.08.007 Gomez, E. A., Wu, D., & Passerini, K. (2010). Computer-supported team-based learning: The impact of motivation, enjoyment and team contributions on learning outcomes. Computers & Education, 55(1), 378–390. https://doi.org/10.1016/j.compedu.2010.02.003 Graesser, A. C., Fiore, S. M., Greiff, S., Andrews-Todd, J., Foltz, P. W., & Hesse, F. W. (2018). Advancing the Science of Collaborative Problem Solving. Psychological Science in the Public Interest, 19(2), 59–92. https://doi.org/10.1177/1529100618808244 Griffin, P., & Care, E. (2015). The ATC21S Method. In P. Griffin & E. Care (Eds.), Assessment and Teaching of 21st Century Skills (pp. 3–33). https://doi.org/10.1007/978-94-017-9395-7_1 Jerrim, J., Oliver, M., & Sims, S. (2019). The relationship between inquiry-based teaching and students’ achievement. New evidence from a longitudinal PISA study in England. Learning and Instruction, 61, 35–44. https://doi.org/10.1016/j.learninstruc.2018.12.004 Mayer, R. E. (1992). Thinking, problem solving, cognition, 2nd ed. New York, NY, US: W H Freeman/Times Books/ Henry Holt & Co. Morgeson, F. P., Reider, M. H., & Campion, M. A. (2005). Selecting individuals in team settings: the importance of social skills, personality characteristics, and teamwork knowledge. Personnel Psychology, 58(3), 583–611. https://doi.org/10.1111/j.1744-6570.2005.655.x OECD (2017). PISA 2015 Assessment and Analytical Framework: Science, Reading, Mathematic, Financial Literacy and Collaborative Problem Solving. https://doi.org/10.1787/9789264281820-en Scoular, C., & Care, E. (2019). Monitoring patterns of social and cognitive student behaviors in online collaborative problem solving assessments. Computers in Human Behavior, 105874. https://doi.org/10.1016/j.chb.2019.01.007 StataCorp. (2019). Stata Statistical Software: Release 16. College Station, TX: StataCorp LLC. von Davier, A. A., Hao, J., Liu, L., & Kyllonen, P. (2017). Interdisciplinary research agenda in support of assessment of collaborative problem solving: Lessons learned from developing a Collaborative Science Assessment Prototype. Computers in Human Behavior, 76, 631–640. https://doi.org/10.1016/j.chb.2017.04.059