Background

The term “21^st century skills” has recently emerged in educational discourses, and is claimed to be important for success in academic and work contexts. The term describes a set of skills including collaborative problem solving (CPS), critical thinking, creativity, and many more. Even though these skills are not necessarily new, the argument is that they point toward a need for general skills taking precedence over the routine procedural skills, in a quickly changing world (Greiff & Kyllonen, 2016).  

Following this growing interest in these skills and aiming to ensure that students are equipped for the demands of their future careers, an increasing number of initiatives have recently developed assessments of students’ CPS skills. The most popular examples at international level involve the 2015 Programme for International Student Assessment (PISA) (OECD, 2017) and the Assessment and Teaching of 21^st century skills (ATC21S) (Griffin & Care, 2015). In a similar vein, several research groups around the globe are in the process of creating tasks for CPS assessments.

One could argue that the complexity around the definition of CPS stems from the fact that the term brings together thinking and research about the separate topics of collaboration and problem solving, both of which have a substantial research history (Cukurova, Luckin, Millán, & Mavrikis, 2018). Collaboration has been defined as “a coordinated and synchronous activity that is the result of a continued attempt to construct and maintain a shared conception of a problem” (Roschelle & Teasley, 1995, p. 70). Specific interest in cooperative learning gathered momentum in the early 1980s with the publication of the first meta-analysis on the effects of cooperative, competitive, and individualistic goal structures on student achievement (Johnson, Maruyama, Johnson, Nelson, & Skon, 1981) followed by a large number of evidence syntheses and more meta-analyses over the years. Problem solving has been defined as “cognitive processing directed at transforming a given situation into a goal situation when no obvious method of solution is available” (Mayer, 1990, p.284). It has been assessed for several decades in mathematics, flourishing also in the 1980s, following Polya’s work (1945) on the four step process: understanding the problem; devising a plan; carrying out the plan; looking back.

For CPS, the vast majority of reviews of research and meta-analyses tend to be based on studies examining cooperative and/or collaborative learning. Even though there are some clear distinctions drawn between such group work pedagogies, these are rarely found in papers or reviews of empirical work and thus, the overlaps between terms create difficulties in analysing the literature (Cukurova, Luckin, & Baines, 2018). Additionally, relevant literature researching CPS has used a range of different but overlapping terms including cooperative learning, collaborative learning, teamwork, group work, peer learning, problem based learning and other terms and phrases. Many authors have used these terms interchangeably, while others have tried to be quite distinct in how they define and use them. Differences in the definition of the term CPS and their implications for the design of assessment tasks are considered significant in informing the assessment and educational communities about the nature of the construct (Scoular, Care, & Hesse, 2017). Therefore, further investigation into the conceptualisation of CPS and its measurement would shed more light in the defining features of this construct and how they are operationalised in research so far.

Aim

This study investigates the conceptualisation and measurement of CPS as presented in the related educational and social research via a systematic review of the academic literature. The study is guided by the following research question:

How is collaborative problem solving (CPS) conceptualised and measured within existing recent studies?

A systematic search using as main keywords “collaborative problem solving” and “student” (student/pupil/learner) was conducted to gather relevant evidence using several electronic databases (i.e. SCOPUS, Web of Science, ERIC, and BEI). The search was applied to the fields of title and abstract of the studies and was restricted to journal articles published in English from 2000 to 2018. Since the aim of the review is to map current research, studies were limited to the last 18 years, starting from the beginning of the 21st century. The keyword search resulted in about 300 studies for further selection. Specifically, titles and abstracts of these studies were screened to assess relevance for inclusion based on the following agreed selection criteria. First, the review targeted student populations in mainstream educational settings, and thus studies examining teacher, parent or professional populations, in special schools or other professional contexts were excluded. Second, studies included in this review were limited to those that had CPS as a main aspect/focus, and therefore studies mentioning CPS only as an example among other aspects of learning were excluded. If a decision for inclusion could not be made by reading only the title and abstract, then the study was included in the sample for further screening on full text. After retrieving full reports of the studies, screening conducted on full text applying an additional inclusion criterion introduced at this stage: i.e. studies had to clearly provide a definition, conceptualisation, framework, or taxonomy of CPS, in order to be included for analysis. The first step of the analysis was to extract information from studies in the form of literal quotes, gathering small paragraphs on definitions of CPS and theories consulted or used. In addition, studies were coded based on the stage of schooling that student population is taken from, the type of data presented, the measurement approach taken for CPS, and the country that research has been conducted. As a next step, the definitions extracted from the studies were studied more closely with the aim to identify and classify defining features central to their conceptualisation of CPS. Strategies such as highlighting central words/phrases and paraphrasing were used when reading through the extracts multiple times. This procedure led to codes that were constantly refined, in order to advance the analysis towards the generation of more general categories.

Preliminary findings from this analysis include about 70 studies that met the inclusion criteria mentioned above and revealed three different categories of conceptualisation of CPS as: (i) an activity, (ii) a teaching approach/method, and/or (iii) a skill/learning construct. CPS as an activity. The first group of studies was characterised by a definition of CPS which emphasised students working together in small groups towards the solution of a shared problem. In this group of studies, CPS has been described as ‘joint activity’, ‘coordinated and synchronous activity’, ‘performance activity’, ‘complex process’, and ‘multilevel process’. CPS as an approach/method. A second group of studies conceptualised CPS as an educational approach helping students to develop higher-order thinking skills and acquire knowledge. In some cases CPS was linked with a particular subject i.e. CPS as a ‘method of mathematics pedagogy’ or specific teaching approaches i.e. CPS as a ‘student-centred approach’. In addition, other learning methods such as collaborative and cooperative learning among others appeared to be central in the conceptualisation of CPS in several studies. CPS as a skill/construct. A final group includes studies that conceptualised CPS as an individual’s skill using various terms such as: ’skill’, ‘competency’, ‘capacity’, ‘ability’. Two CPS conceptualisations that appeared to be central and mostly used by authors are the PISA 2015 and ATC21S. A significant element, which was also echoed in the two frameworks mentioned above, is the conceptualisation of CPS as a combination of social and cognitive skills. This systematic review of the literature is expected to provide more clarification into the conceptualisation of CPS and the way it had been studied and measured in recent studies.

Cukurova, M., Luckin, R., & Baines, E. (2018). The significance of context for the emergence and implementation of research evidence: the case of collaborative problem-solving. Oxford Review of Education, 44(3), 322–337. https://doi.org/10.1080/03054985.2017.1389713 Cukurova, M., Luckin, R., Millán, E., & Mavrikis, M. (2018). The NISPI framework: Analysing collaborative problem-solving from students’ physical interactions. Computers & Education, 116, 93–109. https://doi.org/10.1016/j.compedu.2017.08.007 Greiff, S., & Kyllonen, P. (2016). Contemporary Assessment Challenges: The Measurement of 21st Century Skills. Applied Measurement in Education, 29(4), 243–244. https://doi.org/10.1080/08957347.2016.1209209 Griffin, P., & Care, E. (2015). The ATC21S Method. In P. Griffin & E. Care (Eds.), Assessment and Teaching of 21st Century Skills (pp. 3–33). Dordrecht: Springer Netherlands. https://doi.org/10.1007/978-94-017-9395-7_1 Johnson, D. W., Maruyama, G. M., Johnson, R., Nelson, D., & Skon, L. (1981). The effects of cooperative, competitive, and individualistic goal structures on achievement: a meta-analysis. Psychological Bulletin, 89, 47–62. Mayer, R. E. (1990). Problem solving. In M.W. Eysenck (Ed.), The Blackwell dictionary of cognitive psychology (pp. 284-288). Oxford: Basil Blackwell. OECD. (2017). PISA 2015 Results (Volume V): Collaborative problem solving. Paris: OECD Publishing. https://doi.org/10.1787/9789264285521-en Oliveri, M., Lawless, R., & Molloy, H. (2017). A Literature Review on Collaborative Problem Solving for College and Workforce Readiness. ETS Research Report Series, 2017(1), 1–27. https://doi.org/10.1002/ets2.12133 Pólya, G. (1945). How to solve it: a new aspect of mathematical method (New Princeton Science Library edition.). Princeton: Princeton University Press. Roschelle, J., & Teasley, S. D. (1995). The Construction of Shared Knowledge in Collaborative Problem Solving. In Computer Supported Collaborative Learning (pp. 69–97). Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-85098-1_5 Scoular, C., Care, E., & Hesse, F. W. (2017). Designs for Operationalizing Collaborative Problem Solving for Automated Assessment. Journal of Educational Measurement, 54(1), 12–35. https://doi.org/10.1111/jedm.12130