Introduction

To succeed in the knowledge society, learners and knowledge workers need to combine their expertise and ideas in various collaborative situations, solve problems, and create new information and knowledge. Collaborative problem solving (CPS), which is a specific type of collaboration, has received increasing interest as one of the central 21st century skills suitable for assessment. According to Schwartz (1995), the power of collaborative learning comes from the effort necessary for the group to build a shared understanding. A high-level collaboration does not happen naturally; people vary in the extent of their capability to collaborate with each other. Researchers have shown that when learners are left on their own, they rarely engage in productive interactions and knowledge-generative activities, such as asking each other questions, explaining and justifying their opinions, articulating their reasoning, or elaborating and reflecting upon their knowledge (Kobbe, Weinberger, Dillenbourg, Harrer, Hämäläinen, Häkkinen & Fischer, 2007).  

 In PISA 2015 study, CPS competency is described as the capacity of an individual to effectively engage in a joint and shared activity and, requires sharing their understanding and successfully combining their knowledge, skills and efforts to reach the common goal (OECD, 2013). Collaborative problem solving is also defined as a joint activity between dyads or small groups to transform a current problem state into a desired goal state (Hesse, Care, Buder, Sassenberg & Griffin, 2015). Accordingly, CPS is organised through the use of directly observable, verbal and nonverbal signals. That is, to work successfully, the participants need to communicate, exchange, and share in the process of identifying the parts of the problem; interpreting the connections between the parts, and relationships between action and effect (i.e., rules); and proposing generalizations in search for a shared solution (Hesse, Care, Buder, Sassenberg & Griffin, 2015).  

As a directly observable activity, CPS provides the opportunity to assess a collaborative problem-solving activity through a certain set of qualities that are inherent in a high-quality CPS activity (Hesse, Care, Buder, Sassenberg & Griffin, 2015). CPS is not a uniform process, but a complex, coordinated activity among two or more individuals. Collaborative problem solving, particularly for ill-defined problems, cannot be carefully scripted, but requires learners to take into account several factors that themselves depend on situational affordances (Hesse, Care, Buder, Sassenberg & Griffin, 2015). In more specific terms, the five broad strands defining the collaborative problem solving construct behind the assessment in ATC21S™ portal are: (1) participation (readiness to share information and externalize thoughts), (2) perspective taking (the ability to take the others’ perspectives into account), (3) social regulation (awareness of the strengths and weaknesses of group members), (4) task regulation (planning and monitoring skills for developing strategies for problem solving and shared problem representation), and (5) knowledge building (the ability to learn and build knowledge through group interaction).

Aim and research question

The international research and development project Assessment and Teaching of 21st Century Skills (ATC21S) explores new ways of assessing 21st century skills and links them to teaching interventions aimed at deepening learning and moving student to higher levels of skill (Csapo, P., Ainley, J., Bennett, R.E., Latour, T. Law, N., 2012). The main international objective set for participating countries was to test and validate assessment materials in schools. The field trial using the ATC21S™ tool was organized to collect sufficient data to finalize the scoring and calibration of the assessment tool and thereby provide a picture of students’ CPS skills. The present study utilizes this data and aims to answer the following research question: How do items from the online CPS tasks measure Finnish students' skills across social and cognitive dimensions? 

Adams, R. J., Wilson, M., and Wang, W.-C. (1997). The multidimensional random coefficients multinomial logit model. Applied Psychological Measurement, 21, 1–23. Care, E., Griffin, P., Scoular, C., Awwal, N., Zoanetti N. (2015). Collaborative problem solving tasks. In , P. Griffin, E. Care (Eds.) Assessment and teaching of 21st century skills. Methods and approach (pp. 85–104). Dordrecht: Springer. Csapo, P., Ainley, J., Bennett, R.E., Latour, T. Law, N. (2012). Technological issues for Computer-Based Assessment. In P. Griffin, B. McGaw & E. Care (Eds.) Assessment and teaching of 21st century skills. (pp. 143–230). New York: Springer. Griffin, P., Care, E., and Harding, S. (2015). Task characteristics and calibration. In E. Care, P. Griffin (Eds.) Assessment and teaching of 21st century skills. Methods and approach. (pp. 133–177). Dordrecht: Springer. Harding, S. & Griffin, P. (2016). Rasch measurement of collaborative problem-solving in an online environment. Journal of applied measurement, 17(1), 35–53. Hesse, F., Care, E., Buder, J., & Sassenberg, K., & Griffin, P. (2015). A framework for teachable collaborative problem solving skills. In P. Griffin & E. Care (Eds.), Assessment and teaching of 21st century Skills: Methods and Approach. (pp. 37–56) Dordrecht: Springer. Kobbe, L., Weinberger, A., Dillenbourg, P., Harrer, A., Hämäläinen, R., Häkkinen, P & F. Fischer (2007). Specifying computer-supported collaboration scripts. International Journal of Computer-Supported Collaborative Learning, 2(2-3), 211– 224. Mislevy, R. J. (1991). Randomization-based inference about latent variables from complex samples. Psychometrika, 56, 177–196. OECD (2013). Draft PISA 2015 collaborative problem solving framework. Retrieved 14th of January 2014 from http://www.oecd.org/pisa/pisaproducts/Draft%20PISA%202015%20Collaborative%20Problem%20Solving%20Framework%20.pdf Schwartz, D. (1995). The emergence of abstract representations in dyad problem solving. Journal of the Learning Sciences, 4(3), 321–354.