OECD’s Program for International Student Assessment has gained a strong interest by its intention to measure the skills students need in real life and out of the school. One of these skills is creative problem solving, which has been in the scope of PISA studies’ so called changing domains two times in years 2003 and 2012. The first assessment was conducted as paper based test and the latter study was computer based. Problem solving has been defined as one of essential 21^st century skills in many of the existing frameworks (Ananiadou & Claro, 2009; Binkley et al., 2012; Gordon et al., 2009; Pepper, 2011; P21Skills, 2013).

Collaborative problem solving (CPS), which is a specific type of collaboration, has received increasing interest as one of the central 21st century skills suitable for assessment (Binkley et al, 2012; Hesse et al., 2015). In PISA 2015 study, student collaborative problem solving skills were assessed as an innovative domain. The competency of collaborative problem solving is described as the capacity of an individual to effectively engage in a joint and shared activity and, requires sharing their understanding and efforts and successfully combining their knowledge, skills and efforts to reach the common goal (OECD, 2013).

Perseverance has found to be among the important factors that correlate with success in studies (Duckworth & Cross, 2014; OECD, 2014). Grit has been defined as perseverance and passion for long-term goals, as well students with low potential but high level of perseverance, are more probably successful than students, who have high potential but low level of perseverance (Duckworth, Peterson, Matthews, & Kelly, 2007). In this sense perseverance is closely related to both longer term goal orientations (Scherer, Greiff, & Hautamäki, 2014) and task oriented self-regulatory processes of learning (Schunk & Zimmerman, 2008). PISA 2012 Volume 3 reports perseverance involving stamina and persistence to remain in the task, which is essential in achieving goals and completing tasks (OECD, 2013). In PISA 2015 student’s perseverance was not similarly measured. There was no index for perseverance nor identical single items related directly with perseverance. Still, when perseverance did have rather significant effect in performance in some countries in PISA 2012, the aim of this study is to examine what kind of latent factors may have similar association with PISA 2015 collaborative problem solving. Effects of potentially intervening student level background variables measuring students’ goal orientation and motivation in PISA 2015 student questionnaire are considered.

The association between problem solving performance and student’s perseverance in PISA 2012 study clearly indicates that there are connections between high performance and perseverant attitude towards working in the creative problem solving tasks (OECD, 2014). If students did not work hard and perseverant manner, they will most probably not achieve mastery level. The association between achievement and perseverance was the highest in Finland, where 30 points of the difference of problem solving score (mean score 523 points) was associated with the index of perseverance. In this study we chose three countries with strong association and three countries with weak association with perseverance for further examination. In addition to Finland, countries with strong association were Sweden 25 (491), and Norway (503) 22 points. The countries with low association were Netherlands 6 (511), Estonia 1 and Italy (510) 0 points, the OECD average being 15 points. The correlation between creative problem solving in PISA 2012 and collaborative problem solving was within OECD countries overall r=0.70, but when controlling the effect of other domains the correlation was only r=0.26. Performance of collaborative problem solving in PISA 2015 in these selected countries was: Finland (mean score 534 points), Sweden 510, Norway 502, Netherlands 518, Estonia 535 and Italy 478. 

We examined a group of three countries with a strong relation between achievement and perseverance in PISA 2012 and three countries with weak or no such relation. Sample represents the 15-year-old students from six OECD countries: Estonia, Finland, Italy, Netherlands, Norway, and Sweden. We use only the datasets of the students who took the perseverance items and the final sample is comprised based on that selection. In PISA 2012 the index of perseverance (PERSEV) was constructed using student responses over questionnaire item ST93, whether they reported that the following statements describe them very much, mostly, somewhat, not much, not at all: In When confronted with a problem, I give up easily; I put off difficult problems; I remain interested in the tasks that I start; I continue working on tasks until everything is perfect; when confronted with a problem, I do more than what is expected of me (OECD, 2013). Items related with students’ perseverance did not remain the same from 2012 to 2015 study. Still, some scales and single items were measuring somewhat similar construct. ST 119 was answering students’ goal orientation and ambitions with following four items on achievement motivation: I want top grades in most or all of my courses, I want to be able to select from among the best opportunities available when I graduate, I want to be the best, whatever I do, I see myself as an ambitious person, I want to be one of the best students in my class. ST 121 was measuring students’ motivation with three items: is motivated? Gives up easily when confronted with a problem and is often not prepared. is motivated? Mostly remains interested in the tasks she starts and sometimes does more than expected. is motivated? Wants to get top grades at school and continues working on tasks until perfect. As a start, a confirmatory factor analysis (CFA) was conducted with Mplus to find out the suitability of the used items for PERSEV index in PISA 2012 and find possible latent factor effecting CPS in selected countries. Still, the main tool of statistical analysis is multivariate multilevel regression modelling, supported with basic descriptive statistics. The regression models are employed in examining the sizes of the effects of perseverance and other variables on the problem solving and CPS scores in six countries, as well as the relation between perseverance and background variables.

Students’ reported perseverance has effect on problem solving score in dual means according to analysis in Finnish data from PISA 2012 (Ahonen & Nissinen, 2015). Firstly, the presence of high level of perseverance effects significantly on the students’ scores. Supported by CFA these items were put on Principal component analysis and build as two components “Willingness for effort” and “Pursuit for good performance”. These two components have distinct associations with problem solving scores in each country, and in all countries they explained more of the variance of problem solving in PISA 2012 than the PERSEV-index alone. “Pursuit for good performance” items explained 7 % of variance in Finland, 2 % in Sweden and 3% in Norway, but in Netherlands and Italy one per cent and none of the variance of creative problem solving scores. Meanwhile “Willingness for effort” items had significantly higher explanatory percentages in Nordic countries FIN 8, SWE 10 and NOR 6 per cent. PISA 2015 student questionnaire items were measuring goal orientation (ST121) and motivation (ST119) with relatively solid construct validity in the selected six countries. When completing a multilevel multivariate regression analysis on PISA 2015 CPS scores it appeared that MOTIVAT items explained somewhat similar share of the variance as “Pursuit for good performance” variables did explain in PISA 2012 creative problem solving in NED, ITA and EST but they did not so in Finland, Sweden and Norway, explaining 2, 2 and 4 per cent respectively. It also appeared that ST 121 items explanatory force is much stronger, explaining 15 per cent of the variance in Estonia, ITA 8 %, NED 9%, SWE 9 %, NOR 10% and FIN 7 %. This paper discusses these findings as well as explores for other solutions on measuring latent factor effects on collaborative problem solving and perseverance.

Ahonen, A.K. & Nissinen, K. (2015) Ongelmanratkaisusta ymmärrykseen. [Problem solving in PISA 2012, results in Finland]. In J. Välijärvi & P. Kupari PISA 2012 tutkimustuloksia. Millä eväillä uuteen nousuun? Opetus-ja kulttuuriministeriön julkaisuja 2015:6.Opetus- ja kulttuuriministerio, (pp. 50–71). Ananiadou, K., & Claro, M. (2009). 21st century skills and competences for new millennium learners in OECD countries. (OECD Education Working Papers. No. 41.) Paris: OECD. Binkley M., Erstad, O., Herman J., Raizen, S. Ripley, M., Miller-Ricci, M., & Rumble, M. (2012). Defining Twenty-First Century Skills. In P. Griffin, B. McGaw & E. Care (Eds.) Assessment and Teaching of 21st Century Skills. (pp. 17–66). New York: Springer. Duckworth, A., Peterson, C., Matthews, M., & Kelly, D. (2007). “Grit: Perseverance and passion for long-term goals”, Journal of Personality and Social Psychology, 92(6), pp.1087-1101. Gordon, J., Halasz, G., Krawczyk, M., Leney, T., Michel, A., Pepper, D., & Wiśniewski, J. (2009). Key competences in Europe. Opening doors for lifelong learning. No. 87. Warsaw: CASE - Center for Social and Economic Research. Hesse, F., Care, E., Buder, J., Sassenberg, K., & Griffin, P. (2015). A framework for teachable collaborative problem-solving skills. In P. Griffin & E. Care (Eds.), Assessment and teaching of 21st century skills: Methods and approach (pp. 37–56).Dordrecht: Springer. https://doi.org/10.1007/978-94-017-9395-7_2 OECD (2017). PISA 2015 technical report. Paris: OECD Publishing. OECD (2013). PISA 2012 results: Ready to learn – Students’ engagement, drive and self-beliefs (vol. III) Paris: OECD Publishing. OECD (2014). PISA 2012 results – Creative problem solving (vol. V) Paris: OECD Publishing. P21Skills (2013). Framework for 21st century learning. Partnership for 21st century skills. Retrieved from http://www.p21.org/about-us/p21-framework Pepper, D. (2011). Assessing key competences across the curriculum and Europe. European Journal of Education, 46(3), 335–353. Scherer, R. Greiff, S. Hautamäki, J. (2015). Exploring the relation between time on task and ability in complex problem solving. Intelligence, 48(1), 37–50. Schunk, D.H., & Zimmerman, B. J. (Eds.) (2008). Motivation and self-regulated learning: Theory, research, and applications. NY: Lawrence Erlbaum Associates.