The need for education to go beyond curricular knowledge and skills to meet the expectations of the 21^st century has put pressure on the enhancement of students’ general thinking skills as the core of lifelong learning (Cattle & Howie, 2008). Accordingly, concepts such as key competencies, transversal skills, learning to learn, and 21st century skills (Rocard et al., 2007; OECD, 2009; OECD, 2013; OECD, 2015) have been added to curricula across the OECD. The common core for all these new or newly introduced concepts is that they capture the general cognitive competences underlying subject-specific knowledge and skills as well as domain-general skills to allow the application of knowledge and skills to novel situations (Vainikainen, Hautamäki, Hotulainen, & Kupiainen, 2015). Accordingly, education for thinking has recently received considerable interest in the literature and at an organization level (European Council, 2006; Kuhn, 2005; Rocard et al., 2007; OECD, 2010). In few countries, however, thinking skills are but mentioned as an overarching theme in curricula. This is also the case in Finland, the context of the current study, even if the assessment of general cognitive competence under the somewhat fuzzy notion of learning to learn has been widespread in both municipal and nation level assessments since the mid 1990s (Hautamäki & Kupiainen, 2014).

The introduction of computer-based assessment (CBA) has enriched assessment through the affordance of log data of response time (RT) as an indicator of student effort, a crucial prerequisite for the reliability and validity of assessment results (Barry et al. 2010; Lee & Chen, 2011; Wise, 2006). Interpreted as an indicator for effort or engagement, RT relates to Carroll’s (1963) earlier notion of time-on-task (TOT) in the context of learning, which features regularly in meta-analyses of factors pertaining to learning and school achievement (Hattie, 2005; Scheerens & Bosker, 1997). TOT has proven a valuable and useful indicator on the side of other performance-enhancing motivational constructs based on data acquired through self-report questionnaires (Goldhammer et al., 2014; Kupiainen et al., 2014; Wise & Kong, 2005).

The present study incorporates two strands of educational research and theory: The relative impact of cognitive competence and motivational factors in explaining test and school achievement (c.f., Adey et al., 2007; Demetriou et al., 2011; Spinath et al., 2006) and the role of TOT on students’ attainment in low-stakes assessment of general cognitive competence (Kupiainen & Hotulainen, 2018). The study presents a longitudinal set-up of two age cohorts with three time-points within one and a half years of basic education (from the beginning of grade 5 to grade 6 and from the beginning of grade 7 to grade 8).

Based on earlier literature, it was hypothesised that (1) TOT mediates the role of Time 1 GPA as an indicator for prior ability (Gustafsson & Carlstedt, 2006) and Time 2 motivational attitudes for students’ success in a low-stakes assessment at Time 2 while working memory (VM) at Time 2 has an independent effect on it. It was also hypothesised that (2) cognitive competence at Time 2 mediates the role of Time 1 GPA while VM and students’ motivational attitudes have an independent effect on GPA at Time 3. It was also hypothesised that (3) there would be a gender difference in the relative weight of the different factors reflecting boys’ weaker school achievement, typical for Finnish basic education (c.f., Kupiainen, 2014, 2019).

The data is drawn from a four cohort municipal intervention study implemented in years 2017–2018 in two semi-urban municipalities in Southern Finland, covering all classes in the participating schools (Grade 5: 9 schools, approximately 280 students, 51 % boys, mean age at the beginning 12.3 years; Grade 7: 3 schools, approximately 250 students, 52 % boys, mean age at the beginning 12.3 years). The assessment was intended as a tool for measuring the possible effect of an intervention regarding teacher in-service education (results not reported here). All assessments were implemented in class following written rules in autumn 2017, spring 2018, and autumn 2018 (Timepoint 1, 2 and 3, respectively). The cognitive measurements comprised three tasks from the assessment’s wider battery: Mathematical reasoning (a task based on Demetriou et al. 1996), Inductive reasoning (a task adapted from Hosenfeld, van der Boom & Resing, 1997), and Verbal reasoning (a task from Ross & Ross, 1994) and a task on Visuo-spatial working memory. The two motivational constructs of learning-enhancing and detrimental-to-learning motivational attitudes co0mprised scales for Mastery-Intrinsic and Mastery-Extrinsic motivation and Agency-Effort, and Avoidance Orientation, Means-Ends: Chance and Self-Handicapping, respectively. In addition, log data for the three cognitive tasks as well as students’ self-reported grades (marks) in five subjects at the end of grade 4/6 and at the end of the study were used. Two structural equation models (SEM) were built for both age cohorts: In Model 1, students’ achievement in the low stakes cognitive test at Time 2 was predicted with Time GPA, the two sets of motivational constructs, VM, and TOT. In Model 2, Time 3 motivational attitudes and Time GPA were added to the model as indigenous and exogenous variables, respectively. The analyses were performed using IBM SPSS Satistics 24 and AMOS 24.

Despite an adequate root mean square error (RMSEA), Model 1 turned out to have poor fit (TLI and CFI <.900, RMSEA=.068.). Furthermore, students’ motivational attitudes played only a minor role in explaining their performance in the cognitive tasks – when TOT was accounted for. The final Model 2, however, had an adequate even if not especially good fit (TLI=.904, CFI=.923, RMSEA=.064). The model explained 72 percent of the variation in the cognitive test at Time 2 and 78 percent of the variance in the Time 3 GPA. Against expectation, TOT did not mediate the effect of Time 1 GPA on students’ performance in the cognitive test at Time 2 but both had an independent effect on it (β=.56 and β=.49, respectively, VM β=.28). Students’ negative motivational attitudes had no bearing on Time 3 GPA whereas positive motivational attitudes were a stronger predictor on the GPA than the cognitive test at Time 2 (β=.28 and β=.20, respectively, VM β=.14). Nevertheless, Time 1 GPA was clearly the strongest predictor of Time 3 GPA (β=.65) – even if Time 1 GPA regards grades given by the students’ classroom teachers while Time 3 GPA represents grades given by the students’ current subject teachers in a different school. In the presentation, we will compare the results of the younger age cohort to those presented here and look at the possible gender differences (Research question 3). The study sheds light on the impact of actual effort on achievement in low-stake assessment and learning situations. It also brings forth new knowledge of the relation of students’ stated motivational attitudes or beliefs, most often measured through self-report questionnaires, to their actual behaviour in test/learning situations. In this, the study will provide valuable knowledge for teachers but also for the interpretation of the results of international comparative assessments.

Adey, P., Csapó, B., Demetriou, A., Hautamäki, J., & Shayer, M. (2007). Can we be intelligent about intelligence? Why education needs the concept of plastic general ability. Educational Research Review, 2, 75–97. doi:10.1016/j.edurev.2007.05.001 Barry, C. L., Horst, S. J., Finney, S. J., Brown, A. R., & Kopp, J. P. (2010). Do examinees have similar test-taking effort? A high-stakes question for low-stakes testing. International Journal of Testing, 10, 342–363. doi: 10.1080/15305058.2010.508569 Carroll, J. B. (1963). A model of school learning. Teachers College Record, 64, 723–733. Carroll, J. B. (1989). The Carroll model: A 25-year retrospective and prospective view. Educational Researcher, 18, 26–31. doi:10.3102/0013189X018001026 Demetriou, A., Pachaury, A., Metallidou, Y., & Kazi, S. (1996). Universals and specificities in the structure and development of quantitative-relational thought: A cross-cultural study in Greece and India. International Journal of Behavioral Development, 19, 255–290. Demetriou, A., Spanoudis, G., & Mouyi, A. (2011). Educating the developing mind: Towards an overarching paradigm. Educational Psychology Review, 23, 601–663. doi:10.1007/s10648-011-9178-3 Goldhammer, F., Naumann, J., Stelter, A., Tóth, K., Rölke, H., & Klieme, E. (2014). The time on task effect in reading and problem solving is moderated by task difficulty and skill: Insights from a computer-based large-scale assessment. Journal of Educational Psychology, 106, 608–626. doi:10.1037/a0034716 Gustafsson, J.-E., & Carlstedt, B. (2006, August). Abilities and grades as predictors of achievement: The encapsulation theory. Paper presented at the 114th Annual Convention of the American Psychological Association, New Orleans, LA. Hattie, J. (2005). What is the nature of evidence that makes a difference to learning?. 2005-Using data to support learning, 7Meyer, H. D., & Benavot, A. (2013). PISA, power, and policy. In The emergence of global educational governance. Providence, RI: Symposium Books. Lee, Y.-H., & Chen, H. (2011). A review of recent response-time analyses in educational testing. Psychological Test and Assessment Modeling, 53, 359–379. Scheerens, J., & Bosker, R. J. (1997). The foundations of educational effectiveness. Oxford, United Kingdom: Pergamon Press. Spinath, B., Spinath, F. M., Harlaar, N., & Plomin, R. (2006). Predicting school achievement from general cognitive ability self-perceived ability and intrinsic value. Intelligence, 34, 363–374. doi:10.1016/j.intell.2005.11.004 Wise, S. L. (2006). An investigation of the differential effort received by items on a low-stakes computer-based test. Applied Measurement in Education, 19, 95–114. doi:10.1207/s15324818ame1902_2 Wise, S. L., & Kong, X. (2005). Response time effort: A new measure of examinee motivation in computer-based tests. Applied Measurement in Education, 18, 163–183.