Learning to learn has been the centre of growing educational interest for more than a decade. It has been adopted in response to the challenges presented for education and educational assessment by the ever accelerating changes in the spheres of work and everyday life due to technological advancements, and is seen to form the core of lifelong learning (Hautamäki & al., 2002). Learning to learn has also been adopted as one of the eight key competences in the European reference framework for education, leading to Council recommendations for developing indicators for its measuring. Accordingly, under the auspices of European Commission, a new test was compiled and piloted among 14-year-old students in eight European countries in spring 2008. The main results of this study were reported to the Commission in January 2009 (Kupiainen, Hautamäki & Rantanen, 2008). Following the Education Council recommendations’ definition of learning to learn as a key competence for the ability to pursue and persist in learning, the piloted test was compiled to reflect both the social-cultural and the cognitive-psychological traditions, covering the cognitive, the affective and the metacognitive dimension (Hoskins & Fredriksson 2008). Measuring diverse affective factors on side of the ‘core’ cognitive component has become a standard in comparative research. They have a well-established place in both curriculum-based studies like TIMMS and PIRLS and the more curriculum-distanced PISA-survey (e.g., Mullis & al. 2005, OECD 2007). However, the concomitant nature of the cognitive and questionnaire data easily pre-empts fully exploring the apparent contradictions between these two sets of data as can be seen, for example, between the excellent performance of the Finnish students in PISA Science 2006 and their relatively low level of expressed interest and other learning-related attitudes (OECD 2007, Hautamäki & al. 2008, c.f. Turmo & Lie 2007). Building on the report on the eight country data, the focus of this paper will be on further exploring the challenges of comparative research in measuring a complex phenomenon such as learning to learn, comprising dimensions presenting different challenges when compared across countries. Also, the ineluctable use of school attainment (GPA) as a surrogate for the predicted future outcome of learning to learn, lifelong learning, causes its own problems of comparability across countries.

The test comprised paper-and-pencil tasks and questionnaires chosen from among British, Dutch, Finnish and Spanish tests in the fields of learning to learn or cross curricular competencies (Deakin & al. 2006, Meijer & al. 2001, Hautamäki & al. 2002, Moreno 2002). For the Commission report, the data have been analysed using a multitude of methods from simple descriptives and factor analyses to IRT and SEM modelling to reveal differences in the fit of the test in the eight countries. In the present paper, on side of these results, special attention will be given to analyses of correlational data across the participating countries.

The results of the pilot study revealed clear differences between the students of the participating countries both in their performance in the cognitive tasks and in their affective profiles. Furthermore, the differences were not restricted to the level of students’ cognitive performance and their learning related attitudes as implied by their answers to the attitudinal items but also to the relationships between these two and with students’ school attainment. In light of the role of students’ motivation, interest and other affective factors in school and in later achievements, the results raise questions concerning the use and interpretation of questionnaire data in comparative studies. Allowing the combining of data on cognitive tasks, attitudinal scales, students’ appraisals of their interest and attainment, and their GPA, the study allows to explore the relations between these as well as cultural differences in students’ use of the Likert-scale in questionnaires and while evaluating their performance.