Nowadays there is an international consensus regarding the importance of being a scientific literate citizen in order to understand and participate in an active and democratic way in a world where science, technology, and environment turn out to be global issues of the daily life. Science education community and other sectors of the society are well aware of the importance of scientific literacy. As Drori (2004) says “policy-makers world-wide regard economic grow as dependent on the scientific and technological capabilities of the labour force (…)” (p. 25). Consequently, to promote and assess scientific literacy had become, in many countries, part of the political agenda, turning out to be a national purpose. Naturally, the next step was to develop international comparative studies like TIMSS and PISA and also national ones which results of the assessment of scientific literacy became powerful tools to promote reflection and educational changes at all levels aiming to achieve better results. Different conceptual approaches of scientific literacy led to its assessment considering different aspects not always consensual. Although, there is a core of aspects commonly assessed: knowledge of science content; understanding science as a way of knowing, and understanding and conducting scientific inquiry” (Wilson and Berthanthal, 2005, p. 39). Nevertheless, according to Schreiner and Sjoberg (2004), “most of the exams in S&T put the emphasis on testing the grasp of the factual or cognitive concepts – the concepts, laws and theories” (p.20), “measuring recall resulting from school science knowledge” (Shwartz; Ben-Zvi and Hofstein, 2006, p. 204). Nevertheless, if factual knowledge is included as an assessment dimension, the results can be masked by the information spread by sources, like, for example, TV commercials or campaigns where pupils get information, but not necessarily knowledge. Factual knowledge is fundamental to scientific literacy but a scientific literate person is not the one that only can identify and enunciate scientific factual knowledge. An individual to be scientific literate must have the capability to use simultaneously higher order thinking skills in order to argue, formulate hypothesis, draw conclusions and, in a scientifically informed and critical way be able to take positions and decisions and, act accordingly. Therefore, we decided to clarify if effectively factual knowledge should be included when assessing scientific literacy. A research study was developed aiming to analyse the underlying structure of a scale intended to assess pupils’ scientific literacy based upon a conceptual model of three dimensions, including factual knowledge.

A stratified cross-national sample of 1156 pupils of the 9th grade (545 male and 611 female) with an average age of 14.7, pertaining to 60 classes from 21 different schools and from all seven NUTS II of the country (Nomenclature of Territorial Units for Statistics II). A pool of science units selected from the released ones of PISA 2000, 2003 and 2006 was constituted to assess pupils’ performance in scientific literacy. To get pupils’ full collaboration, the data were personally collected by one of the researchers during classes with the presence of one of the pupils’ teachers. The reliability of the scale was assessed using Cronbach alpha coefficient. To explore the underlying structure of the scale, an exploratory factor analysis (EFA), using Principal Components Analysis (PCA) to extract the factors and confirmatory factor analyses (CFA) using, respectively, the statistic software’s SPSS 16.0 and AMOS 16 were performed.

The Cronbach alpha value for the scale was 0.77, revealing internal consistency. PCA revealed three factors solution in line with the factors “factual knowledge”, “higher order thinking skills infused in scientific knowledge” and “attitudes”. The three-factor model identified in the EFA was investigated with the CFA using maximum likelihood estimation. Whilst the structure matrix in the EFA indicated that the components were distinguishable constructs, the structure matrix for this CFA model indicated that the items of the two first components were not distinct from each other. There is a large amount of overlap between them. The results enlighten that the “factual knowledge” and the “higher order thinking skills infused in scientific knowledge” are so overlapped that it does not make sense to assess them as distinct constructs. A two-factor model was tested. At the end of the process, the items concerning factual knowledge were removed and the scale improved.