One of the major goals in science education is to help students become scientifically literate. Although not all students will ultimately pursue a career in science, the thinking skills they use when engaged in inquiry activities are relevant to their everyday life decision-making (Kuhn, 2007). In order to successfully participate in the process of inquiry, students are expected to demonstrate fundamental scientific thinking skills (Kuhn & Pease, 2008).

Kuhn defined scientific thinking as purposefully seeking knowledge that drives the conceptual change process and lead to the goal of constructing scientific understanding (2007). Scientific thinking encompasses the abilities needed for the application of scientific inquiry, such as designing and evaluating scientific investigations, evaluating evidence, and making causal inferences for forming and modifying theories related to  the phenomenon under investigation (Zimmerman, 2007).

Scientific thinking skills comprise a number of key subskills; yet, existing research has mostly focused on a single subskill rather than taking an integrated and much broader perspective. The present study therefore takes an integrated approach to investigating scientific thinking. Specifically, it studies the skills that students apply when they solve problems that require the coordination of the effects of multiple variables and the coordination of theory and evidence. An analysis of the PISA 2015 science data will provide insight into both the quality and the strategies of students’ scientific thinking. An in-depth case study will also be used to further explore the difficulties students encounter in performing scientific thinking. The research questions that guide this study are:

(1)   What do the 2015 PISA science data tell us about the level of students’ ability to coordinate the effects of multiple variables?

(2)   Which strategies do high and low achievers use to coordinate the effects of multiple variables?

(3)   What is the quality of the students’ reasoning as they solve problems that require the coordination of theory and evidence?

(4)   What are the challenges that students face in coordinating the effects of multiple variables and in coordinating theory and evidence?

The ability to predict outcomes based on the simultaneous effects of multiple variables is essential for producing experiments that yield interpretable evidence and for facilitating inferential skills (Zimmerman, 2007).  One approach to examining this ability is to identify the strategies students use for creating desired outcomes. Research on scientific thinking has heavily focused on a specific strategy called control-of-variables strategy (CVS). This strategy creates an experimental situation where the effects of independent variables on dependent variables can be disentangled.  There is a growing body of research analysing the CVS on a small-scale basis (review in Schwichow, Croker, Zimmerman, Höffler, & Härtig, 2015). A standardized assessment such as PISA, however, offers an opportunity to study students’ strategic approaches further on a large-scale basis with representative samples and the potential of generalizability.

Coordinating theory and evidence is another important inquiry skill. Kuhn and Pearsall (2000) claim that it is the essence of advanced scientific thinking because it requires an openness to revise or completely reformulate one’s initial theory or belief in response to a pattern of evidence. Interpreting evidence from first-hand data is crucial in assessing participants’ scientific reasoning (Zimmerman, 2007). Delen and Krajcik (2015) suggested the importance of collecting data through first-hand investigation in creating a quality of reasoning explanation. Research also shows that there is no significant difference in participants’ reasoning quality when they interpreted data from physical or simulated systems (Klahr, Triona, & Williams, 2007). Due to the possibilities to incorporate simulated hands-on environments into computer-based science tests such as the one administered in PISA, novel opportunities exist to assess a range of reasoning skills rather than a single skill.

Delen, I., & Krajcik, J. (2015). What do students’ explanations look like when they use second-hand data? International Journal of Science Education, 37(12), 1953-1973. Klahr, D., Triona, L. M., & Williams, C. (2007). Hands on what? The relative effectiveness of physical versus virtual materials in an engineering design project by middle school children. Journal of Research in Science Teaching, 44(1), 183-203. Kuhn, D. (2007). What Is Scientific Thinking and How Does It Develop? Blackwell Handbook of Childhood Cognitive Development (pp. 371-393): Blackwell Publishers Ltd. Kuhn, D., & Pearsall, S. (2000). Developmental origins of scientific thinking. Journal of Cognition and Development, 1(1), 113-129. Kuhn, D., & Pease, M. (2008). What needs to develop in the development of inquiry skills? Cognition and instruction, 26(4), 512-559. OECD. (2013). PISA 2015 Released Field Trial Cognitive Items. Paris: OECD. Schwichow, M., Croker, S., Zimmerman, C., Höffler, T., & Härtig, H. (2016). Teaching the control-of-variables strategy: A meta-analysis. Developmental Review, 39, 37-63. Zimmerman, C. (2007). The development of scientific thinking skills in elementary and middle school. Developmental Review, 27(2), 172-223.