Assessment of Students’ Affective Outcomes in the Programme for International Student Assessment (PISA) 2015

Author(s):

Derek Cheung(presenting / submitting)

Conference:

ECER 2017

Network:

09. Assessment, Evaluation, Testing and Measurement

Format:

Paper

Session Information

09 SES 03 B, Investigating Affective Outcomes in the STEM-Field at Primary and Secondary School Level

Paper Session

Time:

2017-08-22

17:15-18:45

Room:

W5.18

Chair:

Rolf Vegar Olsen

Contribution

The results of the latest round of Programme for International Student Assessment (PISA) were released in December 2016 (OECD, 2016a). Most educators worldwide have focused on the science test results and the rankings of countries. Few educators have attempted to explore the relationships among variables in the affective domain. Science educators worldwide have made efforts to attract more students to study science at school (Jack & Lin, 2014; Logan & Skamp, 2013). Although interest is considered to be an important source of intrinsic motivation (Hidi, 2006) and often assessed in educational studies, few assessment researchers have investigated the influence of personal and classroom factors affecting students’ interest in school science. The present study was designed to address this gap in the assessment literature, using science education in Australia, Denmark, Sweden, Finland, and Hong Kong as the research context. In PISA 2015, five questionnaire items were constructed to assess 15-year-old students’ interest in five broad science topics: biosphere (e.g., ecosystem services, sustainability), motion and forces (e.g., velocity, friction), energy and its transformation (e.g., conservation, chemical reactions), the Universe and its history, and how science can help us prevent disease. The PISA affective framework (OECD, 2016b) also provided questionnaire items measuring students’ science self-efficacy, enjoyment of learning science, instrumental motivation to learn science, epistemological beliefs about science. The present study was guided by two research questions: (1) Are 15-year-old students interested in studying the five broad science topics? (2) What are the most important personal and classroom factors affecting students’ interest in studying the broad science topics? The findings have important implications for educational assessment of students in Europe.

Method

Data were from the PISA 2015 study conducted in Australia, Denmark, Sweden, Finland, and Hong Kong. Students were asked to rate the interest items on a five-point scale with the categories “not interested”, “hardly interested”, “interested”, “highly interested”, and “I don’t know what this is”. To answer the first research question, an index of student interest in broad science topics was computed. The last response category “I don’t know what this is” was recoded as a missing value for the purpose of computing the interest index. The OECD average of the interest index was equal to zero. To answer the second research question, multiple linear regression analyses were used to analyze the data. Students’ interest in the broad science topics was the criterion. A total of seven predictors of students’ interest were included: students’ science self-efficacy, enjoyment of learning science, instrumental motivation to learn science, epistemological beliefs about science, inquiry-based teaching and learning practices, environmental awareness, and students’ participation in science activities (e.g., watching TV programmes about science, visiting web sites about science topics). All these predictors were entered into the regression model simultaneously. The beta values and zero-order correlations were used to determine the relative importance of these seven predictors.

Expected Outcomes

The means of the interest index for Australia, Denmark, Finland, Sweden, and Hong Kong were 0.04, 0.18, -0.09, -0.02, and 0.25, respectively. This finding indicates that amongst these five participating countries and places, students in Finland showed the lowest level of interest in the five broad science topics. For example, only 27.2% of Finnish students reported that they are interested or highly interested in biosphere, whereas 63.7% of Hong Kong students are interested or highly interested in biosphere. Such a big difference should be a major concern to science educators in Europe. Multiple regression analyses revealed that the most important predictor of students’ interest in science topics was their extent of enjoyment of learning science, followed by their frequency of participation in science activities. These findings were consistent across Australia, Denmark, Finland, Sweden, and Hong Kong. In Denmark, for example, the linear combination of the seven predictors was significantly related to students’ interest in broad science topics, F(7, 5609) = 581.918, p < 0.001, R square = 0.421. The most powerful predictor was students’ enjoyment of studying science (beta = 0.40, zero-order correlation = 0.59, p < 0.001), followed by students’ participation in science activities (beta = 0.14, zero-order correlation = 0.41, p < 0.001). Surprisingly, the extent of implementation of inquiry-based teaching and learning practices in school was not a significant predictor of students’ interest in science topics (beta = 0.02, zero-order correlation = 0.18, p = 0.164). One possible explanation is that the assessment items in the PISA project measured students’ interest in studying the contents of five broad science topics rather than their individual interest in inquiry-based learning. The implications of these findings for designing valid and reliable ways to assess students’ individual and situational interests (Hidi & Renninger, 2006; Palmer, 2009) in school science are discussed.

References

Hidi, S. (2006). Interest: A unique motivational variable. Educational Research Review, 1, 69-82. Hidi, S. & Renninger, K. A. (2006). The four-phase model of interest development. Educational Psychologist, 41, 111-127. Jack, B. M. & Lin, H. S. (2014). Igniting and sustaining interest among students who have grown cold toward science. Science Education, 98, 792-814. Logan, M. R. & Skamp, K. R. (2013). The impact of teachers and their science teaching on students’ science interest: A four-year study. International Journal of Science Education, 35, 2879-2904. OECD (2016a). PISA 2015 results (volume I): Excellence and equity in education. Paris: OECD Publishing. OECD (2016b). PISA 2015 assessment and analytical framework: Science, reading, mathematic and financial literacy. Paris: OECD Publishing. Palmer, D. H. (2009). Student interest generated during an inquiry skills lesson. Journal of Research in Science Teaching, 46, 147-165.

Author Information

Derek Cheung (presenting / submitting)

The Chinese University of Hong Kong

Curriculum and Instruction

Shatin

Search the ECER Programme

Search for keywords and phrases in "Text Search"
Restrict in which part of the abstracts to search in "Where to search"
Search for authors and in the respective field.
For planning your conference attendance you may want to use the conference app, which will be issued some weeks before the conference
If you are a session chair, best look up your chairing duties in the conference system (Conftool) or the app.