Session Information
09 SES 10 B, Innovations, Challenges, and Insights from International Large-Scale Assessments (Part 5): Examining ILSAs: Measurement and Sampling
Symposium
Contribution
This study examines the relationship between reading demand and performance in science items in the Programme for International Student Assessment (PISA) in Finland, focusing on changes between 2012 and 2022. Reading demand was defined as the combined number of words, scientific terms, symbols, and visual elements in each item, following Mullis, Martin, and Foy (2013). The analysis included 38 science items common to both assessment cycles. In addition to reading demand, item characteristics such as content area, response format, competence, and difficulty level were considered (OECD, 2017). Results show that Finnish students’ science performance declined significantly during the period, with the largest decreases observed in items with medium and high reading demand. Of the 38 tasks analyzed, 30 showed a statistically significant decrease in solution rates. While reading demand was associated with overall item solution rates, items with higher reading demand tended to have lower solution rates. Changes in performance were not clearly linked to reading demand alone. Among the examined other item characteristics, response format emerged as the most influential factor: open-ended tasks and complex multiple-choice items exhibited the largest declines, whereas simple multiple-choice items showed smaller changes. Performance differences were also examined by reading proficiency and immigrant background. Students with weaker reading skills experienced steeper declines in solution rates compared to those with stronger skills, although changes occurred across all reading demand categories. Immigrant students performed worse overall than their native peers, but their performance changes were smaller, likely because their initial solution rates were lower. The findings highlight the importance of reading literacy in science education and indicate that item characteristics, including reading demand, have a complex relationship with performance trends. However, the reading demand measure used here does not fully capture the unique linguistic features of Finnish, which may affect the actual cognitive demands of the items. These results have implications for assessment design and curriculum development, particularly in supporting students with lower reading proficiency and diverse linguistic backgrounds.
References
OECD. (2017). PISA 2015 Assessment and Analytical Framework. OECD Publishing, Paris. Mullis, I. V. S., Martin, M. O., & Foy, P. (2013). The impact of reading ability on TIMSS mathematics and science achievement at the fourth grade. Boston College.
Update Modus of this Database
The current conference programme can be browsed in the conference management system (conftool) and, closer to the conference, in the conference app.
This database will be updated with the conference data after ECER.
Search the ECER Programme
- Search for keywords and phrases in "Text Search"
- Restrict in which part of the abstracts to search in "Where to search"
- Search for authors and in the respective field.
- For planning your conference attendance, please use the conference app, which will be issued some weeks before the conference and the conference agenda provided in conftool.
- If you are a session chair, best look up your chairing duties in the conference system (Conftool) or the app.