The process data precedes a stage of generating scientific knowledge and making it available. The contribution introduces example considerations and solutions to make the data available in a format and with additional documentation that allows the generation of reliable, reproducible and valid scientific knowledge. The TIMSS 2019 Problem Solving and Inquiry (PSI) tasks were developed to gain insights into how using digitally-based interactive assessment items to capture students’ responses could be incorporated into TIMSS (Mullis et al., 2021). They aimed to collect information that could help enhance and extend the breadth of the TIMSS assessment providing more comprehensive coverage of problem solving and inquiry as already described in the study assessment frameworks. The eTIMSS PSI student achievement data files include a responder classification and percentage of items not reached allowing some further insights into students’ non-response, as well as students’ interactions with particular elements of PSI items (for example see Salles and Lacroix, 2024) and their use of the interface calculator for particular items, in addition to time per screen and frequency of visits. With the digital transition since then other IEA studies derive process variables from event (log) data and make them available in the International Database (IDB). Especially in the last cycle of ICILS, significantly more process data was collected and made available in the IDB. However, there is a need to review the structure and format of these derived variables, including the rationale behind data logging and the aggregation of events during task completion. The process data is not given. These are derived from log-files by data aggregation and often come with assumptions about the completion behavior or definitions made during the calculation of the process variables. This brings new requirements and demands regarding the documentation of these data as well as challenges with validity of their interpretation and use. Several analysis examples from ICILS will be explored to demonstrate how the variables can be used and discuss possible extensions in two directions: a) deriving more and other process data and b) utilizing the log-data and deriving own process indicators.

Mullis, I. V. S., Martin, M. O., Fishbein, B., Foy, P., & Moncaleano, S. (2021). Findings from the TIMSS 2019 Problem Solving and Inquiry Tasks. https://timssandpirls.bc.edu/timss2019/psi/ Salles, F., & Lacroix, A. (2024, June). Using Process Data in Large-Scale Assessments: An Example With an eTIMSS Problem Solving and Inquiry Task. IEA Compass: Briefs in Education No 24. IEA