This study examines the solution processes of Grade 4 students in TIMSS computer-based mathematics, focusing on problem-solving and inquiry-oriented (PSI) tasks administered only in a subset of test booklets (Mullis & Martin, 2017; Mullis, Martin, Fishbein, et al., 2021; von Davier et al., 2024). Computer-based large-scale assessments generate process (log/event) data that provide insights into how students engage with assessment tasks beyond final scores (Chen et al., 2024; Papanastasiou, 2023). We analyse a thematically coherent PSI task, Little Penguins, consisting of six interconnected items and an introductory motivational screen. The task was selected because it has been released and made available for research use and because it is complex, allowing students to solve different types of problems within a single context; most other TIMSS items do not offer such a context. However, our approach is also transferable to individual TIMSS items if needed.

The study adopts a combined macro–micro analytical perspective on students’ solution pathways. At the macro level, we analyse students’ pathways across the entire PSI task to capture navigation, action sequencing, and time allocation as they progress through the task set. This task-level analysis is complemented by a micro-level investigation of one focal item, using fine-grained event data to examine detailed interaction patterns during problem-solving. Students’ solution pathway profiles are modelled as data-driven types of behavioural sequences by combining process mining and sequence analysis of event streams with clustering of sequence variants and derived process indicators (e.g., screen visits, time on screens, action diversity and ordering, revisits and jumps, and timing metrics such as time-to-first-action and time-on-task).

The study addresses the following research questions:
(1) What solution pathway profiles can be identified in Grade 4 students’ interactions with the TIMSS computer-based PSI mathematics task Little Penguins, and which process indicators characterise these profiles?
(2) How are the identified solution pathway profiles associated with students’ performance outcomes and solution success?
(3) How does students’ digital self-efficacy predict membership in different solution pathway profiles?

Empirically, the analysis uses TIMSS Grade 4 computer-based mathematics data from nine European countries, purposefully selected to include the Czech Republic’s neighbouring countries and to cover a range of overall mathematics achievement relative to the Czech Republic (comparable as well as substantially higher performance). Cross-national analyses benchmark results against the Czech Republic and compare countries with one another. The study identifies educationally interpretable pathway profiles (e.g., efficient linear solvers, revisitors/checkers, and trial-and-error patterns) and examines whether profile–performance associations differ across outcome levels (focal item, whole-task success (0–10), and overall mathematics achievement). By integrating macro-level pathway analysis with micro-level event-based insights, the study advances methodological approaches to using process data from large-scale international assessments and highlights their potential to improve the interpretation and design of computer-based PSI tasks.

Data and sample. The analysis draws on TIMSS Grade 4 computer-based mathematics data from nine European countries (Czech Republic, Germany, Denmark, England, Finland, Lithuania, Poland, Slovakia, and Sweden; approximately 6,500 students). Because PSI tasks are administered only in a subset of TIMSS booklets, the analytic sample is restricted to students who completed Booklets 6 and 7, which include the released Little Penguins PSI task (Mullis, Martin, Fishbein, et al., 2021; Mullis, Martin, & von Davier, 2021; von Davier et al., 2024). Mode-related measurement issues are considered (Fishbein, 2018). Overall mathematics achievement is analysed using plausible values and international benchmark levels. Process and event data; feature engineering. We use timestamped event logs at the screen and task levels, covering navigation/transition events and task-specific interactions. Event streams are cleaned, time-ordered, and mapped to analytically meaningful states while preserving timing information. Derived indicators capture revisits and jumps, action diversity and sequencing, response changes, and timing metrics (e.g., time-to-first-action and time-on-task). Timing is treated as a core process dimension (Bulut & Gorgun, 2023). Sequence modelling and pathway profiling. We combine process mining (van der Aalst, 2016) with sequence analysis. Sequence representations (variants and distance-based measures) and derived indicators are used to derive solution pathway profiles via clustering; interpretability is supported by process maps and representative sequences (Gabadinho et al., 2011). Outcomes and contextual predictors. Performance is analysed at three outcome levels: focal item correctness (0/1), whole-task success (0–10), and overall mathematics achievement (plausible values and benchmark categories). As a key predictor of profile membership, we use a three-level digital self-efficacy index constructed from multiple student questionnaire items (ordinal). Instructional context is incorporated through teacher-reported access to digital devices during mathematics lessons (yes/no) and frequency of device use. Analyses. Associations between pathway profiles and outcomes are examined using appropriate models (e.g., logistic for item correctness; linear/ordinal/count for whole-task-level scores; linear models for plausible values; multinomial/ordinal models for benchmark categories), incorporating TIMSS sampling weights and plausible-value combination rules where applicable.

Addressing RQ1–RQ3, we identified a small set of educationally interpretable solution pathway profiles in the Little Penguins PSI task across countries, including efficient linear solvers, revisitors/checkers, and trial-and-error patterns characterised by high response instability and dense action bursts. Profile membership was associated with performance: some profiles were more closely related to focal-item correctness (0/1), whereas others were more informative for whole-task success (0–10) and overall mathematics proficiency (plausible values and benchmark levels). Students’ digital self-efficacy was also related to pathway profile membership, suggesting links between perceived ease with digital technologies and more structured navigation in computer-based PSI tasks (Siiman et al., 2017). By triangulating outcomes at the item, task, and overall achievement levels, the study helps clarify whether observed process behaviours reflect item-specific tactics, task-level strategies, or broader proficiency-related patterns. Cross-national comparisons show that the prevalence of pathway profiles varies across countries and aligns with classroom digital learning conditions, based on teacher reports of device access and frequency of use in mathematics instruction. Methodologically, we provide a cross-nationally replicable workflow for transforming event logs into sequence representations, deriving pathway profiles, and linking them to multiple achievement measures while respecting TIMSS design features (weights and plausible values). Combining process mining and sequence analysis with multi-level outcome modelling, the study advances process-data analytics as a form of “knowing” that can inform “acting”: interpretation and design of computer-based PSI tasks and targeted support for early mathematics problem solving (van der Aalst, 2016; Gabadinho et al., 2011).

Bulut, O., & Gorgun, G. (2023). Utilizing response time for scoring the TIMSS 2019 problem solving and inquiry tasks (Preprint). PsyArXiv. https://doi.org/10.31234/osf.io/zc98s Chen, G., Liu, Y., & Mao, Y. (2024). Understanding the log file data from educational and psychological computer-based testing: A scoping review protocol. PLOS ONE, 19(5), e0304109. https://doi.org/10.1371/journal.pone.0304109 Fishbein, B. G. (2018). Preserving 20 years of TIMSS trend measurements: Early stages in the transition to the eTIMSS assessment (Doctoral dissertation, Boston College). eScholarship@BC. http://hdl.handle.net/2345/bc-ir:107927 Gabadinho, A., Ritschard, G., Studer, M., & Müller, N. S. (2011). Mining sequence data in R with the TraMineR package: A user’s guide. Journal of Statistical Software, 40(4), 1–37. Mullis, I. V. S., & Martin, M. O. (Eds.). (2017). TIMSS 2019 assessment frameworks. International Association for the Evaluation of Educational Achievement (IEA). Mullis, I. V. S., Martin, M. O., Fishbein, B., Foy, P., & Moncaleano, S. (2021). Findings from the TIMSS 2019 problem solving and inquiry tasks. International Association for the Evaluation of Educational Achievement (IEA). Mullis, I. V. S., Martin, M. O., & von Davier, M. (Eds.). (2021). TIMSS 2023 assessment frameworks. TIMSS & PIRLS International Study Center, Boston College. https://timssandpirls.bc.edu/timss2023 Papanastasiou, E. K. (2023). What can process data tell us about students’ persistence? Evidence from the eTIMSS 2019 assessment. In Assessment reform journeys: Intentions, enactment and evaluation (Book of Abstracts) (p. 23). AEA Europe. Siiman, L. A., Mäeots, M., & Pedaste, M. (2017). A review of interactive computer-based tasks in large-scale studies: Can they guide the development of an instrument to assess students’ digital competence? In D. Joosten-ten Brinke & M. Laanpere (Eds.), Technology enhanced assessment (TEA 2016) (pp. 148–158). Springer. https://doi.org/10.1007/978-3-319-57744-9_13 van der Aalst, W. (2016). Process mining: Data science in action (2nd ed.). Springer. von Davier, M., Fishbein, B., & Kennedy, A. (Eds.). (2024). TIMSS 2023 technical report: Methods and procedures. TIMSS & PIRLS International Study Center, Boston College. https://timss2023.org/methods