The primary purpose of educational assessment is to support learning by identifying students’ strengths and weaknesses rather than merely reporting a single performance score. Nevertheless, summative assessment practices—typically based on Classical Test Theory or Item Response Theory (IRT)—remain dominant in many educational systems (Hambleton et al., 1991). Although linear tests might provide reliable overall ability estimates, they offer limited diagnostic information about learners’ mastery of specific skills (Choi, 2010; de la Torre & Karelitz, 2009). Cognitive Diagnostic Models (CDMs) have been developed to address this limitation by classifying examinees according to mastery or non-mastery of predefined skills or attributes (de la Torre & Douglas, 2004). CDMs enable the provision of structured, fine-grained feedback that can support formative assessment and instructional decision-making, particularly in primary education where early identification of learning gaps is crucial (Huebner, 2010).

In parallel, advances in digital testing have led to increasing use of adaptive testing approaches such as Computerized Adaptive Testing (CAT) and Multistage Testing (MST). Adaptive tests can improve measurement precision while reducing test length (Embretson, 1996; Weiss, 2004). Among these approaches, MST offers practical advantages for educational settings, including content balancing, quality control, and examinee navigation flexibility (Sari et al., 2016; Yan et al., 2014). However, MST implementations are typically based on IRT and therefore still yield a single overall score. To enhance the formative potential of adaptive assessment, recent research has focused on integrating CDMs with adaptive testing. Cognitive Diagnostic Computerized Adaptive Testing (CD-CAT) and Cognitive Diagnostic Multistage Testing (CD-MST) represent promising approaches for delivering mastery-based feedback within adaptive frameworks (Cheng, 2009; de la Torre, 2019; Kim & Yoo, 2023; von Davier & Cheng, 2014). Despite this promise, empirical research on CD-MST remains limited and has largely relied on simulation studies (Kim & Yoo, 2023; Li et al., 2021; Liu et al., 2018). Evidence from live, large-scale implementations—especially with young learners—is scarce.

Beyond methodological innovation, the use of a live CD-MST is particularly important for classroom assessment because it bridges the gap between psychometric sophistication and instructional usability. Unlike simulation-based studies, live tests capture authentic student behaviors, contextual constraints, and instructional realities that directly affect the validity and utility of diagnostic feedback. Collecting live data therefore allows for evaluation of whether CD-MST results are interpretable, stable, and actionable for teachers and learners in real educational settings. Moreover, empirical evidence from classroom use is essential to demonstrate CD-MST’s practical feasibility for formative assessment and inform educators and policymakers about its potential to support data-driven instruction and early intervention—particularly in primary education, where timely and accurate diagnostic information can have long-lasting effects on students’ learning trajectories.

Thus, the present study addresses this gap by developing and implementing a live CD-MST for fourth-grade mathematics in Türkiye and Spain. The test is designed to provide diagnostic feedback on core arithmetic skills (addition, subtraction, multiplication, division, routine and nonroutine problem solving) while ensuring cross-cultural comparability. By administering the test digitally in real school settings, the study aim to contribute empirical evidence on the feasibility, validity, and formative value of CD-MST in European educational contexts. The guiding research question is: To what extent do Turkish and Spanish fourth-grade students demonstrate mastery of mathematics skills as assessed by a live Cognitive Diagnostic Multistage Test?

Participants The study will be conducted in two phases. During the pilot phase, data will be collected from approximately 800 fourth-grade students (400 in Türkiye and 400 in Spain). In the live CD-MST phase, an additional 200 students (100 per country) will participate, resulting in a total sample of 1000 students. Due to school-level permission requirements, convenience sampling will be employed. In Türkiye, participating schools will be recruited from the Istanbul region, and in Spain from the Madrid region. Efforts will be made to include schools from diverse socioeconomic backgrounds and both public and private sectors to ensure variability in student ability levels. Students will be aged 9–10 years, with an approximately balanced gender distribution. All testing will take place digitally (computers or tablets) within school environments, typically in computer laboratories. Each session will last approximately 50 minutes. Participation will be voluntary, with no incentives provided. Ethical approvals and relevant institutional permissions were obtained prior to data collection. Instrument A common table of specifications aligned with both Turkish and Spanish mathematics curricula were developed. From an existing pool of 540 fourth-grade mathematics items aligned with TIMSS cognitive domains, 60 items were selected based on curricular relevance, difficulty range, and suitability for CD-MST. Item adaptation from Turkish to Spanish followed a multistage translation procedure, in which the original items were first translated into English and then translated into Spanish by experts to ensure linguistic and conceptual equivalence. Validity evidence will be gathered using Q-matrix validation procedures. The pilot phase for estimating item parameters will employ a multiple-matrix sampling design with four linked booklets, each containing 20 items, to reduce students’ testing burden. Additionally, the differential item functioning (DIF) analyses will be conducted and biased items will be excluded before final test assembly. Development of CD-MST The final live CD-MST will employ a 1–3–3 multistage design, with approximately 12–15 items administered per student. Both the pilot and final live assessments will be implemented using the Concerto platform. Module selection will be guided by indices based on the Kullback–Leibler distance or the generalized DINA model discrimination index. Data Analysis After development, a sample of students (100 Turkish and 100 Spanish) will take the CD-MST to assess its functionality. Students’ mastery profiles will be estimated using DINA or G-DINA models (de la Torre, 2011). Attribute-level probabilities will be reported to address the research question.

The study is expected to demonstrate the feasibility of implementing a live CD-MST with primary school students in authentic classroom settings. The results are anticipated to show that diagnostic information can be obtained with a substantially reduced testing burden compared to traditional fixed-form assessments. Previous simulation studies have indicated that CD-CAT achieves classification accuracy comparable to that of paper-and-pencil CDM tests (de la Torre, 2019); therefore, similar levels of accuracy with CD-MST across cultures are expected. From a European perspective, findings are expected to provide empirical evidence on the cross-cultural applicability of diagnostic adaptive assessments. Mastery profiles are expected to reveal meaningful patterns in students’ strengths and weaknesses, highlighting the potential of CD-MST to support formative assessment and early intervention across educational systems. Methodologically, the study will contribute to the limited body of empirical research on live CD-MST implementations and inform future European initiatives in digital assessment design. Practically, the findings may support the broader adoption of mastery-based adaptive testing as a tool for promoting equity by providing all learners with access to individualized, data-driven feedback regardless of resource constraints.

Cheng, Y. (2009). When cognitive diagnosis meets computerized adaptive testing: CD-CAT. Psychometrika, 74, 619-632. Choi, H. J. (2010). A model that combines diagnostic classification assessment with mixture item response theory models (Doctoral dissertation). University of Georgia, Athens de la Torre, J. (2019, June). Recent advances in cognitive diagnosis computerized adaptive testing. Paper presented at the Seventh International Meeting of the International Association for Computerized Adaptive Testing, Minneapolis. de La Torre, J., & Douglas, J.A. (2004). Higher-order latent trait models for cognitive diagnosis. Psychometrika 69, 333–353 de La Torre, J., & Karelitz, T. M. (2009). Impact of diagnosticity on the adequacy of models for cognitive diagnosis under a linear attribute structure: A simulation study. Journal of Educational Measurement, 46(4), 450-469 Embretson, S. E. (1996). The new rules of measurement. Psychological Assessment, 8(4), 341-349 Hambleton, R. K., Swaminathan, H., & Rogers, H. J. (1991). Fundamentals of item response theory (Vol. 2). Sage. Huebner, A. (2010). An overview of recent developments in cognitive diagnostic computer adaptive assessments. Practical Assessment, Research, and Evaluation, 15(3), 1-7. Kim, R. Y., & Yoo, Y. J. (2023). Cognitive diagnostic multistage testing by partitioning hierarchically structured attributes. Journal of Educational Measurement, 60(1), 126-147. Li, G., Cai, Y., Gao, X., Wang, D., & Tu, D. (2021). Automated test assembly for multistage testing with cognitive diagnosis. Frontiers in Psychology, 12, 509844. Liu, S., Cai, Y., & Tu, D. (2018). On-the-Fly Constraint-Controlled Assembly Methods for Multistage Adaptive Testing for Cognitive Diagnosis. Journal of Educational Measurement, 55(4), 595-613. Sari, H. İ., Yahsi-Sari, H., & Huggins-Manley, A. C. (2016). Computer adaptive multistage testing: Practical issues, challenges and principles. Journal of Measurement and Evaluation in Education and Psychology, 7(2), 388-406. Weiss, D. J. (2004). Computerized Adaptive Testing for Effective and Efficient Measurement in Counseling and Education. Measurement and Evaluation in Counseling and Development, 37(2), 70–84. von Davier, M. & Cheng, Y. (2014). Multistage testing using diagnostic models. In D. L. Yan, A. A. von Davier, & C. Lewis (Eds.), Computerized multistage testing: Theory and practice (pp. 219–227). Boca Raton, FL: Chapman & Hall/CRC. Yan, D., Lewis, C., & von Davier, A. A. (2014). Overview of computerized multistage tests. In D. Yan, C. Lewis, & A. A. von Davier (Eds.), Computerized multistage testing: Theory and applications (pp. 3–20). New York: Chapman and Hall/CRC.