In international large-scale assessments, students may not be compelled to answer every test item; hence, how these missing responses are treated may affect item calibration and ability estimation. Nevertheless, using a tree-based item response model or IRTree, it is possible to disentangle the probability of attempting to answer an item from the probability of a correct response (Debeer, Janssen, & De Boeck, 2017). In an IRTree, intermediate individual decisions are represented as intermediate nodes and observed responses as final nodes. Intermediate and end nodes are connected by branches that depict all possible outcomes of a cognitive subprocess. For each branch, it is possible to estimate a distinct probability (De Boeck, & Partchev, 2012). In the present study, we evaluate the usefulness of an IRTree model for skipped (omitted) responses, first with a simulation study and then on PIRLS data from 2006, 2011, and 2016. In the simulation study, we tested missing at random (MAR) and missing not at random (MNAR) scenarios. Moreover, we tested four missing response treatments, simulating the strategies of different large-scale assessments.The simulation study proved that the IRTree model maintained a higher accuracy than traditional imputation methods within a high proportion of omitted answers. In a second step, the IRTtree model for skipping responses was implemented for data of the last three cycles (2006, 2011, and 2016) of the Progress in International Reading Literacy Study (PIRLS). Correspondence between the official PIRLS results, a Rasch model, and the IRTree model was compared at three levels: items, students, and countries. We found some differences between the PIRLS and the Rasch model estimates at the item level; however, these do not significantly impact either the estimation of student ability or the country means and rankings. Moreover, the correlation between the scores estimated by the Rasch model and the IRTree model at the student level is high; however, it is not linear. In general, the results showed that while a change in the model may impact specific countries, it did not significantly impact the overall results or the country rankings. Nonetheless, when the information is disaggregated to compare a country's results over time, it is possible to observe how the increase or decrease in the proportion of skipped items can affect their overall results.

Debeer, D., Janssen, R., & De Boeck, P. (2017). Modeling Skipped and Not-Reached Items Using IRTrees. Journal of Educational Measurement, 54(3). 333-363. https://doi.org/10.1111/jedm.12147 De Boeck, P., & Partchev, I. (2012). IRTrees: Tree-Based Item Response Models of the GLMM Family. Journal of Statistical Software, 48, 1-28. https://doi.org/10.18637/jss.v048.c01