The shift from paper-based to computer-based survey administration represents a major change in international comparative studies. Digital surveys offer several advantages over paper-based surveys, they are faster, less expensive, and can enhance data entry efficiency and consistency (Dillman, 2011). Despite these benefits, some research in educational contexts has found that survey participation tends to be lower on surveys administered digitally (e.g., Denscombe, 2006; Fan & Yan, 2010; Weigold et al., 2019). Missing data can distort results and reduce statistical power, presenting a critical challenge to the success of these studies. To address these challenges, this paper examines two questions: (1) How does survey mode affect response rates for TIMSS and PIRLS Grade 4 context questionnaires? (2) Does the survey mode effect vary by student characteristics? Context questionnaires, completed by parents, teachers, and principals, provide key data on students’ home and school learning environments. Historically, paper-based surveys in IEA studies achieved high response rates (e.g., average 90% participation on home questionnaires). However, countries making the transition to digital formats, have seen participation rates decline significantly (e.g., average 70% participation on home questionnaire). To explore survey mode and response trends, we use data from all previous PIRLS and TIMSS cycles and apply a difference-in-differences model to estimate the impact of administration mode on response and completion rates in background questionnaires. Since the mode of survey administration is not randomized—countries decide whether to administer questionnaires online or on paper—this analytical strategy allows us to control for potential selection effects. Separate models are estimated for home, teacher, and principal data, with interaction terms included to examine how the mode effect varies by student background. Preliminary model estimates reveal a significant negative effect of transitioning from paper-and-pencil to digital-based administration modes on survey response rates. These findings underscore the challenges and trade-offs involved in adopting digital methods. The study contributes to the literature on the effects of survey administration mode on questionnaire response, providing international evidence across a broad set of countries. The findings suggest that digital methods may enhance efficiency but risk reducing participation rates, potentially compromising data representativeness. Questionnaire responses from study participants will be analyzed to identify challenges and successes experienced in making the digital transition. Evidence from this study has practical implications for ILSA participants, showing the anticipated effects of switching administration modes on questionnaire response rates, while also highlighting effective strategies for collecting data under this new mode.

Denscombe, M. (2006). Web-based questionnaires and the mode effect: An evaluation based on completion rates and data contents of near-identical questionnaires delivered in different modes. Social Science Computer Review, 24(2), 246-254. Dillman, D. A. (2011). Mail and Internet surveys: The tailored design method--2007 Update with new Internet, visual, and mixed-mode guide. John Wiley & Sons. Fan, W., & Yan, Z. (2010). Factors affecting response rates of the web survey: A systematic review. Computers in human behavior, 26(2), 132-139. Weigold, A., Weigold, I. K., & Natera, S. N. (2019). Response rates for surveys completed with paper-and-pencil and computers: using meta-analysis to assess equivalence. Social Science Computer Review, 37(5), 649-668.