International large-scale assessments (ILSA) administrate home questionnaires to parents or guardians to collect important information regarding students’ home context and early learning experience literacy that cannot be surveyed from students directly. Non-participation rates in these home surveys are often much higher than in student surveys and thus the representativeness of these surveys may be in jeopardy. Non-participation in these questionnaires may result in biased estimates. To make the home surveys more accessible, countries now have the option of administering them either via a paper-and-pencil or a computer-based format (e.g., DeLeeuw, 2018). In this study, we investigate two conditions related to parent’s non-participation. Firstly, we will investigate the non-participation bias accordingly to different levels of non-participation in home questionnaires in both TIMSS and PIRLS studies. Secondly, we investigate whether non-response rates can be attributed to an administration mode effect. To identify this mode effect, we take advantage of the international data from several TIMSS and PIRLS cycles and compare the participation rates on the home questionnaires before and after the switch from a paper-and-pencil format to an online survey system. Based on data from more than 70 countries, we examine the effect of changing the form of administration in fixed-effects analyses for countries and study cycles. In further analysis, we examine the interaction between respondents’ characteristics (e.g., education level) and mode effect on the participation rates. We find lower participation in almost all countries that switch to online surveys, with dramatic differences of up to about 20 percent. Furthermore, we find evidence that especially socially weak families participate less in the surveys.

DeLeeuw, E. D. (2018). Mixed-Mode: Past, Present, and Future. Survey Research Methods, 12(2), 75-89. https://doi.org/10.18148/srm/2018.v12i2.7402