A central aim of compulsory education is to foster learning for all students, independent of their family backgrounds (cf. Marks, 2005). If an education system did indeed succeed in fulfilling the goal of providing the same learning opportunities at all schools, then school effectiveness measures should not vary between schools. This notion of equality of school effectiveness implies that students should make the same learning progress, independent of their school’s socioeconomic composition, for instance. We consider equality of school effectiveness to be an important facet of educational effectiveness because it depicts whether some students are left behind at their schools (e.g., Chapman et al., 2012).

This study proposes a regression discontinuity-based measure for the equality of school effectiveness. In the present case, the simple regression discontinuity approach implies that differences in the birth date (running variable age) decide whether a student is placed in a lower grade or upper grade (treatment added year of schooling) due to school entry regulations that have fixed birth-date-related thresholds (scoring rule). The birth date is assumed to be independent of confounders such as socioeconomic status. The outcome (achievement scores) is regressed on both the binary treatment (added year of schooling) and the running variable (age) and therefore allows to disentangle schooling from aging effects. This approach can be applied if enough students comply with the scoring rule (Schochet et al., 2010), i.e., attend the formally correct grade in accordance with their birth date and school entry regulations. This is the case in education systems that have strict school entry regulations as well as rare grade retentions and accelerations.

Numerous previous studies have applied the simple regression discontinuity approach to disentangle the effects of age from the effects of schooling on school achievement outcomes (e.g., Ali & Heck, 2012; Cahan et al., 2008; Cliffordson, 2010; Crone & Whitehurst, 1999; Gormley & Gayer, 2005; Heck & Moriyama, 2010; Kyriakides & Luyten, 2009; Luyten, 2006; Luyten et al., 2008, 2009; Perry, 2017; Webbink & Gerritsen, 2013). In summary, they show that both age and schooling usually have positive effects on achievement. However, the effect sizes vary between countries as well as younger and older students.

In this study, we translate the simple regression discontinuity model to a multi-level framework to separate the between-school differences in school effectiveness from other between-school differences. The two-level model includes a random intercept, a random slope for the added-year effect, and a fixed slope for the age effect. Hence, the model captures differences in intercepts and added-year effects between schools, while the age effect is the same across schools. Between-school variation in intercepts reflects that schools differ in the achievement levels in the lower grades. Such differences can root in different socioeconomic student compositions, for instance. Between-school variation in the added-year effects reflects that schools vary in their school effectiveness. Although it is probably difficult to reduce the between-school variation in intercepts, countries can aim to minimize the between-school variation in school effectiveness.

The variation in the added-year effects between schools, which serves as the central measure of interest in this study, has only been investigated in six previous studies (Ali & Heck, 2012; Heck & Moriyama, 2010; Kyriakides & Luyten, 2009; Luyten, 2006; Luyten et al., 2008, 2009). These usually found significant between-school variation in intercepts but not always in added-year slopes. This study aims to build on this prior research to disentangle the between-school variation in school effectiveness from other sources of between-school variation with the two-level regression discontinuity approach.

We selected 13 samples from TIMSS 1995 and 2015 (Third/Trends in International Mathematics and Science Study) in which two adjacent grades from the same schools were assessed and at least 93% of the students attended the formally correct grade given their age. These encompassed six primary (Norway 2015, Norway 1995, Iceland 1995, Cyprus 1995, Greece 1995, Scotland 1995) and seven secondary school samples (Norway 2015, Norway 1995, Iceland 1995, Sweden 1995, Cyprus 1995, Scotland 1995, Slovak Republic 1995). Within these samples, we excluded schools with less than 10 students in either the lower or upper grade. The outcome variables were student achievement scores in mathematics and science. The predictor variables were the treatment (0 = lower grade, 1 = upper grade) and running variable age in months. Age in months was recoded relative to the threshold age between grades (0 = oldest students in lower grade, 1 = youngest student in upper grade) so that the age variable ranges between -11 (youngest students in lower grade) and 12 (oldest students in upper grade). We ran two types of regression discontinuity models in Mplus. First, we conducted student-level analyses to illustrate general age and added-year effects on mathematics and science achievement and to ensure that regression discontinuity analyses prerequisites were met. Second, we ran school-level analyses to isolate the between-school differences in school effectiveness (i.e., added-year slopes) from others (i.e., intercepts). We ran separate analyses for all 13 samples and for both outcome domains. We applied sampling weights to account for the stratified complex sampling designs. The analyses were run separately for the five plausible values and the results were combined using Rubin’s rules. In addition, we accounted for the nesting of students in schools in all analyses.

In the simple regression discontinuity models, we found positive effects of both age and schooling on mathematics and science achievement. These differed however vastly between the 13 samples. Method-related requirements (cf. Schochet et al., 2010) were met. In the two-level regression discontinuity models, we found significant between-school variation in the intercepts but not in the added-year effects in all samples. We observed low degrees of between-school variation in added-year slopes in primary school samples, especially in Norway, Iceland, and Cyprus 1995. At the secondary school level, we observed low degrees of between-school variation in added-year slopes in Norway 1995 and 2015 as well as Cyprus 1995 samples. More pronounced between-school variations were found in the secondary school samples in Sweden, Scotland, and the Slovak Republic 1995. Although interpretations must consider methodological issues such as varying sample sizes, some countries seemed to attain a high degree of equality of school effectiveness. The current paper demonstrates how the RD-based equality of school effectiveness measure can be compared between countries and over a 20-year interval. To complement previous research, central strengths of this study lie in the fact that we ran simpler and more targeted models to identify the effects of interest and that we complemented reanalyzed data from previous studies with new data. We discuss advantages of the RD-based measure for the equality of school effectiveness against measures that require longitudinal data. We conclude that in countries with strict school entry policies and low rates of grade repetition and acceleration, such as the Nordic countries, the regression discontinuity approach is a promising way for obtaining robust estimations of the equality of school effectiveness. We therefore recommend that countries extend their participation in cross-sectional assessments by including at least two adjacent grades from the same schools.

Ali, E., & Heck, R. H. (2012). Comparing the contexts of middle-grade schools, their instructional practices, and their outcomes: A regression discontinuity approach. NASSP Bulletin, 96(2), 93–118. Cahan, S., Greenbaum, C., Artman, L., Deluya, N., & Gappel-Gilon, Y. (2008). The differential effects of age and first grade schooling on the development of infralogical and logico-mathematical concrete operations. Cognitive Development, 23(2), 258–277. Chapman, C., Armstrong, P., Harris, A., Muijs, D., Reynolds, D., & Sammons, P. (2012). School effectiveness and improvement research, policy, and practice. Routledge. Cliffordson, C. (2010). Methodological issues in investigations of the relative effects of schooling and age on school performance: The between-grade regression discontinuity design applied to Swedish TIMSS 1995 data. Educational Research and Evaluation, 16(1), 39–52. Crone, D. A., & Whitehurst, G. J. (1999). Age and schooling effects on emergent literacy and early reading skills. Journal of Educational Psychology, 91(4), 604–614. Gormley, W. T., & Gayer, T. (2005). Promoting school readiness in Oklahoma. Journal of Human Resources, XL(3), 533–558. Heck, R. H., & Moriyama, K. (2010). Examining relationships among elementary schools’ contexts, leadership, instructional practices, and added-year outcomes. School Effectiveness and School Improvement, 21(4), 377–408. Kyriakides, L., & Luyten, H. (2009). The contribution of schooling to the cognitive development of secondary education students in Cyprus: An application of regression discontinuity with multiple cut-off points. School Effectiveness and School Improvement, 20(2), 167–186. Luyten, H. (2006). An empirical assessment of the absolute effect of schooling: Regression‐discontinuity applied to TIMSS‐95. Oxford Review of Education, 32(3), 397–429. Luyten, H., Peschar, J., & Coe, R. (2008). Effects of schooling on reading performance, reading engagement, and reading activities of 15-year-olds in England. American Educational Research Journal, 45(2), 319–342. Luyten, H., Tymms, P., & Jones, P. (2009). Assessing school effects without controlling for prior achievement? School Effectiveness and School Improvement, 20(2), 145–165. Marks, G. N. (2005). Cross-national differences and accounting for social class inequalities in education. International Sociology, 20(4), 483–505. Perry, T. (2017). Inter-method reliability of school effectiveness measures: A comparison of value-added and regression discontinuity estimates. School Effectiveness and School Improvement, 28(1), 22–38. Schochet, P., Cook, T., Deke, J., Imbens, G., Lockwood, J. R., Porter, J., & Smith, J. (2010). Standards for regression discontinuity designs. What Works Clearinghouse. Webbink, H. D., & Gerritsen, S. (2013). How much do children learn in school? Netherlands Bureau for Economic Policy Analysis.