There is a need for longitudinal studies and generalizable causal inference in the field of Educational Effectiveness Research (EER) (Muijs, 2012). In line with this, the present study argues that exploiting the trend-design of large-scale surveys like TIMSS and conducting analyses at the country level, provides improved support for causal interpretations (Gustafsson, 2013). There is evidence that students’ educational outcomes and changes in these outcomes are influenced by school climate (e.g. Creemers &, Kyrikides, 2010; Kyriakides et al., 2010). An important aspect of school climate is School Emphasis on Academic Success (SEAS) (Hoy et al., 2006; Martin et al., 2013) which has been found to have a positive effect on students’ achievement (Martin et al., 2013). SEAS reflects a clear priority for academic success and includes teachers’ beliefs in their own capabilities, schools’ trust in parents and students, and teachers’ expectations for students’ success (Hoy et al., 2006). Utilizing data from TIMSS 2007 and 2011, the aim of this paper is to investigate the impact of changes in country-level SEAS on changes in country-level mathematics achievement from 2007 to 2011. We included all countries (N=38, grade 8) who participated in TIMSS 2007 and 2011. SEAS was measured by teachers’ ratings and students’ educational outcomes were measured by the plausible values for mathematics achievement. Using a Difference in Differences analysis, we aggregated the data to the country level and investigated the country-level change in mathematics achievement regressed upon country-level change in mean SEAS. Our main finding is that changes in country-level SEAS have a significant and positive impact on changes in country-level achievement in mathematics. We discuss how trend-analysis controls for time-invariant omitted variables and we also discuss possible remaining threats to valid causal inference.

Creemers, B., & Kyriakides, L. (2010). School factors explaining achievement on cognitive and affective outcomes: Establishing a dynamic model of educational effectiveness. Scandinavian Journal of Educational Research, 54, 263–294 Gustafsson, J. E. (2013). Causal inference in educational effectiveness research: a comparison of three methods to investigate effects of homework on student achievement. School Effectiveness and School Improvement, 24(3), 275-295. Hoy, W. K., Tarter, C. J., & Woolfolk Hoy, A. (2006). Academic optimism of schools. American Educational Research Journal, 43, 425–446 Martin, M. O., Foy, P., Mullis, I. V. S., & O'Dwyer, L. M. (2013). Effective schools in reading, mathematics, and science at fourth grade. In M. O. Martin, & I. V. S. Mullis (Eds.), TIMSS and PIRLS 2011. Chestnut Hill, MA: TIMSS & PIRLS International Study Center, Boston College. Kyriakides, L., Creemers, B., Antoniou, P., & Demetriou, D. (2010). A synthesis of studies searching for school factors: Implications for theory and research. British Educational Research Journal, 36(5), 807-830. Muijs, D. (2012). Methodological change in educational effectiveness research. In C. P. Chapman, Armstrong, P., Harris, A., Muijs, D. R., Reynolds, D. ,Sammons, P. (Ed.), School effectiveness and improvement research, policy and practice: challenging the orthodoxy. (pp. 58-66). Abingdon: Routledge.