International comparative studies in different school subjects showed that countries differ greatly in both level and equity of educational achievement (e.g., Mullis, Martin, Foy, Drucker, 2012; OECD, 2014). One possible explanation of these differences is that there is variation in opportunity to learn (OTL, e.g., Schmidt, Zoido & Cogan, 2014) due to differences in curricula, organization of the educational system, quality of instruction, contextual factors, and home conditions.

Results from meta-analyses suggested that OTL has a strong average-effect size. Reviewed 51 research studies that were carried out during the last twenty years, and a secondary analysis of TIMSS and PISA results, Scheerens (2012) concluded that OTL has a larger effect on achievement than other effectiveness enhancing instruction conditions, such as learning time and instructional leadership.

Even though the importance of OTL is highly recognized, the effect sizes vary across PISA and TIMSS. Luyten (2017) examined the effect of OTL on math and science achievements of 22 countries participating in both TIMSS 2011 and PISA 2012. The average effect across the countries was 0.37 in PISA but basically zero in TIMSS. The explanation may have to do with the measurement property of OTL instrument used in these studies. For example, Luyten attributed the finding to the fact that “TIMSS OTL measures were based on teacher responses and the PISA OTL measures on student responses” (p. 111, Luyten, 2017). Further, the both studies not only use different sourced but also different items to measure OTL. It can thus be argued that the measures of OTL may be multidimensional and this needs to be attended to when study the effect of OTL on achievement.

The purpose of this proposed study is to scrutinize the construct validity of OTL, through gauging the measurement property of OTL measures. Specifically, we question the construct validity of the OTL measure used in PISA. For this purpose, we examine if the OTL measures in PISA 2012 involve construct irrelevant components such as self-concept. By separating OTL and other dimensions, we aim to show overlaps and differences in the OTL measures used in PISA and TIMSS. In further multi-level analyses with the adjusted OTL measure, we estimate the relationship between OTL and student achievement.    

Scheerens, J. (2017). Opportunity to Learn, Curriculum Alignment and Test Preparation: A Research Review. Springer International Publishing. Luyten, H. (2017). Predictive Power of OTL Measures in TIMSS and PISA. In Opportunity to Learn, Curriculum Alignment and Test Preparation (pp. 103-119). Springer International Publishing. Schmidt, W. H., Burroughs, N. A., Zoido, P., & Houang, R. T. (2015). The Role of Schooling in Perpetuating Educational Inequality an International Perspective. Educational Researcher, 44(7), 371-386. doi: 10.3102/0013189X15603982 Schmidt, W., P. Zoido and L. Cogan (2014), "Schooling Matters: Opportunity to Learn in PISA 2012", OECD Education Working Papers, No. 95, OECD Publishing, Paris. Paper retrieved from DOI: http://dx.doi.org/10.1787/5k3v0hldmchl-en OECD (2014), PISA 2012 Technical Report, PISA, OECD Publishing, Paris. Report retrieved from https://www.oecd.org/pisa/pisaproducts/PISA-2012-technical-report-final.pdf Mullis, I.V.S., Martin, M.O., Foy, P., & Drucker, K.T. (2012). The PIRLS 2011 International Results in Reading. Chestnut Hill, MA: TIMSS & PIRLS International Study Center, Boston College. OECD (2014). PISA 2012 Results in Focus: What 15-year-olds know and what they can do with what they know: Key results from PISA 2012. OECD: Paris.