Since the late 1990s, the World Bank and the OECD have become highly influential international organizations that promote an agenda for evaluation on a global scale (Mundy and Verger, 201501; Sellar and Lingard, 2013; Lewis, 2017).  Alongside these powerful organizations, a number of ‘complex multilateral networks’ have also arisen and work to promote, on a supra-national level, a ‘common standards of evaluation practice’ for the developing world with an emphasis on impact evaluation (Rutkowski and Sparks, 2014).  Others argue that the use of evaluation, and the evidence-for-policy paradigm itself, has helped lay the groundwork for the rise of the ‘global education industry’ whose private actors use the language and imagery of scientific ‘evidence’ to promote their products and services (Hogan, Sellar and Lingard, 2016; Verger, Lubienski and Steiner-Khamsi, 2016). 

Within this evolving context, global interest in the use of an experimental design approach to generating evidence for education policy and practice appears to be increasing.  This approach advocates conducting randomized controlled trials (RCTs) to assess the effectiveness of educational practices and promotes carrying out systematic literature reviews based upon hierarchies of evidence in which RCTs are deemed ‘most rigorous’.  Advocates argue the approach can lead to knowledge of ‘best practice’, or ‘what works’, that can be shared across contexts and contribute to an expanding global evidence base to raise students’ academic achievement.  Proponents insist the approach is not about “telling teachers what to do” but rather “empowering teachers and setting a profession free” from the constant barrage of reforms from government and educational popular fads (Goldacre, 2013).  Other researchers challenge these arguments, objecting to the view of RCTs as an evidentiary ‘gold standard’(Hanley, Chambers and Haslam, 2016) and arguing that such experimental designs cannot accurately determine ‘what works’ in education without accounting for contextual differences and temporal processes (e.g., Tikly, 2015). 

Nevertheless, RCTs in education have been promoted within England with the founding of the Education Endowment Foundation (EEF) in 2011 by a grant of £125 million (146 million Euros) from the U.K Department of Education.  The EEF aims to raise the educational attainment of children facing disadvantage by funding and evaluating promising educational innovations and encouraging others to use evidence and adopt innovations that are found to be effective.  Between 2011 and 2015, the EEF commissioned 115 experimental trials in England involving over three-quarters of a million pupils.  The EEF also developed an online Teaching and Learning Toolkit designed to provide accessible information to school leaders about the relative effectiveness, estimated cost, and likely benefit of different pedagogical techniques based on the best available evidence.

This paper considers how RCTs fit into the global script of evidence-based education and how and why the approach is being adopted and adapted in a variety of international contexts.  Using, a theoretical and conceptual framework proposed by Verger, Lubienski, and Steimer-Khamsi (2016), this paper seeks to explains the EEF’s approach to evidence for educational decision-making in England and explore and document the international spread and appeal of its approach and Toolkit.  A number of organizations in other countries have become interested in learning from and/or partnering with the EEF to utilize its approach and Toolkit, and at least three organizations across three continents have done so.  Through an investigation into these cases, the paper explores:  Why and how is the RCT-based approach in education, as exemplified by the EEF, being adopted and translated in other countries?  And what types of international networks exist, or are forming, to further the use of RCTs to build a ‘global evidence bank’ for educational decision-making?

Data for the study was gathered through fifteen semi-structured interviews with key informants whose organizations are involved in the generation, translation, and/or mobilization of evidence from RCTs for education policy or practice. Data collection began with a desk-based review of the history, goals, and current research of England’s Educational Endowment Foundation (EEF) and the development of its Teaching and Learning Toolkit. Next, three rounds of interviews were carried out. First, interviews were conducted with key informants from EEF to gain further insight to the evolution of their approach to conducting and using RCTs to build their online Toolkit. From these interviews, three instances in which the EEF’s work was being used and adapted elsewhere were identified in Scotland, Chile, and Australia. Then, in a second round of interviews, leads attained from the first round were followed up and interviews with the staff of the organizations partnering with the EEF in Scotland, Chile, and Australia were conducted. These interviews aimed to collect data on how and why the EEF’s Toolkit was being used and combined with what approaches to knowledge mobilization. Finally, a third round of interviews were conducted with the funders and supportive partners of these new organizations in Chile and Australia to further gain insight into how and why the RCT approach was being utilized and promoted in the field of education. At this stage, other organizations utilizing an evidence toolkit for use in the education field were also identified and representatives from those organizations interviewed to uncover any common links. All interviews were audio recorded and then transcribed to enable further analysis. Finally, network maps were generated using NodeXL to offer a visual representation of the connections between research organizations advocating for the use of RCTs and similar quasi-experimental designs and their funders and other partners. Using NVivo, interview transcripts were analyzed using critical discourse analysis (Fairclough, 2010) to identify themes underpinning the spread and use of RCTs and toolkits for evidence generation and mobilization. In addition, the use of critical discourse analysis enabled the identification of wider contextual factors, personal experiences, and relational power dynamics shaping each actors’ views of RCTs and the use of the Teaching and Leanrning Toolkit. The results of the discourse analysis were then combined with the insights gained from carrying out a documentary analysis of reports and documents produced by the organization studied.

The findings suggest there is growing interest in RCT evidence and systematic review to inform educational decision-making among a number of researchers, funders, and policy actors in a variety of contexts. A variety of rationales were uncovered to explain the appeal of RCTs and the EEF Toolkit, and it became evidence that both contextual factors and actors’ personal background and experiences shaped such rationales. Certainly, the openness of the EEF’s leaders toward requests for collaboration from organizations abroad facilitated the building of new networks and the spread of the EEF approach and Toolkit beyond England. However, the EFF’s collaboration came with certain conditions, such as the guarantee that its Toolkit not be commercialized, and thus furthered the Foundation’s goal of promoting evidence-based practices amongst practitioners and policy-makers. Interestingly, the outcome of the collaboration between these organizations was not uniform. While the Toolkit was taken and adapted to new contexts in all cases, the extent to which the EEF’s approach and practice of commissioning studies was embraced varied among the partnering organizations. It is arguably too soon to assess the wider impacts of these organizations and their work on policy and practice in the various countries in which they are situated. Still, an additional case was also uncovered of a non-profit organization that developed an Early Years Toolkit – independently of the EEF – to promote effective early educational interventions in developing countries. Interestingly, this particular international non-profit was founded in 2008 by a former World Bank Vice President. This connection and others uncovered lends further weight to the argument that expanding networks of supra-national institutions and non-governmental organizations are cultivating particular evaluation regimes and user-friendly dissemination tools to encourage evidence-based practice across countries. Still, the particular context of the host country clearly shapes the way in which evidence is viewed.

Fairclough, N. (2010) Critical discourse analysis. Abingdon, Oxon, U.K.: Routledge (Longman applied linguistics). Goldacre, B. (2013) ‘Building evidence into education’. Hanley, P., Chambers, B. and Haslam, J. (2016) ‘Reassessing RCTs as the “gold standard”: synergy not separatism in evaluation designs’, International Journal of Research & Method in Education, 39(3), pp. 287–298. doi: 10.1080/1743727X.2016.1138457. Hogan, A., Sellar, S. and Lingard, B. (2016) ‘Corporate Social Responsibility and Neo-Social Accountability in Education’, in Verger, A., Lubienski, C., and Steiner-Khamsi, G. (eds) World Yearbook of Education 2016: The Global Education Industry. Abingdon, Oxon, U.K.: Routledge, pp. 107–124. Lewis, S. (2017) ‘Policy, philanthropy and profit: the OECD’s PISA for Schools and new modes of heterarchical educational governance’, Comparative Education, 53(4), pp. 518–537. doi: 10.1080/03050068.2017.1327246. Mundy, M. and Verger, A. (201501) ‘The World Bank and the global governance of education in a changing world order’, International Journal of Educational Development, 40, pp. 9–18. Rutkowski, D. and Sparks, J. (2014) ‘The new scalar politics of evaluation: An emerging governance role for evaluation’, Evaluation, 20(4), pp. 492–508. doi: 10.1177/1356389014550561. Sellar, S. and Lingard, B. (2013) ‘The OECD and global governance in education’, Journal of Education Policy, 28(5), pp. 710–725. doi: 10.1080/02680939.2013.779791. Tikly, L. (2015) ‘What works, for whom, and in what circumstances? Towards a critical realist understanding of learning in international and comparative education’, International Journal of Educational Development, 40(Supplement C), pp. 237–249. doi: 10.1016/j.ijedudev.2014.11.008. Verger, A., Lubienski, C. and Steiner-Khamsi, G. (2016) ‘The emergence and structuring of the global education industry: Towards an analytical framework’, in Verger, A., Lubienski, C., and Steiner-Khamsi, G. (eds) World Yearbook of Education 2016: The Global Education Industry. New York, NY: Routledge, pp. 3–24.