The drive for school ‘effectiveness’ has taken various turns in recent decades, generating high-stakes accountability regimes, data mountains, a cult of leadership and most recently a demand for teaching to be securely ‘evidence-based’. The search for Evidence-Based Teaching (EBT) has resulted in phenomena such as the international appeal of John Hattie’s best-selling Visible Learning (2009), along with national products such as England’s Toolkit for Teaching and Learning (Higgins et al 2014), now marketed in various other national versions, Denmark’s Clearinghouse for Educational Research, and Austria’s Federal Institute for Educational Research, Innovation and Development (BIFIE). 

All of these have in common a search for certainty, which takes little account of the complexity of educating young people and the messiness of their lives. In an age of environmental, social and cultural risk, ‘evidence’ offers an illusory promise of certainty in schooling the next generation; it is premised on technical-rationalist discourses of efficiency and loses sight of the curricular need to address the diverse global challenges we face. This reinforces the power of globalised neoliberalism (Rømer 2014) whilst sheltering behind the appearance of technical neutrality.

Frequent appeals are made to Medicine as an exemplar of evidence-based practice, overlooking the disagreements and ongoing difficulties within that field. The demand for research to show ‘what works’ has a commonsense appeal, but forgets the need to consider aims and context. In addition to curricular and pedagogical narrowing, research activity is also at risk by marginalising or occluding anything not based on randomised controlled trials (RCTs) and meta-analysis. 

This symposium aims to outline a range of difficulties with evidence-based teaching. Drawing on real examples, it will examine problems of adapting RCT design to education settings, including randomisation, the nature of control groups, human agency and subjectivity (Pawson 2006), and the choice of outcome measures (Wrigley 2018). It will argue that a series of misunderstandings surround meta-analysis (in reality, statistical synthesis) including confusion over the meaning of effect size (Simpson 2018), too diverse a collection of activities subsumed within a single meta-analysis (the Apples and Oranges problem), and the inappropriateness of averaging. It will show how the top level of synthesis, which we could call meta-meta-analysis (eg Hattie; the Toolkit), compounds rather than negates these difficulties and presents teachers with misleading information and advice.

The symposium is however not only concerned with technical difficulties. It aims to look at philosophical issues concerning this form of research. We intend to provoke discussion of:

• educational activity as an open, semiotic and recursive system (Biesta 2010);
• the need to understand emergence  (Sayer 2010), rather than assumptions that learning accumulates in additive or linear ways;
• the appeal to ‘science’ by advocates of EBT;
• scientific experiments as attempts to move beyond surface appearances and identify and test out deep causes (Wrigley 2018) ; 
• the role of theory;
• the need for a stratified understanding of education and its disciplined study;
• the dangers of empiricism in statistical studies.

We will draw on the ontological and epistemology challenge of Critical Realism (eg Bhaskar 1978), as a theoretical safeguard against empiricism and reductionism, with references to parallel problems in natural and social sciences (Sayer 2010; Rose 2005).

 We also hope to stimulate a creative discussion on what kinds of research can best engage busy teachers whilst avoiding the reductionism of EBT.

Bhaskar, R (1978) A realist theory of science (Harvester) Biesta, G (2010) Why 'what works' still won't work: From evidence-based education to value-based education. Studies in the Philosophy of Education 29, pp 491-503 Hattie, J (2009) Visible learning: A synthesis of over 800 meta-analyses relating to achievement. (Routledge) Higgins, S, Katsipataki, M, Kokotsaki, D, Coleman, R, Major, L and Coe, R (2014) The Sutton Trust - Education Endowment Foundation Teaching and Learning Toolkit. London: Education Endowment Foundation https://educationendowmentfoundation.org.uk/evidence-summaries/teaching-learning-toolkit/ (accessed 12 Nov 2018) Pawson, R (2006) Evidence-based policy: A realist perspective (SAGE) Rømer, T (2014) The relationship between education and evidence. In K Petersen, D Reimer and A Qvortrup (eds) Evidence and evidence-based education in Denmark: The current debate. Cursiv no 14, Department of Education, Aarhus University http://edu.au.dk/cursiv (accessed 24 August 2018) Rose, S (2005) Lifelines: life beyond the gene (Vintage) Sayer, A (2010) Reductionism in social science. In R Lee (ed) Questioning Nineteenth-Century assumptions about knowledge: II Reductionism (SUNY Press) Simpson, A (2018) Princesses are bigger than elephants: Effect size as a category error in evidence-based education. British Educational Research Journal 44(5) pp897-913 Wrigley, T (2018) The power of 'evidence': reliable science or a set of blunt tools? British Educational Research Journal 44(3) pp359-376