Strict adherence to explicit methodological precepts is widely taken to be the hallmark of high-quality research. What is often referred to simply as “rigour” is understood to signify reliability, objectivity and quality. This is true, in particular, for the natural sciences, medicine, and quantitative social science. The format of systematic reviews is a case in point. Performed in accordance with explicit, predetermined protocols, systematic reviews are promoted as a scientifically respectable alternative to traditional, “narrative” reviews. The basic assumption is that the methodology of reviews be just as rigorous as that required in primary studies. Introduced in clinical research, the format of systematic reviews has since been established as a key component of evidence-based practice in many other areas, too. This development is closely linked to the current emphasis on transparency and accountability in policy discourse, and to increasing demands that professional practice be based on the best available knowledge. In policy circles across the developed world, systematic reviews are regarded as an indispensable instrument in attempts to bridge the perceived gap between research and practice.

Systematic reviews of educational research were introduced in 2000, in an initiative funded by the UK government. The programme was organised by the EPPI-Centre at the University of London, which is still a major actor in this area. Two years later, the What Works Clearinghouse was established by the US Department of Education, and subsequently, agencies conducting systematic reviews of educational research have been set up in numerous other countries. These initiatives have been heavily criticised by researchers belonging to the qualitative tradition, and systematic reviews remain controversial.

The aim of this paper is to contribute to the understanding of the resistance to systematic reviews of educational research. I have no stakes in the debate, and wish to favour neither advocates nor critics of this method of syntesising findings published in the primary literature. Irrespective of the merits and limitations of the methodology of systematic reviews, which are perceived very differently, I believe that elaborate criticisms of its character may carry an interesting message. I take it that sophisticated critics are positing a limit to reasonable attempts at formalising research methodology. At least some of them seem fully prepared to accept formal tools as being useful for many research purposes, and yet they vehemently resist formalisation of other parts of research practice.

Protocols, criteria and other formal tools employed in systematic reviewing may be regarded as a form of technology. Enabling practitioners to perform operations which would not have been possible without them, they also tangibly constrain the activity of their users. The interaction of human actors and technical artefacts is an important topic of Science and Technology Studies (STS). In this paper, I will employ three theoretical concepts drawn from leading STS analysts. The first concept is delegation. Introducing new tools in a given form of practice means delegating certain tasks to artefacts, and hence changing the nature of the practice. The second concept is the closely related one of script. Technical artefacts may be understood as carrying scripts prescribing the behaviour of their users. The third concept is the distinction between polimorphic and mimeomorphic actions. While the latter kind of action, presumably, can be delegated to technology, the former category refers to actions which can only be performed by humans.

This paper is an attempt to demonstrate that these concepts may throw new light on the barrier to meaningful formalisation posited by some of the critics of systematic reviews of educational research.

Bohlin, Ingemar 2000. A Social Understanding of Delegation. Studies in History and Philosophy of Science 31, 731-50. Collins, Harry & Martin Kusch 1998. The Shape of Actions: What Humans and Machines Can Do. Cambridge, Mass.: The MIT Press. Hammersley, Martyn 1997. Educational Research and Teaching: A Response to David Hargreaves’ TTA Lecture. British Educational Research Journal 23, 141-61. Hammersley, Martyn 2001. On ‘Systematic’ Reviews of Research Literatures: A ‘Narrative’ Response to Evans & Benefield. British Educational Research Journal 27, 543-54. Hammersley, Martyn 2005. Is the Evidence-Based Practice Movement Doing more Good than Harm? Reflections on Iain Chalmers’ Case for Research-Based Policy Making and Practice. Evidence & Policy 1, 85-100. Hammersley Martyn 2006. Systematic or Unsystematic, Is That the Question? Reflections on the Science, Art, and Politics of Reviewing Research Evidence. In A. Killoran, C. Swann & M.P. Kelly (eds.), Public Health Evidence: Tackling Health Inequalities. Oxford: Oxford University Press, 239-50. Latour, Bruno 1992. Where Are the Missing Masses? The Sociology of a Few Mundane Artifacts. In W.E. Bijker & J. Law (eds.), Shaping Technology/Building Society: Studies in Sociotechnical Change. Cambridge, Mass.: The MIT Press, 225-58. Latour, Bruno 1995. Reassembling the Social: An Introduction to Actor-Network-Theory. Oxford: Oxford University Press.