In the last decades several studies had been looking to the nature and effects of policy instruments for the field of education, such as the production of data (Ozga, 2009) supranational instruments such as PISA (Grek, 2009; Carvalho & Costa, 2014) and the EU’s Open Method of Coordination (Alexiadou, 2014) or the rise of evaluation in schools and universities to make them more accountable (Ozga, 2011; Rosli & Rossi, 2016). For the field of educational research policies, the studies on the nature and effects of research funding instruments has been punctual, often looking to a single instrument in a particular moment (Oancea, 2014). The research evaluation systems and thematic programmes for research have been acknowledged as policy instruments that have been rising in form and extent across disciplines and countries (Lepori et al, 2007). Thus, having as a case study the last 30 years of educational research policies in England, I intend to analyze the nature, evolution and effects of two of the most emblematic policy instruments for the field of educational research – the Research Excellence Framework (REF) and the Teaching and Learning Research Program (TLRP). For that purpose I conceptualize policy instruments as institutions (Lascoumes & Le Galès, 2007), with regulative, cognitive, and normative dimensions, incorporating and expressing specific ideas, norms, and values for the organizational field of educational research.  The central aim of the paper is to discuss the nature and evolution of the REF and TLRP (macro level) and the effects of such instruments in the research organizations and individuals (meso and micro level) in order to answer to the following research question – How do educational research instruments impact the field of educational research and how do research organizations respond to them? Thus, based on political sociology and sociology of science I first present an analytical model to analyze policy instruments for contemporary research based on the Lascoumes and Le Galès’ instrumentation approach and complement it with the institutional pillars proposed by Scott (2013), in order to identify research policy instruments, such as research evaluation systems and thematic programmes as well as their cognitive, normative and regulative dimensions. Secondly, I apply such model to the case of educational research policies in England. Based on literature review and policy documents content analysis I trace the evolution of the REF and the TLRP shading some light on their cognitive, normative and regulative dimensions. Thirdly, based on a content analysis of projects funded by the TLRP, an analysis of submission for the unit of assessment of Education for the last three assessment exercises and a set of interviews to policy experts and actors, I discuss how the impact of the ideas, norms and values conveyed by these two policy instruments are leading to an institutional change for the educational research field. 

To respond to my aim and research question a complex set of methods were employed in order to collect and analyse a set of secondary and empirical data. Thus, I first conducted a literature review of previous studies either focusing on the evolution of educational research either on the development of the research assessment exercise. Secondly, I used policy documents content analysis for the REF and TLRP (n=26) looking at particular to White papers for the field of higher education, to the Higher Education Funding Council for England reports, the research assessment exercises guidelines and reports, and to the Economic and Social Research Council (ESRC) reports for the TLRP. Therefore, for the specific case of the TLRP I gathered the 54 ESRC-funded projects for England and look at the project’s themes, methodologies, levels, objects and disciplines of the contents supported by such programme. For the case of REF I gathered the Education unit of assessment submissions for the last three research assessment exercises (RAE 2001, n =82; RAE 2008, n =81; and REF 2014, n =76) and looked in particular to the number of submitted staff, the external funding received and the type and quantity of outputs submitted in order to understand the characteristics of the submission behaviour. Finally, I conducted a series of interviews with experts and actors in the field of educational research, in order to understand the effects and the responses from the research organisations and individuals to such policy instruments. All the content analysis (policy documents, research projects and interviews) were coded in MAXQDA and followed a thematic analysis approach – familiarisation with data, generation of initial codes, searching for themes, reviewing themes and, defining and naming the themes (Braun & Clarke, 2008).

The results show that both policy instruments convey ideas of quality, relevance and internationalisation, the later for the REF. For the specific case of TLRP the results show a greater focus on teaching and learning themes, a mix-method approaches as preferred methods, higher education as one of the privileged sectors and a greater attention to students and teachers as the main objects of analysis. For REF, the results show a growing selectivity on the submitted staff, the articles as the preferred outputs formats and a progressive concentration of funding in the high-rated departments. The impact of TLRP is also visible on the research assessment submission, especially in the last two cycles. The actors in the field report both positive – strength of research cultures, collaboration, legitimacy for the educational research field, and negative consequences – stratification among departments, some feeling of playing the game, recruitment strategies, preference for specific types of research and outputs. Overall, the results also show the role of both policy instruments to create or reinforce new normative pressures for the field of educational research such as the reflexivity of the educational researchers about the relevance and the impact of their work, the need to attract external funding from different sources, and the will to secure or maintain their position in the stratified higher education system. I discuss such changes as epistemic drift in the field of educational research since progressively the attention on internal quality control is replaced by external relevance, putting in evidence how policy instruments have the capacity to inflict cognitive and normative scripts and, thus, govern research.

Alexiadou, N. (2014) Policy Learning and Europeanisation in Education: the governance of a field and the transfer of knowledge, in A. Nordin & D. Sundberg (eds) Transnational Policy Flows in European Education. The Making and Governing of Knowledge in the Education Policy Field, 123-140. Oxford: Symposium Books. Braun, V. & Clarke, V. (2006) Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2), 77-101. Carvalho, L.M. & Costa, E. (2014) Seeing education with one’s own eyes and through PISA lenses: considerations of the reception of PISA in European countries, Discourse: Studies in the Cultural Politics of Education, 36(5), 638-646. Grek, S. (2009) Governing by numbers: the PISA ‘effect’ in Europe, Journal of Education Policy, 24(1), 23-37. Lascoumes, P. & Le Galès, P. (2007) Introduction: Understanding Public Policy through Its Instruments – From the Nature of Instruments to the Sociology of Public Policy Instrumentation. Governance, 20, 1–21. Lepori, B., van den Besselaar, P., Dinges, M., Potì, B., Reale, E., Slipersæter, S., Thèves, J. & van der Meulen, B. (2007a) Comparing the Evolution of National Research Policies: What Patterns of Change? Science and Public Policy, 34(6), 372-388. Oancea, A. (2014) Research Assessment as Governance Technology in the United Kingdom, Zeitschrift für Erziehungswissenschaft, 17(6), 83-110. Ozga, J. (2009) Governing education through data in England: from regulation to self-evaluation, Journal of Education Policy, 24(2), 149-162. Ozga, J. (2011) Governing Narratives: “local” meanings and globalising education policy, Education Inquiry, 2(2), 305-318. Rosli, A. & Rossi, F. (2016) Third-mission policy goals and incentives from performance-based funding: Are they aligned?, Research Evaluation, 25(4), 427-441. Scott, W.R. (2013) Institutions and Organizations. Thousand Oaks, CA: SAGE.