Evaluation as a public policy instrument: a study on calls for R&D projects in educational research
Author(s):
Sofia Viseu (presenting / submitting) Estela Costa
Conference:
ECER 2015
Format:
Paper

Session Information

23 SES 11 B, Research Policies and the Politics of Research

Paper Session

Time:
2015-09-10
17:15-18:45
Room:
418.Oktatóterem [C]
Chair:
Peter Kelly

Contribution

On a transnational scale, in what seems to be a policy transfer process (Dolowitz & March, 2000), R&D policies embrace performance-based research funding systems (Hicks, 2012). Often justified by the importance of knowledge in public policies (Freeman et al 2012), or conceived as part of the new managerialism regime in public policies (Curtis, 2008), evaluation becomes a powerful instrument of science regulation: the rising of indirect financial provisions, the ability to raise financial provisions in competitive calls, the evaluation based on bibliometric criteria and publishing in refereed journals are now central in scientific activity. As in education, in scientific systems it seems that evaluation is becoming the system itself (Ball, 2001).

It should be noted that this phenomenon dates back to a wider process of reconfiguration of the State's role, through which traditional modes of regulation, based on formal rules and authority, are gradually replaced by other a posteriori based control action (Maroy, 2004). In fact, there is an increase and diversification of instruments that mediate the relationship between policy makers and social actors: calls and awards, contracts, best practices and, of course, evaluation are preferred modes of action orientation (Pons & van Zanten, 2007).

Stressing the non-neutral character of these devices, Lascoumes and Le Galés (2007) conceive public policy instruments as “both technical and social, [because it] organizes specific social relations between the state and those it is addressed to, according to the representations and meanings it carries.” (p. 4). As Ozga (2009) shows, from this perspective, evaluation is a policy instrument. Therefore, in this presentation, evaluation will be conceived as an instrument in R&D public policies. More than comparing performances or reflecting on research quality, our interest is show how public authorities seek to coordinate, influence and guide the scientific work through evaluation. Inspired by the work of Lascoumes and Le Galès (2004, 2007) and  using different levels of observation of public policy instruments, our interest lies on the techniques and social tools at a micro scale that operationalize evaluation in R&D policies.

To do so, we will present an empirical study focused on calls for funding R&D Projects, in Portugal, a Foundation to Science and Technology (FCT) initiative, the public agency responsible for the main R&D policies. In this competitive call, evaluated by peer review panels, research centres and universities can apply to public funding for research projects in all scientific areas, including educational research. We intent (1) to describe and analyse the techniques and tools designed by public authorities for this call; (2) to show how that the techniques and tools for projects evaluation carries a certain referential about what science is supposed to be (Muller, 2004).

Despite the focus on Portuguese reality, as Kassim and Le Galès (2010, p. 1) recall, "the study of policy instruments in national settings has made an important contribution to our understanding of political systems, public policy, and relations between state and citizen". Simultaneously, research has shown the importance of Europeanization through instrumentation, especially in science public policy (see Bruno, Jacquot & Mandin, 2006). Therefore, this presentation aims to be taken as a typical example of public policy process instrumentation. 

Method

The study followed a qualitative approach (Bogdan & Bilken, 1994; Cohen & Manion, 1994), by analysing a wide range of documents about R&D Projects calls for funding in the last decade: regulations, online submission of forms, guidelines to evaluation panels, statistics. These documents are public and available on the FCT web site. Documental analysis was carried out according to two main orientations. Firstly, we took a close look on the forms, filing instructions and evaluators guideline, regarding the structure, the dimensions of scientific activity they mention, in sort, what was asked and how. Secondly, we studied the characteristics of the projects approved for funding in educational research in the last decade, namely: research themes; type of research (fundamental or applied); team size; team composition (national or mix/ international). Through descriptive statistics procedures, we identified the major trends and changes along 10 years of R&D projects call evaluation and which are the preferential features of the projects approved for funding.

Expected Outcomes

Data showed that there isn’t an explicit orientation a prior on the R&D Projects calls concerning the characteristic of the research projects that showed be financed. However, the documental analysis on the evolution of the projects approved for funding allowed us to observe what evaluation induces, as a regulation tool, and the emergence of a certain referential about what is research and scientific work: collaborative, published and internationalized, applied and with technical or socio-economic impacts. It turns out that the evaluation of scientific activity, as a public policy instrument, is not value-neutral, it transport values and interpretations of the social and conceptions about the modes of regulation, and its properties are inseparable of the purposes assigned to it.

References

LASCOUMES, Pierre & LE GALÈS, Patrick (2004). L’action publique saisie par ses instruments. In P. Lascoumes & P. Le Galès (Dir.), Gouverner par les instruments. Paris : SCIENCES PO, pp.11-44. LASCOUMES, Pierre & LE GALÈS, Patrick (2004). Instrument. In L. Boussaguet; S. Jacquot & P. Ravinet (Dir.), Dictionnaire des Politiques Publiques. Paris: SCIENCES PO, pp. 267-275. Maroy, C. (2009). Convergences and hybridization of educational policies around ‘post‐bureaucratic’models of regulation. Compare, 39(1), 71-84. MULLER, Pierre (2004). Les Politiques Publiques. Paris: Puf. MULLER, Pierre & SUREL, Yves (1998). L’Analyse des Politiques Publiques. Paris: Montchrestien. Ozga J (2008) Governing Knowledge: research steering and research quality, European Educational Research Journal, 7(3), pp.261-272.

Author Information

Sofia Viseu (presenting / submitting)
University of Lisbon
Institute of Education
Lisboa
Instituto de Educação, Universidade de Lisboa
Lisboa

Update Modus of this Database

The current conference programme can be browsed in the conference management system (conftool) and, closer to the conference, in the conference app.
This database will be updated with the conference data after ECER. 

Search the ECER Programme

  • Search for keywords and phrases in "Text Search"
  • Restrict in which part of the abstracts to search in "Where to search"
  • Search for authors and in the respective field.
  • For planning your conference attendance, please use the conference app, which will be issued some weeks before the conference and the conference agenda provided in conftool.
  • If you are a session chair, best look up your chairing duties in the conference system (Conftool) or the app.