There has been much debate over the past decade about the quality of the education research in the UK and elsewhere (Oancea, 2005). The argument shifted in recent years from sharp criticisms towards a more constructive recognition of the strengths of research in the field. However, existing knowledge of what types of criteria are currently in use in different contexts, and how exactly they are used, is at best patchy. For example, assumptions are often made about what aspects of research are prioritised by scholarly journals in their refereeing processes, as being different from those prioritised in, for example, assessments leading to award of grants, or in evaluations of completed research by user or commissioning bodies. However, the evidence to back these assumptions is mostly based on individual experiences of being part of such assessment processes. This paper is based on a short project funded by the ESRC/TLRP and aiming to produce an overview of differences and similarities between the concepts of quality with which different stakeholders operate in different contexts of assessment and in different countries of the UK, and to relate it to similar international efforts made by, for example, the UNESCO, the ESF, or the ICSS. Existent reviews of quality of research in education (e.g. NERF, 2000; Thomas, 2007; SFRE, 2008) and in other fields (e.g. Cabinet Office, 2003; Becker at al, 2006) have also played with this idea, but they had different purposes and thus any such work remained unsystematic and small-scale, without actually looking in detail at the existent sets of criteria and at their use. The “framework” by Furlong and Oancea (2005) was one attempt to draw together about 40 existent sets of criteria, but it did not organise them systematically by site of assessment, nor did it distinguish between criteria-in-use and aspirational sets of criteria proposed in the literature. The small-scale project reported in this paper builds on that work. The following questions are being addressed: 1. How do quality criteria vary with the context of assessment and with the agenc(ies) deciding the aims, process and use of the assessment outcomes? 1.1.What are the current main sites of assessment of education research? 1.2.What criteria are currently used in each site, and how do they compare across sites? 2. To what extent are these different operationalisations of conceptions of quality compatible with each other? 2.1.Are they internally and externally consistent? 2.2.What are the areas of commonality and of difference?

The project consists of documentary analysis of criteria and guidelines collected from the following sites and agents of assessment: 1. Journal publication: refereeing guidelines 2. Grant awarding: assessment criteria used by different funders for different funding schemes and programs 3. Commissioning: criteria used in selection of bids 4. Output evaluation: criteria used in the assessment and/or grading of completed projects and publications 5.[Other contexts]: as feasible, criteria from consultancy, private research organisations, and users. The above will be compared with criteria used in research training (if feasible within the time available), and with aspirational criteria from the literature (“ideal” criteria), i.e.: 6. [Degree awards]: criteria used for the assessment of PhD and EdD theses. 7. Aspirational criteria from the literature on research quality 8. Criteria for knowledge intermediation, for developmental work, for evaluation research, and for practice-based and practitioner research

The project is still ongoing, and its preliminary report is due in spring 2009. We expect to be able to build a comparative framework outlining concepts of quality in different assessment contexts, as evidenced by current use of criteria and guidelines. Although the UK is used as a case study, many of the contexts analysed are international in nature (e.g. journal guidelines), and our discussion will also incorporate several quality-related initiatives in Europe and internationally. This project was also be the first step in a longer-term research initiative, which intends to look at the qualitative detail of actual assessment practices to which education research is subject(thus moving beyond documentary analysis).