Session Information
22 SES 10 E, Quality Assessment and Evaluation
Paper Session
Contribution
The Australian research quality assessment, Excellence in Research for Australia (ERA), commenced in 2010, was repeated in 2012, and will run again in 2015 (Australian Research Council (ARC), 2014). In brief, each discipline submission includes an Explanatory Statement, six years of publication data with 30% identified for review, and three years of grant income data. The purpose of ERA is “to progressively increase the number of research groups performing at world-class levels, as measured by international performance benchmarks” (Commonwealth of Australia, 2009, p. 33). The research quality score for each group discipline submission rated within the range of 5 to 1 with 5 and 1 being ‘well above’ and ‘well below’ world standard respectively. Elsewhere, research assessments have long been part of the higher education sector (Research Assessment Exercise, 2008).
Based on ERA results, there is considerable room for research improvement for some university discipline groups because only “40 per cent (sic) of Australian university research groups were rated as ‘above world standard” in ERA 2010 (Norton, 2013, p. 8). As a discipline, Education fared poorly in ERA 2010 and 2012. For example, in ERA 2012, 42 of the 85 units of evaluation (submissions of a discipline group) were assessed at ‘below’ or ‘well below’ world standard (ARC, 2012a, 2012b). The research questions are:
- Which university discipline groups in Education require support to build research capacity?
- How can ERA data be used to support the improvement of research performance in Education?
The ERA process uses the discipline norms for quality assessment. For example, Education is assessed by peer review (e.g., Ramsden, 1998) whereas science disciplines use citation data. In peer review disciplines, such as Education, multiple reviews of the selected ERA research outputs are sought from peer reviewers by discipline panels of the ERA Research Evaluation Committee (REC). These discipline panels then synthesise the reviews and evaluate the overall discipline group submissions relative to international benchmarks.
Research performance across Australian universities differs substantially across universities and at the discipline level (e.g., ARC, 2012b). This variation is not surprising because there are three broad phases of institutional research development (Bosch &Taylor, 2011). At the Instilling Phase, there is little or no research activity in the university. At the Broadening Phase, some discipline groups are performing highly and are recognised externally. At the Honing Phase, research is embedded in institutional life and high performance is part of the institutional culture.
Universities Australia, the peak member body of Australian universities, strongly supports the government’s agenda to improve research performance (Universities Australia, 2014). This accord between the government and Australian universities reflects the relationship of agency in which each assumes particular roles (Kivistö, 2008). The government acts as the principal in the relationship and delegates work in universities as its agents. There are multiple principal-agent relationships at work because each university has a performance agreement with the government (ARC, 2012b). Paralleling the relationship between the government, as principal, and the university, as agent, is the relationship between the ARC and academics. The ARC assumes the role of a principal when it delegates assessment and review work to academics in paid (i.e., REC) or unpaid capacities (i.e., peer reviewers). Despite the work undertaken by academics as reviewers or REC members, universities receive no feedback on the reviews nor are they privy to the rationale for decision making about the discipline group ratings. Instead, a national report is published by university and discipline codes together with benchmark data across the sector (e.g., ARC, 2012b). For more about ERA and educational research see Seddon, Bennett, Bennett, Bobis, Chan, Harrison, and Shore (2013).
Method
Expected Outcomes
References
Australian Research Council (2012a). ERA 2012 evaluation handbook. Retrieved from http://www.arc.gov.au/pdf/era12/ERA%202012%20Evaluation%20Handbook_final%20for%20web_protected.pdf Australian Research Council (2012b). Excellence in Research for Australia 2012: National report. Retrieved from http://www.arc.gov.au/era/era_2012/outcomes_2012.htm Australian Research Council (2013). ARC Open Access Policy. Retrieved from http://www.arc.gov.au/applicants/open_access.htm Australian Research Council (2014). About ARC. http://www.arc.gov.au/about_arc/ Bosch, A. & Taylor, J. (2011). A proposed framework of institutional development phases. Journal of Higher Education Policy and Management, 33(45), 443-457. Commonwealth of Australia (2009). Powering ideas: An innovation agenda for the 21st century. Retrieved from http://sydney.edu.au/documents/about/higher_education/2012/20120308%20PoweringIdeas.pdf Kivistö, J. (2008). An assessment of agency theory as a framework for the government–university relationship. Journal of Higher Education Policy and Management, 30(4), 339-350. Mayer-Schönberger, V. & Cukier, K. (2013). Big data: A revolution that will transform how we live, work, and think. London: John Murray. Norton, A. (2013). Mapping Australian higher education, 2013 version, Grattan Institute. Retrieved from http://grattan.edu.au/static/files/assets/28a92f8b/184_2013_mapping_higher_education.pdf Ramsden, P. (1998). Learning to lead in higher education. London: Routledge Research Assessment Exercise (2008). RAE 2008 Submissions: UOA 45 Education. Retrieved from http://www.rae.ac.uk/submissions/submissions.aspx?id=45&type=uoa Seddon, T., Bennett, D., Bennett, S., Bobis, J., Chan, P., Harrison, N., & Shore, S. (2013). Educational research Australia: A changing ecology of knowledge and practice. Australian Educational Researcher, 40(4), 433-451. DOI 10.1007/s13384-013-0104-1 Universities Australia (2014). The voice of Australia’s universities. Retrieved from https://www.universitiesaustralia.edu.au/research-innovation-workforce
Search the ECER Programme
- Search for keywords and phrases in "Text Search"
- Restrict in which part of the abstracts to search in "Where to search"
- Search for authors and in the respective field.
- For planning your conference attendance you may want to use the conference app, which will be issued some weeks before the conference
- If you are a session chair, best look up your chairing duties in the conference system (Conftool) or the app.