Session Information
Contribution
GCE A-levels and GCSEs are high stakes tests, which affect the life chances of thousands of people (mostly in the UK) hoping to proceed to employment or further study. Accurate marking of these qualifications is therefore crucial. Until recently, two areas of research into these issues were particularly under-explored:o the efficacy of examiner training;o the cognitive strategies used to mark examinations. Baird et al (2004) and Greatorex and Bell (in press) undertook studies evaluating training efficacy for experienced examiners, exploring GCSE history and A-level biology respectively. When marks for the whole examination were considered, aspects of the training were found not to greatly improve average marking accuracy. Suto and Greatorex (2006, in press) and Greatorex and Suto (2006) found that examiners used five different cognitive marking strategies to mark GCSE mathematics and business studies examination questions as well as A-level physics questions. They interpreted these strategies within dual processing theories of human judgement, which currently predominate in cognitive psychology.Suto and Nadas (in submission) categorised these marking strategies as either 'simple' or 'more complex'. They found that after training, there were few differences in the accuracies of experienced and inexperienced GCSE mathematics and physics examiners. However, questions requiring apparently more complex marking strategies were marked less accurately than those requiring apparently simple strategies only. We link these areas of research in this paper. Our aim was to explore how the apparent cognitive marking strategies used to mark questions relate to training efficacy.We re-analysed data from one study (Greatorex and Bell, in press) and present an aspect of the results from another study (Suto and Nadas, in submission). Both studies had a pre- and post- test design: the marking accuracy for every question was measured both before and after examiner training. Changes in marking accuracy were used as indicators of training efficacy.In total, 57 examiners participated in the studies; in each study, all examiners marked identical samples of students' answers. The cognitive strategies apparently needed to mark each question were judged by researchers prior to conducting the statistical analysis. We investigate the following hypotheses:1. questions requiring only simple strategies will be marked accurately by all examiners both before and after training; 2. questions requiring more complex strategies will be marked less accurately than those requiring only simple strategies by both experienced and inexperienced examiners; 3. questions requiring more complex strategies will be marked more accurately after training than beforehand, by both experienced and inexperienced examiners. Baird, J-A., Greatorex, J. and Bell, J.F. (2004) 'What makes marking reliable? Experiments with UK examinations.' Assessment in Education Principles, Policy and Practice 11, (3) 333-347. Greatorex, J and Bell, J. F. (in press) 'What makes AS marking reliable? An experiment with some stages from the standardisation process.' Research Papers in Education. Greatorex J. and Suto, W.M.I. (2006) 'An empirical exploration of human judgement in the context of school examinations'. Paper presented at the annual conference of the International Association for Educational Assessment, 21st to 26th May, Singapore. Suto, W.M.I. and Greatorex, J. (2006) 'A cognitive psychological exploration of the GCSE marking process.' Research Matters: A Cambridge Assessment Publication 2, 7-11. Suto, W.M.I. and Greatorex, J. (in press) 'What goes through an examiner's mind? Using verbal protocols to gain insights into the GCSE marking process.' British Educational Research Journal. Suto, W.M.I. and Nadas R. (in submission) 'The Marking Expertise projects: empirical investigations of some popular assumptions.' Research Matters: A Cambridge Assessment Publication.We are planning to version this paper for an international journal.
Search the ECER Programme
- Search for keywords and phrases in "Text Search"
- Restrict in which part of the abstracts to search in "Where to search"
- Search for authors and in the respective field.
- For planning your conference attendance you may want to use the conference app, which will be issued some weeks before the conference
- If you are a session chair, best look up your chairing duties in the conference system (Conftool) or the app.