A Case of Positive Washback: An Exploration of Pre-release Examinations on Geography Classroom Practice.
Conference:
ECER 2008
Format:
Paper

Session Information

09 SES 03A, Tools and Methods in Assessment (Part 1)

Paper Session

Time:
2008-09-10
14:00-15:30
Room:
AK2 139
Chair:
Sabine Hornberg

Contribution

Assessment in schools in the UK takes many different forms, such as terminal exam papers, coursework elements, open book examinations, synoptic assessments, and the use of pre-release materials. The roots of this variety lie partly in the quest to extend the capacity of assessment tools to assess wider or deeper performance features. Pre-release materials are an established feature of some examinations in the UK. These examinations allow teachers and students access to materials that will be used during assessments at a later date. Although there is a clear implication that these materials provide a useful function in the assessment process there is little research about how teachers make use of them. This study gathers information about teachers’ practices and perceptions around a set of General Certificate of Secondary Education (GCSE) Geography pre-release materials and explore the effectiveness of strategies in terms of learner performance and emotional affect. GCSEs are the main form of national examination taken at the end of compulsory schooling in the UK. The project had three general research questions: did teachers use pre-release materials in different ways (what guided their planning and use of these materials); did students perceive them to be helpful in exam preparation (did they motivate or influence student anxiety levels); and, was there a link between pre-release teaching strategies and student outcomes? A fundamental premise underlying the rationale for this research is to understand better the potential effects that assessment procedures have on classroom practices. This is useful because it allows assessment procedures to be evaluated in terms of the match between their real and intended consequences. This is very important since Messick’s (1989) broad notion of validity is used as a theoretical framework for this study. Messick argues that validity is a multi-faceted construct which requires an integrated judgment to be reached in order to evaluate assessment tools. This study investigates the degree to which the pre-release assessment mode contributes to validity enhancement features such as reduced student stress and anxiety, broader and higher order skills assessment, ‘positive achievement’ elicitation, enhanced access for less able learners, improved student motivation, and higher levels of ecological validity.

Method

The study involved six state comprehensive secondary schools which were entering Year 10 students (approximately 15 years of age) for a GCSE Geography examination. These schools, all located around the east of England, varied in size and attainment levels, with half exhibiting achievement levels above regional and national expectations and the other half falling below these expectations. Data were collected via a mixture of qualitative and quantitative methods, allowing for triangulation. Teachers were interviewed prior to the release of the assessment materials and following the final DME examination to gather data about aspects of their use of pre-release materials and their broader geography teaching practices. Researchers also conducted systematic classroom observations of lessons in which the teachers used the pre-release materials. Students were surveyed to gather their affective responses to the pre-release materials. This survey included an adapted motivational inventory based on the Student Opinion Scale (Sundre, 1999) and a selection of open response questions. This inventory was used because it integrated the related factors of effort and importance which have been successfully used in other contexts to explain differences in student motivation levels during academic tests (Johnson, 2007; Sundre and Moore, 2002; Sundre and Wise, 2003). Finally, using student performance data in the live DME examination, statistical regression analyses were conducted to investigate whether any associations could be discerned between student outcomes, general ability, student gender and teacher practices.

Expected Outcomes

The study findings suggest that the pre-release examination mode had a number of positive effects. It appeared to be beneficial in terms of its motivational effects; providing a positive learning experience; and facilitating the assessment of different and generally higher order, types of skills to those often tested by other exams. The gender effect with regard to motivation levels was a surprising finding, counter to the teachers’ general perceptions. Explanations for this finding are sought in the literature relating to gendered learning styles (Oates, 2007). The statistical performance analysis was able to suggest some tentative relationships between teaching strategy use, student performance and motivation. The data also elicit a number of practical concerns associated with the assessment model which were voiced by teachers and students. These also highlighted some of the tensions that appear to exist between the instrumental aspects of assessment and the role of ‘enjoyment’ in learning.

References

Johnson, M. (2007) Does the anticipation of a merit grade motivate vocational test-takers? Research in Post-Compulsory Education, 12 (2) 159–179. Messick, S. (1989) Validity, in: R.L. Linn (ed.), Educational measurement (3rd ed., pp. 13-103), New York: American Council on Education. Oates, T. (2007) ‘Underachieving boys’ and ‘overachieving girls’ revisited – rhetoric and reality, in: Kate Myers, Hazel Taylor, Sue Adler and Diana Leonard (Eds); Genderwatch: Still Watching…Stoke-on-Trent: Trentham Books, pp. 1-7. Sundre, D. L. (1999) Does examinee motivation moderate the relationship between test consequences and test performance? (Report No. TM029964). Harrisonburg, Virginia: James Madison University. (ERIC Document Reproduction Service No. ED432588). Sundre, D. L. and Moore, D. L. (2002) The Student Opinion Scale: A measure of examinee motivation. Assessment Update, 14 (1) 8-9. Sundre, D. L., and Wise, S. L. (2003) ‘Motivation filtering’: An exploration of the impact of low examinee motivation on the psychometric quality of tests. Paper presented at the annual meeting of the National Council on Measurement in Education, Chicago, IL.

Author Information

Cambridge Assessment [University of Cambridge Local Examinations Syndicate]
Research Division
Cambridge
Cambridge Assessment (University of Cambridge Local Examinations Syndicate), United Kingdom

Update Modus of this Database

The current conference programme can be browsed in the conference management system (conftool) and, closer to the conference, in the conference app.
This database will be updated with the conference data after ECER. 

Search the ECER Programme

  • Search for keywords and phrases in "Text Search"
  • Restrict in which part of the abstracts to search in "Where to search"
  • Search for authors and in the respective field.
  • For planning your conference attendance, please use the conference app, which will be issued some weeks before the conference and the conference agenda provided in conftool.
  • If you are a session chair, best look up your chairing duties in the conference system (Conftool) or the app.