Conference:
ECER 2009
Format:
Symposium Paper
Session Information
09 SES 11 A, Rasch Measurement in Educational Contexts (Part 4)
Symposium. Continued from 09 SES 10 A
Time:
2009-09-30
16:45-18:15
Room:
HG, HS 50
Chair:
Pekka Antero Kupari
Discussant:
Wolfram Schulz
Contribution
The Systemic Evaluation project in South Africa was launched in response to the National Education Policy Act of 1996 with the purpose of evaluating and benchmarking learner performance. The project was first implemented in 2001 at Grade 3 level. Subsequently Grade 6 was also involved and in 2007 a second cycle of the Systemic Evaluation in both grades was conducted. Evidence exists that the information gathered through Systemic Evaluation did in fact inform policy. However, it is at the implementation stage that problems arise, necessitating interpretation and communication of the results to teachers and departmental officials.
These studies draw on the notion from Schmidt et al. (1996) that curriculum is central to the educational experience of children. The curriculum as perceived in these studies is manifested through the curriculum documents, textual resources and evaluations, made up by tests and contextual factors. Andrich (2002) makes the case for component analyses which together constitute higher order concepts.
Our objective in two provinces where the Centre for Evaluation and Assessment (CEA) was involved inter alia in analysing the Systemic Evaluation data and writing a report was to provide richer feedback information that could be used at all levels including the classroom. The necessity of providing qualitative descriptions at graded levels of proficiency let us to explore the question: How can feedback from quantitative results be enhanced through qualitative descriptions?
In the Systemic Evaluations we have test item responses which provide little information in raw form and have to be interpreted in terms of intended curriculum outcomes.
The Rasch measurement model, more specifically the dichotomous and partial credit models, as applied in the WINSTEPS software, provided us with a tool which present information at the system level which could be disaggregated to district, class and individual level. Through the logarithmic transformation of the Rasch model learner proficiency and item difficulties could be located on a common linear scale. Strategic points on this scale served as benchmarks around which items and learners were clustered. Through qualitative analyses of items, descriptions of attainment at given levels were possible, for example at the lowest Numeracy proficiency level, simple place value understanding was evident.
Method
Providing information for learners at graded levels of proficiency and items at graded levels of difficulty has the potential to inform role-players at different levels of the system (including teachers) and guide strategic implementation for the improvement at the micro-level (classroom) and macro-level (provincial structures) education. We are of the belief that this outcome might aid implementation of improved educational strategies.
The richness of the benchmark descriptions are dependent on the quality of the tests. The detailed feedback provided by the Rasch analysis potentially enable the improvement of the test, which we view as an additional benefit.
Expected Outcomes
References
Andrich, D. (2002). A Framework Relating Outcome Based Education and the Taxonomy of Education Objectives. Studies in Education Objectives, 28, 35-59. Republic of South Africa (RSA) (1996). National Education Policy Act., 27. Schmidt, W.H., McKnight C.C., Valverde, G.A., Houang, R.T. & Wiley, D.E. (1996). Many Visions, Many Aims (TIMSS Volume 1): A Cross-National Investigation of Curricular Intentions in School Mathematics. Dordrecht. Kluver Academic Publishers.
Search the ECER Programme
- Search for keywords and phrases in "Text Search"
- Restrict in which part of the abstracts to search in "Where to search"
- Search for authors and in the respective field.
- For planning your conference attendance you may want to use the conference app, which will be issued some weeks before the conference
- If you are a session chair, best look up your chairing duties in the conference system (Conftool) or the app.