Mathematics Items in TIMSS – a Proposal for Didactic Analysis of Student Responses.

Author(s):

Peter Allerup(presenting / submitting)

Conference:

ECER 2012

Network:

24. Mathematics Education Research

Format:

Paper

Session Information

24 SES 07, Achievement and Performance in Mathematics Education

Parallel Paper Session
Chair: laura Velasco

Time:

2012-09-19

17:15-18:45

Room:

ESI 3 - Aula 2

Chair:

Contribution

The math items in TIMSS are presented to the students either as Multiple Choice MC formatted questions (with prefixed response categories), as Short Answers (leaving space for at short answer e.g in terms of a number) or as Extended Responses (asking students to provide detailed reasons for their response).

The usual way of analyzing TIMSS responses involves the use of student scores, viz. the number of correct responses or as latent Rasch scores. This leads to well known normative assessments and comparisons among the students and, eventually by inclusion of various background variables be the starting point for analyzing contrasts and creating aspects of predictive validity of the scores.

Some information in respect of students thoughts and justification for their responses during the process of solving mathematical tasks, is available through analyzing the specific outcomes of incorrect responses. The fact that a specific distractor is chosen in MC, a specific wrong answer given to either short or extended format is used, may provide useful knowledge regarding the didactic basis available to the student. Considering all item responses in the TIMSS booklet answered by the student may further increase the chances of revealing information regarding the presence of a specific student strategy.

Method

In TIMSS 2007 and 2011 the majority of the Danish 4’th grade students participated in both ordinary TIMSS sessions and, afterwards, in a two hour special session in which the students solved a different set of mathematical problems, the so called extra TIMSS session. These items are all developed (Freudenthal Institute, The Netherlands) in an extended response format and students were specifically encouraged to write as many details as possible about their thoughts during the problem solving process. A comprehensive coding system was developed prior to the construction of the items in order to catch various aspects of student problem solving strategies beyond the fact, that the response was correct or incorrect. When data from extra TIMSS are merged with ordinary TIMSS data, a first problem is to see, if extra TIMSS items, properly scored as correct/non correct constitute a mathematics scale of the same psychometric nature as the ordinary TIMSS scale. Rasch Models are used for this analysis with the intention of applying the same IRT techniques as used earlier for the calibration of the ordinary TIMSS scale.

Expected Outcomes

It turns out, that the extra TIMSS items constitute a scale and this scale is, in fact parallel to the ordinary TIMSS scale in the sense, that all extra TIMSS items can be considered as a simple extension of the original set of ordinary TIMSS items. This psychometric property of the extra TIMSS items provide an excellent opportunity for extending the interpretation, specified by the detailed coding system meant for the extra TIMSS items to be a valid source for analyzing the didactic identification of the ordinary TIMSS items, either it be a wrong or correct answer. The paper demonstrates the steps of the psychometric scale analysis of extra TIMSS items and discusses the analytic condition, if the analysis would have resulted in a scale being not parallel to the ordinary TIMSS scale. Further, the paper demonstrates how aspects of student problem strategies can be distilled from the extra TIMSS responses and how hypotheses as to the didactic background for answering ordinary TIMSS items can be created. The paper will focus on the ways such analysis can be useful for mathematics teachers, when TIMSS responses are going to be used for formative evaluations of the students

References

Allerup, P.: Test Equating using IRT models (proc. 7’th round table conference on Assessment, Canberra November 2002) Allerup, P. : In Jurgen Rost, Rolf Langeheine (eds.): Applications of Latent Trait and Latent Class Models in he Social Sciences” Statistical Analyses of data from the IEA Reading Literacy Study” Waxmann 1997. Allerup, P.: The IEA Study of Reading Literacy. In: Owen, P. & Pumfrey, P. (eds.): Children Learning to Read: International Concerns, Vol. 2, p. 186-297, 1995. Allerup, P. : Rasch Measurement, theory of. The International Encyclopedia of Education, second edition,Pergamon Press. (1994)

Author Information

Peter Allerup (presenting / submitting)

Aarhus University, DPU

Copenhagen

Search the ECER Programme

Search for keywords and phrases in "Text Search"
Restrict in which part of the abstracts to search in "Where to search"
Search for authors and in the respective field.
For planning your conference attendance you may want to use the conference app, which will be issued some weeks before the conference
If you are a session chair, best look up your chairing duties in the conference system (Conftool) or the app.