Conference:
ECER 2009
Format:
Symposium Paper
Session Information
09 SES 10 A, Rasch Measurement in Educational Contexts (Part 3)
Symposium. Continued from 09 SES 09 A, continued in 09 SES 11 A
Time:
2009-09-30
14:45-16:15
Room:
HG, HS 50
Chair:
Tjeerd Plomp
Discussant:
Eugenio Gonzalez
Contribution
The presentation shows the use of a generalized Rasch model in multidimensional adaptive testing (MAT) and illustrates the advantages of this approach regarding measurement efficiency compared to traditional tests with a fixed number of items in a fixed order (FIT).
Multidimensional adaptive testing is a procedure of simultaneously assessing multiple latent traits in which the selection of the test items that are presented to the examinee is based on the responses the examinee gave on previously administered items (e. g. Frey & Seitz, 2009). The aim of this selection procedure is to tailor the item presentation to the trait levels of the examinee. The multidimensional generalization of the Rasch model can be used as measurement model in MAT (e. g. Segall, 1996). Since in MAT more information can be derived from each response compared to FIT and unidimensional adaptive testing (CAT) this testing procedure can increase measurement efficiency substantially. Such a gain in measurement efficiency may be especially interesting for large-scale assessments where large samples are tested with a large number of items, resulting in long testing time and high costs. However, it is not yet known what gains in measurement efficiency can be precisely expected for typical large-scale assessments if MAT is used instead of FIT.
To examine the magnitude of possible gains in measurement efficiency a real data simulation was conducted using the responses of N = 9577 ninth graders who participated in a study assessing the attainment of the German educational standards in mathematics. Empirically obtained person parameters, item parameters, and the covariance structure between five measured dimensions, representing competencies in mathematical sub-domains were used to compare FIT, CAT and MAT regarding measurement efficiency.
Both adaptive procedures proved to be considerably more efficient than FIT. The measurement efficiency was 3.3 times higher for MAT and 1.9 times higher for CAT than for FIT. The result is even more promising because the used set of item parameters is not optimal for adaptive testing. Further gains in measurement efficiency are to be expected if a more optimal item pool is used. Thus, MAT could significantly decrease the testing load of large-scale assessments.
The results of the simulation study are discussed in the light of an operational use of MAT for large-scale assessments. Further, advantages of using the multidimensional Rasch model are addressed compared to other models from Item Response Theory like the multidimensional two parameter logistic test model.
Method
Expected Outcomes
References
Frey, A. & Seitz, N. N. (2009). Multidimensional Adaptive Testing in Educational and Psychological Measurement: Current State and Future Challenges. Manuscript submitted for publication. Segall, D. O. (1996). Multidimensional adaptive testing. Psychometrika, 61, 331-354.
Search the ECER Programme
- Search for keywords and phrases in "Text Search"
- Restrict in which part of the abstracts to search in "Where to search"
- Search for authors and in the respective field.
- For planning your conference attendance you may want to use the conference app, which will be issued some weeks before the conference
- If you are a session chair, best look up your chairing duties in the conference system (Conftool) or the app.