09 SES 02 C, Issues in Test Development and Data Modeling
Parallel Paper Session
The interpretation of the results from different assessments of an educational system comes with the problem of the comparability of scores. This is why the information provided by autonomic, national or international assessment programs can, in some cases, show important discrepancies. Under these circumstances, the questions are: Are the results of these assessments comparable? Is it possible to improve their interpretation? How can we benefit from the information of these assessments? The usual answer implies an equating design involving anchor items. But when one of the assessments affects the whole population of students, it’s extremely hard to warrant the due security of the items used as anchor. So, is it possible to relate the scale of such an assessment program to the scale of a well known program as PISA, which is tested on a sample of the population?
In order to answer these questions this paper introduces a procedure for comparing results from different evaluations. More specifically, it has been proposed a model to compare the results of the Diagnostic Evaluation of Educational System, carried out in the Region of Madrid (Spain), with the information provided by the OECD Program for International Student Assessment (PISA).
According to the Organic Law of Education (2/2006, 3rd of may), the General Subdirection of Assessment and analysis of the Community of Madrid (Spain) (BOCM 31st of august) is responsible of the diffusion, the application and the correction of the tools in the Diagnostic Evaluation of this region and, for this purpose, it has been methodologically advised by the Education System Measurement and Assessment Group (Grupo MESE) of the Complutense University of Madrid. This assessment is intended to students of 4th year of Primary Education (EP) and 2nd year of Compulsory Secondary Education (ESO) of public, semi-private and private schools.
During the academic year of 2009/10, tests used in the program International Schools´ Assessment – ISA, designed and launched by the Australian Council for Educational Research (ACER) in 2001 - were adapted to the Spanish educational system (ESP-ISA tests). ISA is an international assessment program for students in schools over the world in Grades 3 to 10 (9 to 16 years old). It shares the theoretical framework of the PISA program. ESP-ISA tests were constructed from a pool of ISA items, linked to PISA tests. Additionally, ESP-ISA tests included released PISA items. This way, two sets of item parameters were involved, those from English version of ISA items, and those from the Spanish version of released PISA items. Ideally, fixing these parameters would allow to estimate the score for each individual based on their answers to the whole test. But to refine the process, a procedure was implemented to select, as fixing items only those showing invariance when a free calibration took place.
Baker, F. B. (1992). Item response theory : Parameter estimation techniques. New York: M. Dekker. Decreto 118/2007, de 2 de agosto, del Consejo de Gobierno de la Comunidad de Madrid, por el que se establece la estructura orgánica de la Consejería de Educación. BOCM núm. 207 (31 de agosto de 2007), 4-10. Embretson, S. E. (Ed.). (2000). Item response theory for psychologists. Mahwah, New Jersey: Lawrence Erlbaum associates. Hambleton, R. K. (Ed.). (1991). Fundamentals of item response theory. Newbury Park, California: Sage. Kolen, M. J., & Brennan, R. L. (1995). Test equating: Methods and practices. New York: Springer-Verlag. Ley Orgánica 2/2006, de 3 de mayo, de Educación. BOE num. 106 (4 de mayo 2006) 17158-17207. Rasch, G. (1960). Probabilistic Models for Some Intelligence and Attainment Tests. Copenhagen: Danish Institute for Educational Research. Wim J.van der Linden, Ronald K.Hambleton (Ed.). (1997). Handbook of modern item response theory. New York: Springer. Wu, M. L., Adams, R. J., Wilson, M. R., & Haldane, S. A. (2007). ACERConQuest Version 2.0: generalised item response modelling software. Victoria: ACER Press.
00. Central Events (Keynotes, EERA-Panel, EERJ Round Table, Invited Sessions)
Network 1. Continuing Professional Development: Learning for Individuals, Leaders, and Organisations
Network 2. Vocational Education and Training (VETNET)
Network 3. Curriculum Innovation
Network 4. Inclusive Education
Network 5. Children and Youth at Risk and Urban Education
Network 6. Open Learning: Media, Environments and Cultures
Network 7. Social Justice and Intercultural Education
Network 8. Research on Health Education
Network 9. Assessment, Evaluation, Testing and Measurement
Network 10. Teacher Education Research
Network 11. Educational Effectiveness and Quality Assurance
Network 12. LISnet - Library and Information Science Network
Network 13. Philosophy of Education
Network 14. Communities, Families and Schooling in Educational Research
Network 15. Research Partnerships in Education
Network 16. ICT in Education and Training
Network 17. Histories of Education
Network 18. Research in Sport Pedagogy
Network 19. Ethnography
Network 20. Research in Innovative Intercultural Learning Environments
Network 22. Research in Higher Education
Network 23. Policy Studies and Politics of Education
Network 24. Mathematics Education Research
Network 25. Research on Children's Rights in Education
Network 26. Educational Leadership
Network 27. Didactics – Learning and Teaching
The programme is updated regularly (each day in the morning)
- Search for keywords and phrases in "Text Search"
- Restrict in which part of the abstracts to search in "Where to search"
- Search for authors and in the respective field.
- For planning your conference attendance you may want to use the conference app, which will be issued some weeks before the conference
- If you are a session chair, best look up your chairing duties in the conference system (Conftool) or the app.