Session Information
09 SES 04 B, Assessment: Methods and Applications III
Paper Session
Contribution
Purpose of the present study is to investigate distribution of aberrant response patterns across gender, school type, and graduation status (at last grade of high school, recently graduated, etc.) using person-response curves (PRC).
In multiple-choice tests, unexpected behaviors of examinees are considered to be a threat for validity. These behaviors of responses – also called aberrant responses – may stem from cheating, guessing, fatigue or careless, etc. of examinees. For example, if a low-ability examinee answers the hardest items in a test correctly, this can be used a trigger for suspicion of guessing or cheating. Another examinee may give wrong answers to the easiest items in a test and this may indicate a careless person.
One of the methods to investigate individual aberrancy is called person-response curve and proposed by Trabin and Weiss (1983). PRC is an Item Response Theory (IRT) based methodology and relates probability to answer correctly to a group of items with item difficulty. In this approach, discrepancy between observed and expected PRCs is investigated by using a chi-square statistic. Observed curves can be obtained by calculating proportions of correct answer in each strata. To construct expected PRC, items are ordered according to their difficulty levels and groups of items called “strata” are formed. Expected PRC provides a norm to make comparisons with observed curve. Then a statistical test was used to detect discrepancy between observed and expected PRCs. Though there are many indices that are used to detect aberrant response patterns, principle advantage of PRCs is that they provide a visual way to interpret aberrancy.
Comparative investigation of aberrant response patterns across subgroups may yield significant information about differences on examinees’ individual behaviors in testing environments.
Method
Expected Outcomes
References
Nering, M. L., & Meijer, R. R. (1998). A comparison of the person response function and the lz person-fit statistic. Applied Psychological Measurement, 22, 53-69. Meijer, R. R. & Sijtsma, K. (2001). Methodology Review: Evaluating Person Fit. Applied Psychological Measurement 2001; 25; 107-135 Meijer, R. R.,&Sijtsma, K. (1995). Detection of aberrant item score patterns: A review of recent developments. Applied Measurement in Education, 8, 261-272. Trabin, T. E.,&Weiss, D. J. (1983). The person response curve: Fit of individuals to item characteristic curve models. In D. J. Weiss (Ed.), New horizons in testing: Latent trait test theory and computerized adaptive testing (pp. 83-108). New York: Academic Press.
Search the ECER Programme
- Search for keywords and phrases in "Text Search"
- Restrict in which part of the abstracts to search in "Where to search"
- Search for authors and in the respective field.
- For planning your conference attendance you may want to use the conference app, which will be issued some weeks before the conference
- If you are a session chair, best look up your chairing duties in the conference system (Conftool) or the app.