Comparisons between Partial Credit and Rating Scale Rasch models: Perception Scale of Child Well-being

Author(s):

Enrique Navarro(presenting / submitting)Eva Expósito Casas(presenting)Esther López Bianca Thoilliez

Conference:

ECER 2014

Network:

09. Assessment, Evaluation, Testing and Measurement

Format:

Paper

Session Information

09 SES 03 C, Methodological Issues in Tests and Assessments

Paper Session

Time:

2014-09-02

17:15-18:45

Room:

B009 Anfiteatro

Chair:

Jan-Eric Gustafsson

Contribution

The application of Item Response Theory (IRT in advance) has been increasing on psychometric research. Nevertheless the use with the measurement of sociological and psychological variables is not that common and the major limitations of IRT is in the special case of the measurement instrument with polytomous items (Asún & Zuñiga, 2008), this is our case. These are measurement instruments related with variables as attitudes, motivations, interest, etc. where the difference between response categories is the intensity of agreement with the feature presented in the item.

The main objective of the present research is show the advantages and limitations of the use of Rash polytomous models, specifically the Partial credit and the Rating scale models, in the validation of Perception Scale of Child Well-being. In one hand the IRT provides advantages over Classical Test Theory like the invariance of the parameters of items in relation with the sample or the test independent estimation of latent scores. In the other hand obtaining indicators to analyze the living conditions of children is a growing concern, select the best methodological choice in order to validate a Perception Scale of Child Well-being is an important educational research goal, transferable to another areas of social, political and psychological concerns.

In order to obtain this target, we compare forth polytomous IRT model. We use the simple logistic model of Rasch for the model estimation, the differences between these four models is twofold:

The procedure to estimate the probabilities of individuals answers in the different item categories: Partial Credit Model (PCM) (Masters, 1982) / Rating Scale Model (RSM) (Andrich, 1978).
Dimensionality of the latent structure of measurement instrument: Multidimensional five category model/ Unidimensional model.

The combination of these two features leads to the four models analyzed in the present research: a) PCM- Unidimensional; b) PCM-Multidimensional; c) RSM-Unidimensional; d) RSM-Multidimensional.

This study seeks the contrast between these models through the validation of the PSCWI (Perception Scale of Child well-being Indicators)instrument. Different national and international agencies and institutions have directed considerable efforts to obtain accurate measurements of the specific characteristics of child well-being. At the same time, the views of future education professionals about the determinants of child well-being and its impact on the lives of children help closer to this reality and understanding it more deeply. The instrument PSCWI has been made to know what are the determinants of child well-being recognized by education professionals trainees, as well as the importance degree given to each of them. Pick out the best option to validate the EPIBI instrument is essential in order to obtain a good certainty of the functioning of this.

Method

The PSCWI test consists in 80 Likert items with six response categories scale (1=Not important/6=Essential). Factor analyzes conducted in previous studies confirm a underlying five dimension structure (Material wellbeing, Health and Safety, Education wellbeing, Relations with the Environment and Subjective wellbeing) (Thoilliez, et al, 2010; Exposito-Casas, et al, 2010) and the value of Cronbach's alpha was found to be .95 which can be considered an excellent level of reliability (George & Mallery, 1995). The final sample is composed by 805 students of the different degrees related with the educational area belonging to three universities of the Community of Madrid (Spain). The instrument was administered in electronic and paper/pencil format in a control conditions. The four models have been developed using the program ACER-Conquest 2.0 (Wu & Adams, 1997). In order to compare the functioning of this four models it has been used the index of global adjust (deviance), where the lower values show the better fit. In order to check if to models differ statistically we estimate the difference (Δ) between the values of this index in the two contrasted models. This difference follows a chi square distribution (χ2) with freedom degrees equal to the difference between the number of the estimated parameters in the models. If the probability is less than 0.05 the models can be considered different and the low deviance indicates the best model. In order to check the fit of the items, we employed statistics based on the waste as a difference between the empirical answer and the expected probabilities. Has been estimate the quadratic means of this residuals whose distribution approaches to chi square distribution (χ2) and whose mathematical expectation is equal to one. This parameter value suggests a good fit of a particular item in the model proposed. Furthermore, the parameter value is shown in a normal distribution as a T test (values above of 2 shows poor fit). In the results this statistic is shown in two ways: MNSQ (weighted)/ MNSQ (unweighted).

Expected Outcomes

The results show a best fitting o the multidimensional partial credit model (PCM-Multidimensional). At the same time has been detected a good global performance of the instrument. Moreover, poor fitting index were founded in the extreme categories of some items. The theoretical structure based on five dimensions of content have a better adjust, in other words, the multidimensional models in both case (Partial Credit and Rating Scale) shown a better adjust than the unidimensional ones. The complexity of measurement sociological and psychological variables entails the use of complex models, in this sense, the use of IRT allow us to obtain a major certainty in the process of the validation of this kind of instruments. The Partial Credit Model, recon the possible difference between the categories in a Likert scale, and estimate one difficulty parameter in each step inside the item, this parameter can be different between the items. A lot of studies in this area don´t use this kink of models but the use of Classical Test Theory or an inadequate IRT model can be produce misleading conclusions. Establish criteria to judge the fit of the different models is essential in the validation process of measurement instrument, in this sense, the present study show a properly way to select the best choice in the selection of models. Finally, we obtain a good fit of items in the almost all the categories most of them are in the confidence interval. However some items have problems in the extreme categories of answer (0-6) with bad results in both indexes. Improve the characteristics of measurement instrument is possible taking in to account this index, the revision of the worse functioning items is a great way to enhance the properties of a measurement instrument.

References

Abad, F. J., Ponsoda, V., & Revuelta, J. (2006). Modelos Politómicos de Respuesta al Ítem. Madrid: La Muralla. Ackerman, T.A., Gierl, M.J., y Walker, C.M. (2003) Using Multidimensional Item Response Theory to Evaluate Educational and Psychological Tests. NCME. Recuperado el 30/09/2012 de: http://ncme.org/linkservid/80F7088B-1320-5CAE- 6EFAFFB5DE15D512/showMeta/0/ Andrich, D. (1978). A rating formulation for ordered response categories. Psychometrika, 43(4), 561-573. Asún, R., & Zuñiga, C. (2008). Ventajas de los modelos politómicos de Teoría de la Respuesta al Ítem en la medición de actitudes sociales. Estudio de un caso. Psykhe, 17(2), 103-115. Baker, J. A. (1998). The social context of school satisfaction among urban, low-income, African-American students. School Psychology Quarterly, 13, 25-44. Ben-Arieh, A. (2010). From Child Welfare to Children Well-Being: The Child Indicators Perspective. En S. B. Kamerman, S. Phipps & A. Ben-Arieh (Eds.), From Child Welfare to Child Well-Being.An International Perspective in the Service of Policy Making (pp. 9-22). New York: Springer. Bradshaw, J. & Richardson, D. (2009). An index of child well-being in Europe. Child Indicators Research, 2, 319-351. Expósito-Casas, E., López-Martín, E., Navarro-Asencio, E. y Thoilliez, B. (2010). Estudio piloto sobre los determinantes en el bienestar infantil desde la perspectiva de los futuros profesionales de la educación. Comunicación presentada en el IV Congreso Iberoamericano de Pedagogía, 7-11 Septiembre, Toluca, México. Martínez, M. R., Hernández, M. J., & Hernández, M. V. (2006). Psicometría. Madrid: Alianza Editorial. Masters, G. N. (1982). A Rasch Model for Partial Credit Scoring. Psychometrika , 47(2), 149-174. Prieto, G., & Delgado, A. R. (2003). Análisis de un test mediante el modelo de Rasch. Psicothema , 15(1), 94-100. Rasch, G. (1960). Probabilistic Models for Some Intelligence and Attainment Tests. Copenhagen: Danish Institute for Educational Research. Rees, G., Bradshaw, J., Goswami, H. & Keung, A. (2009).Understanding Children’s Well-Being: A national survey of young people’s well-being. London: The Children’s Society. Samejima, F. (1969). Estimation of Latent Ability Using a Response Pattern of Graded Scores. Psychometric Monograph, 34 (17), 1-100 Thoilliez, B. (2011). How to Grow up Happy: An Exploratory Study on the Meaning of Happiness from Children’s Voices. Child Indicators Research, 4 (2), 323-351. UNICEF (2007). Pobreza infantil en perspectiva: Un panorama del bienestar infantil en los países ricos. Innocenti Report Card 7. Florencia: Centro de Investigaciones Innocenti de UNICEF. Wu, M. L., Adams, R. J., Wilson, M. R., & Haldane, S. A. (2007). ACER ConQuest Version 2.0: generalised item response modelling software. Victoria: ACER Press.

Author Information

Enrique Navarro (presenting / submitting)

Universidad Internacional de la Rioja (UNIR)

Education

Madrid

Eva Expósito Casas (presenting)

Universidad Nacional de Educación a Distancia

Métodos de Investigación y Diagnóstico en Educación

MADRID

Esther López

Universidad Nacional de Educación a Distancia (UNED), Spain

Bianca Thoilliez

Universidad Europea de Madrid (UEM), Spain

Search the ECER Programme

Search for keywords and phrases in "Text Search"
Restrict in which part of the abstracts to search in "Where to search"
Search for authors and in the respective field.
For planning your conference attendance you may want to use the conference app, which will be issued some weeks before the conference
If you are a session chair, best look up your chairing duties in the conference system (Conftool) or the app.