In this study, the main aim is to investigate the issue of validity of negatively worded items based on data from PIRLS 2006. Item wording is crucial for any kind of analysis of survey data and for a long time, measurement experts have suggested to utilize both positively and negatively worded items to measure the same construct (e.g. Nunnally & Bernstein, 1994). Reworded items should, for example, prevent acquiescent behavior and make respondents more focused on closely reading the questionnaire. However, the recommendation to use a mixed item format including both positively and negatively worded items is challenged by research findings. Several studies reveal an effect of item wording on descriptive statistics, internal consistence, factor structure and finally, validity (e.g. Barnette, 2000, Hazlett-Stevens, Ullman, & Craske, 2004). Dimensionality analysis often reveals separate factors reflecting the positive and the negative items, respectively (e.g. Marsh, 1996). If respondents interpret negatively worded items differently, how can the differences be explained? Previous research that addressed this issue revealed variables that affect the way respondents deal with reworded items. For example, age, level of education, carelessness when reading (e.g. ignoring the word ‘not’) as well as insufficient cognitive abilities can be relevant factors to influence the responses (Weems, Onwuegbuzie, & Collins, 2006). In this context, it is of particular interest whether reading ability is related to the way participants interpret negatively worded items. Williams and Swanson (2001) found support for such relation for adults (n=48). Weems, Onwuegbuzie and Collins (2006) for a sample of graduate students (n=153), and Marsh (1984) for fifth grade students (n=559). The main purpose of the current study is to examine the impact of item wording on students’ responses in PIRLS 2006. The study is based on international data from the IEA study PIRLS 2006 for students in grade 4. Achievement data and data from the student background questionnaire were included in the study. First, some scales with positively and negatively worded items were built to latent variables through confirmatory factor analysis (CFA) in order to detect the underlying response-finding processes. Second, the correlations between the latent variables that represent the negatively worded items are examined, and third, the relation to reading ability is studied. - --continues in Methodology---

---Preliminary results of our study indicate that some students do not recognize certain negatively worded items, and this is consistent with previous research. Additionally, positive correlations between negatively worded item factors indicate a global construct, which is related to these factors (e.g. carelessness). Finally, the way students respond to mixed worded items is substantially related to the students’ reading scores. A first tentative conclusion to be drawn from our analyses is that it is problematic for the students to respond correctly to negatively worded items. Students with higher reading achievement scores, however, handle the negatively worded items better than low-performing students. With regard to item validity, this constitutes a challenging problem to be kept in mind for instance when developing questionnaires for large-scale studies such as PIRLS.