TIMSS (Trends in International Mathematics and Science Study) is one of the most influential international studies. The large number of publications have been conducted to investigate attitudes based on the results of TIMSS applications. This can be considered as an indication of the growing international interest in differentiation of attitudes. One of the affective factors having significant effect on mathematics achievement is attitude toward mathematics (Farooq & Shah, 2008). It comprises individual’s liking for mathematics, participation in activities related to mathematics and views concerning the usefulness of mathematics and their own achievement (Neale, 1969). TIMSS applications measure the three psychological constructs which are “students’ liking for mathematics”, “students’ self-confidence in learning mathematics”, and “students’ valuing mathematics” (Kadijevich, 2006).  

Self-confidence in mathematics is the level of self-esteem and self-perceptions of students’ concerning their competence in learning mathematics (Demir & Kılıç, 2010). Liking for mathematics is related to students’ interest in mathematics and their internal motivation (Mullis, Martin, Foy & Arora, 2012). Valuing mathematics concerns the external motivation of individuals and reflects their attitudes toward usefulness and importance of mathematics in various stages of their life (Wigfield & Eccles, 2000). In brief, the common point of these attitude-related concepts is that they all have an important role in students’ mathematics achievement. Therefore, it is important to focus on attitude-related concepts to obtain an understanding of the differentiation in mathematics achievement of students in different countries.

In the literature, comparative studies investigating the relationship between attitude and achievement have reported conflicting results. Some researchers (Marsh, Trautwein, Lüdtke, Köller & Baumert, 2005) indicated that students with a high level of achievement tended to hold positive attitudes toward mathematics while others (Van de Gaer & Adams, 2010)  found that these students had negative attitudes despite their high achievement. The direction of the relationship between these variables varies according to the investigation being conducted at an individual or group level. In this regard, attitude-achievement paradox is the relationship between attitude and achievement being positive at the individual level but negative at the group level (Van de gaer, Grisay, Schulz & Gebhardt, 2012).

One of the factors causing attitude-achievement paradox is the response style (Buckley, 2009). It is the tendency to respond systematically to items of a questionnaire regardless of their content (Paulhus, 1991). The response style leads to various psychometric problems which threatens validity of the test scores (Bolt & Newton, 2011). In the literature, there are many response style types. One of them is extreme response style (ERS).  It is the tendency to respond using the extreme endpoints of the scale (Baumgartner & Steenkamp, 2001).

 In this study, the research questions were formulated as follows:

For countries that participated in the TIMSS 2015,

1. How do the two models, one including ERS and the other excluding ERS as a dimension, fit the data related to the responses of the students to the attitude scales?
2. How does the relationship between mathematics achievement and unadjusted and ERS-adjusted attitude scores vary at the country-level?

This research is valuable because it provides a model-based approach to adjust the effect of response styles, rather than post hoc comparisons of differences in response styles. Additionally, the current study contributes to the understanding of the relationship between attitude and achievement at the country level and valid interpretation of these relationships by going beyond theories that only offer information at the individual level

The sample of this correlational research composed of eighth-grade students who were participated in TIMSS 2015 and selected by two-stage stratified sampling method (LaRoche, Joncas & Foy, 2016). In order to better represent the pattern of the relationship between attitude and achievement, a cluster analysis was performed. Accordingly, at the country level a negative correlation between mathematics achievement and attitude toward mathematics was found. Moreover, three groups of countries were found; the first group had high mathematics achievement but low attitude scores, the second group had moderate mathematics achievement and attitude scores, and the last group had low mathematics achievement but high attitude scores. Countries from each cluster were selected. Accordingly, five countries were selected from each of these groups. They were Singapore, Korea, Chinese, Hong-Kong, Japon, Sweden, Italy, Malta, Australia, Norway, Turkey, Chile, Kuwait, Egypt, and Saudi Arabia. For the selected countries, the relationship between mathematics achievement and attitudes toward mathematics at the individual level was found to be positive. This confirms the presence of the attitude-achievement paradox in the selected countries. In order to reduce the complexity of analyses, 500 students from each of these countries were selected randomly. In the current study, student questionnaire and mathematics achievement test were used as the data collection instruments. Student responses to these instruments were modelled by using MIRT due to the limitations of the other methods used to determine ERS. It considers response styles as a psychological dimension affecting the responses of an individual and analyzes it together with the effect of the trait measured (Bolt & Johnson, 2009). Two different models were developed in this study. The first model (the 3-dimensional model) contained the three attitude-related constructs. In the second model (the 4-dimensional model), in addition to these dimensions, ERS was included as a separate dimension. The items and individual parameters related to the attitude dimensions and ERS were estimated using the 3- and 4-dimensional models to obtain the students’ unadjusted and adjusted attitude scores, respectively. Model-fit indices and item parameters were obtained using Latent Gold software and individual parameters were obtained by WinBUGS. Also, the correlation between students’ mathematics achievement and their attitudes was determined by using observed scores and scores based on MIRT models.

The first finding of this study was that the model-fit index values estimated by the 4-dimensional model were lower than those obtained using the 3-dimensional model. It indicates that the former model better fits the data. Thus, ERS had an effect on the students’ responses. Furthermore, students’ attitude scores being more positive or negative was found to be influenced by their ERS level. At the country level, negative correlations between the attitude of the students and mathematics achievement were found by using students’ observed scores. When these correlations were estimated by using the 3-dimensional model, their direction moved in the positive direction and the amount of correlation was reduced, and when the effect of ERS was included in the model (4-dimensional model), there were positive correlations at similar levels. This result indicates that controlling the ERS variable has a role in changing the direction of negative correlations toward the positive, but on its own, it is not sufficient to explain the attitude-achievement paradox. This may have been caused by other response styles of the students (ARS, DRS, etc.), which affect the responses of individuals, thus the correlation between attitude and achievement. Another factor that may have influenced the relationship between mathematics achievement and attitude toward mathematics is the big-fish-little-pond effect.

Baumgartner, H. & Steenkamp, J. E.M. (2001). Response styles in marketing research: A cross-national investigation. Journal of Marketing Research, 38, 143-156. Bolt, D. M., & Newton, J. (2011). Multiscale measurement of extreme response style. Educational and Psychological Measurement, 71, 814-833. Bolt, D. M., & Johnson, T. R. (2009). Addressing score bias and differential item functioning due to individual differences in response style. Applied Psychological Measurement, 33, 335-352. Buckley, J. (2009). Cross-national response styles in international educational assessment: Evidence from PISA 2006. NCES Conference on the Program for International Student Assessment: What we can learn from. Retrieved from https://edsurvey.rti.org/PISA/ Demir, İ. & Kılıç, S. (2010). Öğrencilerin matematik başarısına etkileyen faktörlerin PISA 2003 kullanılarak incelenmesi. Hacettepe Üniversitesi Eğitim Fakültesi Dergisi, 38, 44-54. Farooq, M. S. & Shah, S. Z. U. (2008). Students’ attitude toward Mathematics. Pakistan Economic and Social Review, 48(1), 75-83. Kadijevich, D.J. (2006). Developing trustworthy TIMSS background measures: A case study on mathematics attitude. The Teaching of Mathematics, 9 (2), 41-51. LaRoche, S., Joncas, M., & Foy, P. (2016). Sample Design in TIMSS 2015. In M. O. Martin, I. V. Mullis, & M. Hooper (Eds.), Methods and Procedures in TIMSS 2015 (pp. 3.1-3.37). Retrieved from Boston College, TIMSS & PIRLS International Study Center website: http://timss.bc.edu/publications/timss/2015-methods/chapter-3.html Marsh, H. W., Trautwein, U., Lüdtke, O., Köller, O & Baumert, J. (2005). Academic self-concept, interest, grades and standardized test scores: Reciprocal effects models of causal ordering. Child Development, 76(2), 397-416. Mullis, I. V., Martin, M. O., Foy, P. & Arora, A. (2012). TIMSS 2011 international results in mathematics. Chestnut Hill, MA: TIMSS & PIRLS International Study Center, Boston College. Neale, D. (1969). The role of attitudes in learning mathematics. The Arithmetic Teacher, 16, 631-641. Paulhus, D. L. (1991). Measurement and control of response bias. In J. P. Robinson, P. R. Shaver & L. S. Wrightman (Eds.), Measures of Personality and Social Psychological Attitudes. San Diego, CA: Academic Press. Van de gaer, E., Grisay, A., Schulz, W. & Gebhardt, E. (2012). The reference group effect an explanation of the paradoxical relationship between academic achievement and self-confidence across countries. Journal of Cross-Cultural Psychology, 43(8), 1205-1228. Van de Gaer, E. & Adams,R.(2010). The Modeling of Response Style Bias: An Answer to the Attitude-Achievement Paradox?, Paper presented at annual conference of the American Educational Research Association, Colorado, USA, 30 April – 4 May. Wigfield, A. & Eccles, J. S. (2000). Expectancy-value theory of motivation. Contemporary Educational Psychology, 25, 68-81.