Session Information
09 SES 11 B, School and Class Composition Effects in Assessments
Paper Session
Contribution
The current trend of transferring national and international large-scale assessments on computer-based platforms gives new possibilities for evaluating students' effort in these assessments by utilizing log data. The most common way of doing this is analysing students' time investment in assessment tasks (e.g. Schnipke & Scrams, 1997; Wise & Kong, 2005). Recently, log file analyses of Finnish low-stakes assessment of cross-curricular learning to learn skills showed that time on task mediated the effects of 15-years-old students' detrimental learning-related attitudes, and it was a strong predictor of performance in the assessment. (Kupiainen, Vainikainen, Marjanen, & Hautamäki, in press, for the Finnish framework for learning to learn assessment see also Hautamäki et al, 2002).
In large-scale educational assessments it is typical to analyse between-school differences (e.g. OECD, 2013) and look at how much of variance is explained by systematic school-level effects. In countries with relatively small differences between schools much more of the variation is found within schools, but systematic class or peer group -level effects are only seldom analysed due to sampling prodecures and lack of information regarding social relationships within classes. This is despite of the fact that it has been understood for decades that peers have a influence on each others' performance at school (Hanushek, Kain, Markman & Rivkin; Ryan, 2012; Song & Grabowski, 2006; Wentzell and Caldwell, 1997)
The aim of the present study is to first replicate with 12-years-old students the results of Kupiainen et al (in press) regarding the mediating role of time on task between learning-related attitudes and performance in a low stakes learning to learn assessment. After that it is explored whether there are systematic school, class and peer group level effects on time investment. Before studying peer group effects a new method for defining approximations of peer groups in large-scale data without sociometric information is developed based on another data set with sociometric information available. It is expected the variation of time investment does not happen only at an individual level, but classes and peer groups have systematic effects too. This would be both because of peers affecting each others' attitudes towards the assessment situation, and also because in a school computer lab it is hard to entirely prevent th students from following each others' progress even if the order of tasks was randomized. Thus, the reseach questions are:
1. Does time on task mediate the effects of 12-years-old students learning-related attitudes on actual performance in a low-stakes assessment of cross-curricular learning to learn skills?
2. Are there systematic school, class and peer group level effects in explaining the variation of time on task?
Method
Expected Outcomes
References
Chapman, M., Skinner, E. A., & Baltes, P. B. (1990). Interpreting correlations between children's perceived control and cognitive performance: Control, agency or means-ends beliefs. Developmental Psychology, 26 (2), 246–253. Demetriou, A., & Kazi, S. (2006). Self-awareness in g (with processing efficiency and reasoning). Intelligence, 34, 297-317. Elliot, A. J., & Dweck, C. S. (1988). Goals: an approach to motivation and achievement. Journal of Personality and Social Psychology, 54 (1), 5–12. Harackiewicz, J. M., Barron, K. E., Pintrich, P. R., Elliot, A. J., & Thrash, T. M. (2002). Revision of achievement goal theory: necessary and illuminating. Journal of Educational Psychology, 94 (3), 638–645. doi: 10.1037//0022-0663.94.3.638 Hanushek, E.A., Kain, J.F., Markman, J.M., & Rivkin, S.G. (2003). Does peer ability affect student achievement? Journal of Applied Econometrics, 18, 527-544. Hautamäki J., Arinen P., Eronen S., Hautamäki A., Kupiainen S., Lindblom B., Niemivirta M., Pakaslahti L., Rantanen P., & Scheinin P. (2002). Assessing Learning-to-Learn. A Framework. National Board of Education, Evaluation 4/2002. Kupiainen, S., Vainikainen, M.P., Marjanen, J., & Hautamäki, J. (in press). The role of time on task in computer-based low stakes assessment of cross-curricular skills. Journal of Educational Psychology. Ryan, A.M. (2012). Peer relationships and adjustments at school: An introduction. In A.M Ryan & G.W. Ladd (eds.) Peer Relationships and Adjustment at School, pp. 1-10. Charlotte, NC: Information Age Publishing, Inc. Schnipke, D. L. & Scrams, D. J. (2002). Exploring issues of examinee behavior: Insights gained from response-time analyses. In C. N. Mills, M. T. Potenza, J. J. Fremer, & W. C. Ward (Eds.), Computer-based testing: Building the foundation for future assessments (pp. 237–266). Mahwah, NJ: Erlbaum. Song, H.-D., & Grabowski, B. (2006). Stimulating Instrict Motivation for Problem Solving Using Goal-Oriented Context and Peer Group Composition. Educational Technology Research and Development, 54 (5), 445–466. Wentzel, K.R. & Caldwell, K. (1997). Friendships, peer acceptance, and group membership: Relations to academic achievement in middle school. Child Development, 68(6), 1198-1209. Wise, S. L. & Kong, X. (2005). Response time effort: A new measure of examinee motivation in computer-based tests. Applied Measurement in Education, 18(2), 163–183. doi: 10.1207/s15324818ame1802_2
Search the ECER Programme
- Search for keywords and phrases in "Text Search"
- Restrict in which part of the abstracts to search in "Where to search"
- Search for authors and in the respective field.
- For planning your conference attendance you may want to use the conference app, which will be issued some weeks before the conference
- If you are a session chair, best look up your chairing duties in the conference system (Conftool) or the app.