Session Information
09 SES 12 B, Methodological Issues in Tests and Assessments
Paper Session
Contribution
Since the 1990s, there has been a growing interest in assessing not just curricular achievement but the more general cognitive and affective goals of education, mixing cognitive tasks and self-report questionnaires pertaining to diverse factors believed to indicate readiness for new learning and successful adaptation to the rapidly changing demands of the future (e.g., OECD PISA). Due to their non-curricular nature, such assessments are generally low-stakes at student level but are used for high stakes benchmarking at education policy level, with data dependent on students’ willingness to participate despite no personal ramification to boost or guarantee their effort (c.f. Barry, Horst, Finney, Brown & Kopp, 2010).
At the same time, computer-based testing (CBT) has gained popularity in education due to the many advantages it offers in data collection (Wang, Jiao, Young, Brooks & Olson, 2007; Wang, Hong, Young, Brooks & Olson, 2008). Wang et al. show in their meta-analyses regarding the impact of test administration mode (CBT vs. paper-and-pencil test PPT) in reading and mathematics assessment at grades K-12 that, overall, the administration mode did not have a statistically significant effect on students’ achievement scores (ibid.; see also Taherbhai, Dayerong & Bowman, 2001).
Building on earlier modelling of the impact of students’ prior knowledge (GPA), some key learning-related affective factors such as achievement goals (e.g., Elliot & Dweck, 1988; Harackiewicz, Barron, Pintrich, Elliot & Thrash, 2002) and means-ends-beliefs (e.g., Dweck, Chiu & Hong, 1995a, 1995b) on their performance in the cognitive tasks of a low-stakes assessment (Kupiainen, Marjanen, Vainikainen & Hautamäki, 2011, 2012), the present study focuses on the additional information CBT affords regarding the impact of the time a student focusses on the task on his/her performance.
The main research questions are:
1) What is the role of time-on-task on students’ performance in low-stakes cognitive tasks measuring reasoning skills, mathematical thinking skills and reading comprehension taking into account their prior knowledge (GPA) and some key affective characteristics as revealed in a simultaneously administered self-report questionnaire?
2) Is the role of time-on-task dependent on the cognitive task in question and/or students’ perception of task interest and task difficulty?
3) Are there gender differences in the role of time-on-task
As the time students concentrate on a specific task is one of the more easily observable aspects of learning in class, the results can be seen to help teachers’ understanding of when – or if – actual learning is happening in class and what factors might be related to its fostering.
Method
Expected Outcomes
References
Barry, C.L., Horst, S.J., Finney, S.J., Brown, A.R., & Kopp, J.P. (2010). Do examinees have similar test-taking effort? A high-stakes question for low-stakes testing. International Journal of Testing. Volume 10, Number 4 October-December, 2010. Dweck, C.S., Chiu, C. & Hong, Y. (1995a). Implicit theories and their role in judgements and reactions - a world from two perspectives. Psychological Inquiry,Vol. 6, No. 4, 267-285. Dweck, C.S., Chiu, C. & Hong, Y. (1995b). Implicit theories: Elaborations and extensions of the model. Psychological Inquiry,Vol. 6, No. 4, 322-333. Elliot, A.J. & Dweck, C.S. 1988. Goals: an approach to motivation and achievement. Journal of Personality and Social Psychology, Vol. 54, No. 1, 5-12. Harackiewicz, J.M., Barron, K.E., Pintrich, P.R., Elliot, A.J. & Thrash, T.M. 2002. Revision of achievement goal theory: necessary and illuminating. Journal of Educational Psychology, Vol. 94, No. 3, 638–645. DOI: 10.1037//0022-0663.94.3.638. Hautamäki J., Arinen P., Eronen S., Hautamäki A., Kupiainen S., Lindblom B., Niemivirta M., Pakaslahti L., Rantanen P. & Scheinin P. (2002). Assessing Learning-to-Learn. A Framework. National Board of Education, Evaluation 4/2002. Retrieved from http://www.oph.fi/download/47716_learning.pdf. Cole, J.S., Bergin, D.A. & Whittaker, T.A. (2008). Predicting student achievement for low stakes tests with effort and task value. Contemporary Educational Psychology, 33 (2008),609-624. Kupiainen, S., Marjanen, J., Vainikainen, M-P. & Hautamäki, J. (2011). Relations between Finnish students’ cognitive abilities, learning-related attitudes, and GPA at grades 3, 6 and 9. Paper presented at EARLI 2011, Exeter, England. Manuscript impending. Kupiainen, S., Marjanen, J., Vainikainen, M-P. & Hautamäki, J. (2012). Explaining achievement at school and in a low-stakes test. Cognitive, affective and contextual factors in 6th and 9th graders attainment. Paper presented at ECER 2012, Cadiz, Spain. Taherbhai, H., Dayerong, S. & Bowman, T. 2001. Comparison of paper-pencil and online performance of students with learning abilities.
Search the ECER Programme
- Search for keywords and phrases in "Text Search"
- Restrict in which part of the abstracts to search in "Where to search"
- Search for authors and in the respective field.
- For planning your conference attendance you may want to use the conference app, which will be issued some weeks before the conference
- If you are a session chair, best look up your chairing duties in the conference system (Conftool) or the app.