Examining The Effect Of Technology Enhanced Formative Assessment On Student Engagement And Learning
Author(s):
Colin Cooney (presenting / submitting)
Conference:
ECER 2016
Format:
Paper

Session Information

ERG SES H 11, ICT and Education

Paper Session

Time:
2016-08-23
11:00-12:30
Room:
OB-E2.14
Chair:
Lázaro Moreno Herrera

Contribution

This paper describes the initial stages of a PhD research project, which investigates the use of technology enhanced formative assessment to improve student engagement and learning for large class sizes. Specifically, the research question will investigate the effect of structured, on-going formative assessments, in the form of online quizzes, on a set of my own students within the School of Business Studies and Humanities, in Dundalk Institute of Technology. 

The paper outlines the main challenges and benefits of designing these online quizzes and the feedback mechanisms to improve them as part of planned, iterative action research cycles. The research explores issues such as quiz and questionnaire design in terms of user experience, user engagement, cognitive load theory and learner analytics. 

The High Level Group on the Modernisation of Higher Education (2014) highlight how the higher education landscape is undergoing significant change as a result of technological innovations. A major recommendation of the report is that the integration of digital technologies and pedagogies should form an integral element of higher education institutions’ strategies for teaching and learning. Furthermore, it is increasingly accepted that assessment is the engine that drives a great deal of students’ learning. (Brown, Race, & Bull, 1999). Assessment for learning (AFL) is a conscious attempt to make assessment a productive part of the learning process. It does this by making classroom assessment an essential part of effective teaching and learning.” (Stobart, 2008). This author believes that formative assessment should constitute part of the learning process, but for many reasons, had not engaged in such assessment in the past. 

The research will also describe the recent developments in on-screen question types, where up until recently, there were only a very limited number of question types available. The recent development of new ‘Technology Enhanced Items’ and question types mean that assessments can better assess lower and higher order learning. This leads to the main research question; does the use of this new technology enable enhanced learning outcomes through enhanced assessment? Like (Kirkwood & Price (2014), this author will explore how will ‘enhancement’ be achieved, and how can an ‘enhancement’ be determined?

The research approach is action research within a case study context, using the mixed methods methodology for data gathering and analysis. 

In summary, the research aims to gauge if there was any effect on the following areas, because of the series of planned interventions:

  •  Students’ engagement in the module content
  •  Students’ standard of learning 
  •  Students’ learning style

Method

The research approach is action research within a case study context, using the mixed methods methodology for data gathering and analysis. Qualitative methods will be employed after an initial and on-going understanding of the students is explored using some quantitative methods. This is the rationale behind adopting the mixed methods approach for this research. This is supported by Creswell (2014) when he states that the assumption is that collecting diverse types of data will provide a more complete understanding of the research problem than either quantitative of qualitative data alone. Specifically, the methods used for data gathering are questionnaires (online and paper-based), focus groups (independently-led and researcher-led), online learner analytics and reflective journaling. The first cycle of this research took place from September to December 2015. The second phase will take place during Semester 2, from January to June 2016. Ethical issues are heightened, as the researcher is an active participant in his own research. Issues like power bias, informed consent, confidentiality, anonymity, potential risks and adherence to professional codes of ethics all needed to be comprehensively described and addressed before the research began. It is tantamount to the success of this research that students do not feel any pressure to participate in the research and also that their level of participation would have any influence on their final exam grade. Every attempt will be made to deal with these power / ethical issues before, during and after the research takes place. Research should be designed and undertaken with ethical considerations in mind, thus ensuring the integrity and quality of all that is done. The ethical issues and the procedures put in place to demonstrate the rigour in the research process, to instil trust in the students and ensure valid and reliable data is collected were vital to the research and to secure ethical approval.

Expected Outcomes

As mentioned above, the first cycle of this action research took place from September to December 2015. The second phase will take place during Semester 2, from January to June 2016. Analysis of the data collected has been taking place from the start of the first research cycle. This on-going analysis, coupled with a comprehensive analysis at the end of the process, means that there will be several outcomes to present at ECER 2016. The analysis incorporated tools such as SPSS, Max QDA, spreadsheet modelling and online learning analytics. Preliminary findings indicate a low level of engagement with the quizzes during the Semester with a sizeable spike in activity directly prior to the final exam. Consequently, several new engagement tactics will be explored in Semester 2 to increase engagement from the start of term. The research aims to improve the learner experience of a specific set of students. However, on a higher level, the research will critically examine the role of technology in assessment, the role of assessment in learning and also gain insights into the limitations of technology and formative assessment in that process. It will examine the role of the student voice in designing formative assessments and investigate the level of learner engagement and how to incentive this engagement. It is hoped that this research will influence syllabi design and inform learning and assessment strategies in Dundalk IT, but also inform the national and international conversation on technology enhanced learning.

References

Assessment Reform Group, 2002. Assessment for Learning: 10 principles, University of Cambridge: Faculty of Education, UK: University of Cambridge. Barnes, K., Marateo, R. & Ferris, S., 2007. Teaching and learning with the net generation. Innovate: Journal of Online Education, 3(4). Black, P. & Wiliam, D., 1998. Inside the Black Box : Raising Standards Through Classroom Assessment. Phi Delta Kappan, 80, pp.139–148. Brown, S., Race, P. & Bull, J., 1999. Computer Assisted Assessment in Higher Education 1st ed., London: Kogan Page. Carr, W. & Kemmis, S., 1986. Becoming critical: Education, knowledge, and action research, London; Philadelphia: Falmer Press. Cohen, L., Manion, L. & Morrison, K., 2011. Research Methods in Education 7th ed., Routledge. Creswell, J.W., 2013. Research Design: Qualitative; Quantitative; and Mixed Methods Approaches. In Research Design. SAGE Publications, pp. 3–23. Dunn, L. et al., 2004. The Student Assessment Handbook: New Directions in Traditional and Online Assessment, Routledge. Elliott, J., 1991. Action Research for Educational Change 1st ed., Open University Press. Falchikov, N., 2005. Improving Assessment through Student Involvement: Practical solutions for aiding learning in higher and further education, London: RouteledgeFarmer. Hattie, J. & Timperley, H., 2007. The Power of Feedback. Review of Educational Research , 77 (1 ), pp.81–112. JISC, 2009. Effective Practice in a Digital Age: A guide to technology-enhanced learning and teaching, Bristol, England. Kincheloe, J., 2003. Teachers as Researchers: Qualitative inquiry as a path to empowerment 2nd ed., Routledge. Kirkwood, A., & Price, L. (2014). Technology-enhanced learning and teaching in higher education: what is “enhanced” and how do we know? A critical literature review. Learning, Media and Technology, 39(1), 6–36. McAleese, M., & Vassiliou, A. (2014). High Level Group on the Modernisation of Higher Education. Luxembourg: Publications Office of the European Union. McNiff, J. & Whitehead, J., 2011. All You Need To Know About Action Research 2nd ed., SAGE Publications. Nicol, D. & Macfarlane-Dick, D., 2006. Formative assessment and self-regulated learning: a model and seven principles of good feedback practice. Studies in Higher Education, 31(2), pp.199–218. Oblinger, D. & Oblinger, J., 2008. Educating the next generation. Science & justice : Journal of the Forensic Science Society, 48(4), p.196. Prensky, M., 2008a. The Role of Technology in teaching and the classroom. Educational Technology, (Nov-Dec), pp.1–3. Sadler, D.R., 1998a. Formative assessment and the design of instructional systems. Instructional Science, 5, p.52909. Stobart, G., 2008. Testing times: the uses and abuses of assessment 1st ed., Oxon: Routledge.

Author Information

Colin Cooney (presenting / submitting)
Dundalk Institute of Technology
Business Studies
Dundalk

Update Modus of this Database

The current conference programme can be browsed in the conference management system (conftool) and, closer to the conference, in the conference app.
This database will be updated with the conference data after ECER. 

Search the ECER Programme

  • Search for keywords and phrases in "Text Search"
  • Restrict in which part of the abstracts to search in "Where to search"
  • Search for authors and in the respective field.
  • For planning your conference attendance, please use the conference app, which will be issued some weeks before the conference and the conference agenda provided in conftool.
  • If you are a session chair, best look up your chairing duties in the conference system (Conftool) or the app.