Collaborative Assessment Alliance: Using Technology to support innovative, collaborative assessment.
Author(s):
Ben Murray (presenting / submitting) Dr. Kevin Marshall (presenting)
Conference:
ECER 2016
Format:
Paper

Session Information

09 SES 11 C, Methodological Issues in Tests and Assessments

Paper Session

Time:
2016-08-25
17:15-18:45
Room:
NM-F107
Chair:
Jan-Eric Gustafsson

Contribution

Significant changes in society require learners to have a wide, adaptive knowledge base and understanding to enable them to be active participants in the communities in which they live and work. The premium in today's world is not merely on students’ acquiring information, but on their ability to analyse, synthesise, and apply what they have learned to address new problems, design solutions, collaborate effectively, and communicate persuasively (Pellegrino, 2014). Recent education reforms aspire to embed key competences in teaching and learning through rich learning outcomes. What is less clear is how existing assessment methods can properly evaluate skills such as critical thinking, problem solving, creativity, communication and collaboration.

No single assessment can evaluate all kinds of learning, rather, a coordinated system of assessment is needed that incorporates the assessment of higher order skills, includes real world skills of collaboration and communication, and engages students in instructionally valuable activities (Darling-Hammond et al., 2013). They argue that students need to engage in tasks that measure these complex skills and not evaluate a proxy for these skills. If assessment is about the process of identifying, gathering and interpreting information about student learning, it is necessary to consider what is assessed, how the assessment takes place and the purpose of the assessment (Gee and Shaffer, 2010). 

Jenkins et al. (2006), acknowledge the digital and participatory worlds that young people need to negotiate. However, student engagement within these worlds is often blurred by the notion of the student as a ‘digital native’ and by technical and technological approaches in schools that replicate traditional methods of assessment and instruction as opposed to embracing new ones (Claxton, 2007). In this traditional format, assessment tends to be associated with institutions and sanctioned assessors, whereas Gee (2010) argues that it has a natural home in human action and learning. This human action now includes interaction with technology and in his thoughts on Actor Network Theory, Latour (2005), places objects and non-human entities on an equal footing, and states that technology and social practices are inextricably linked. Lakhana (2014) agrees with this notion that we cannot separate technology from its social relations, as people are co-constructors of knowledge. The increasing influence of digital worlds means that young people are seen to be taking on new participatory and collaborative roles in learning online and outside the classroom, and there is a growing interest in incorporating these roles and practices inside education.  

This notion of the social and collaborative context of assessment is explored currently through the Collaborative Assessment Alliance project (http://www.caa21.org/), where students are assessed on their ability to collaborate on social and cognitive domains through the medium of online synchronous collaborative tasks. The concept behind the Collaborative Assessment Alliance is to extend the research and outcomes of the ATC21S project, particularly in the area of design, creation and deployment of locally relevant collaborative assessment tasks. The ATC21S project investigated methods whereby large-scale assessment of collaborative problem solving could be undertaken and technology used to collect and interpret the data using learning progressions (Griffin et al, 2012). The CAA initiative aligns strongly with the European DIGCOMP framework for developing and understanding digital competence (Ferrari, 2013). This framework identifies and details the key components of digital competences, develops descriptors at three proficiency levels and proposes a road map for their implementation. Ireland, through the National Council for Curriculum and Assessment (NCCA) is one of the partners in this international alliance alongside Sweden and Australia. This paper will explore the work of the Alliance so far, with a focus on the research in Irish schools. 

Method

Educational design-based research was considered the most appropriate methodology for this research, as this approach addresses complex problems in real contexts with researchers and practitioners striving to find solutions in collaboration with each other. Design-based research involves the negotiation of the research goals between the researcher and the practitioner; the two parties are partners in the process. Following their collaborative research, a design for the learning environment is proposed. This should then follow with and result in the development of theory and theoretical frameworks. After some rigorous and reflective inquiry, following the testing of these solutions, some guiding design principles are generated (Amiel and Reeves, 2008). Amiel and Reeves mention that too often, educational technology is not always researched in the education domain, as the benefits of a new tool are often researched elsewhere. However, this design-based approach allows for the investigation of particular cases in specific schools, working with teachers and students, while fostering teacher agency in the process, as teachers are co-constructors of theory and design principles. Currently, thirteen schools from around Ireland are developing digital synchronous collaborative tasks that challenge students to solve problems through collaborating with their partner. The teachers in these schools have explored the research and theories on collaborative assessment, and their implication for classroom and school practice. In designing their tasks, the teachers have considered how the content and the context of the lesson relates to development of skills and how conceptual and metacognitive knowledge is built. They have further considered how the collected assessment data can be evaluated using a progression framework that identifies the student’s participation and collaboration in the task on both cognitive and social domains. The skills that are targeted in the tasks align closely with the key skills of junior cycle and senior cycle education in Ireland. The tasks are unique in that they provide a window into how students collaborate, and how they approach collaboration; how they set their learning agenda. The professional development that teachers have received as a result of participation in the project has encouraged them to consider the role that digital learning has in assessment of not only the cognitive ability of students but also their social and collaborative abilities.

Expected Outcomes

Following trialling by the member schools, the assessment tasks will be disseminated amongst teachers and students across the country and exchanged between other members of the CAA. The tasks will also be shared with other local alliances, e.g. via CIDREE expert meetings and conferences. The results from the three countries will serve to further build local, national and international capacity in designing and implementing collaborative tasks that can become part of assessment systems. The indicative outcomes of this research include capacity building in the understanding and implementing of digital assessment, something underexplored in the Irish context. The initiative has resulted in the development of expertise in collaborative problem solving and performance assessment, an approach that is better suited to measuring higher order skills (Pecheone et al., 2010). A model for teacher professional development in assessment of collaboration and higher order skills has been developed. There is now research evidence to support the design of tasks that validly assess collaboration and problem solving. One unexpected outcome was the impact on the affective domain of students, particularly in the area of well-being and self-awareness, something that Crook (2011) refers to as been under researched in this digital collaborative domain. These outcomes will inform the work on digital assessment generally; more specifically they will significantly add to the body of knowledge around assessment of skills in the affective domain, and in authentic collaboration. This offers a way of assessing something that up to now was not possible to assess, even though these key skills underpin the curriculum reform curriculum at post-primary level in Ireland. Furthermore, there is strong evidence that multiple modalities of assessment are necessary when learning outcomes are complex and non-linear (Daugherty et al., 2012), and this ensures that the big ideas of these learning outcomes are not lost (Harlen, 2010).

References

Amiel, T., & Reeves, T. C. (2008). Design-Based Research and Educational Technology: Rethinking Technology and the Research Agenda. Educational Technology & Society, 11 (4), 29–40. Claxton, G. (2007) 'Expanding young people's capacity to learn'. British Journal of Educational Studies, 55 (2), pp. 115-134. Crook, C. (2011). Versions of computer supported collaborating in higher education. In Ludvigsen, S., Lund, A., Rasmussen, I. and Saljo, R. Learning Across Sites: New Tools, infrastructures and practices. New York, Taylor and Francis. Darling-Hammond, L., Herman, J., Pellegrino, J., (2013). Criteria for high-quality assessment. Stanford, CA: Stanford Center for Opportunity Policy in Education. Daugherty, R. et al. (2012). Alternative Perspectives on Learning Outcomes: Challenges for Assessment. In Gardner, J. (Ed), Assessment and Learning. London, Sage. Ferrari, A. (2013). DIGCOMP: A Framework for Developing and Understanding Digital Competence in Europe. Publications Office of the European Union, Luxembourg. Gee (2010). Human Action and Social Groups as the natural home of assessment: Thoughts on 21st Century Learning and Assessment. In Shute, V. and Becker, B. (Eds.) Innovative Assessment for the 21st Century. (pp. 13 – 39). Springer, New York. Gee, J.P. and Shaffer, D.W. (2010). Looking where the light is bad: Video games and the future of assessment. Edge: The Latest Information for the educatino practitioner, 6(1) pp. 3 – 19. Griffin, P., Care, E. and McGaw, B. (2012). The Changing Role of Education and Schools. In Griffin, P., Care, E., and McGaw, B. (eds.). Assessment and Teaching of 21st Century Skills. New York, Springer. Harlen, W. (2010). Principles and big ideas of science education. Ashford Colour Press Ltd., Gosport, Hants. Jenkins, H., Clinton, K., Purushotma, R., Robison, A.J. and Weigel, M. (2006) Confronting the Challenges of Participatory Culture: Media Education for the 21st Century. Available from: http://digitallearning.macfound.org/atf/cf/%7B7E45C7E0-A3E0-4B89-AC9C-E807E1B0AE4E%7D/JENKINS_WHITE_PAPER.PDF Lakhana, A. (2014). What is Educational Technology? An Inquiry into the Meaning, use and Reciprocity of Technology. Canadian Journal of Learning and Technology. 40 (3) Latour, B. (2005). Reassembling the Social. Oxford: Oxford University Press. Pecheone, R., Kahl, S., Hamma, J., Jaquith, A. (2010). Through a Looking Glass: Lessons Learned and Future Directions for Performance Assessment. Stanford, CA: Stanford University. Pellegrino, J. W. (2014). Assessment as a positive influence on 21st century teaching and learning: A systems approach to progress. Keynote address in Proceedings of the 2014 Conference of the International Association for Educational Assessment, Singapore

Author Information

Ben Murray (presenting / submitting)
NCCA
NCCA
Dublin
Dr. Kevin Marshall (presenting)
Microsoft, Ireland

Update Modus of this Database

The current conference programme can be browsed in the conference management system (conftool) and, closer to the conference, in the conference app.
This database will be updated with the conference data after ECER. 

Search the ECER Programme

  • Search for keywords and phrases in "Text Search"
  • Restrict in which part of the abstracts to search in "Where to search"
  • Search for authors and in the respective field.
  • For planning your conference attendance, please use the conference app, which will be issued some weeks before the conference and the conference agenda provided in conftool.
  • If you are a session chair, best look up your chairing duties in the conference system (Conftool) or the app.