Fold-back: Using Technology to Provide an Expansive Account of Education

Author(s):

Simon N. Leonard(presenting / submitting)Robert N. Fitzgerald(presenting)

Conference:

ECER 2015

Network:

16. ICT in Education and Training

Format:

Paper

Session Information

16 SES 11, Innovative Learning Environments

Paper Session

Time:

2015-09-10

17:15-18:45

Room:

3005. [Main]

Chair:

Johan van Braak

Contribution

This paper reports on progress on an ongoing design-based research project developing methods of technology assisted developmental evaluation (TADE) at the institutional level that provide greater value for teaching and learning than current predominant business analytics models. New technologies are capable of drawing together and presenting diverse information sources about the learning occurring with educational environments that are providing the capacity to move beyond an over reliance on simplistic and isolated metrics and, arguably, arbitrary measures of quality. The nascent field of learning analytics is already demonstrating that new technologies can greatly assist in small and large scale evaluation within educational environments (Siemens, 2014), although much of the early work in this field has been deployed with a Quality Assurance (QA) or Business Analytics epistemology which tends to have a “service delivery” focus (Shum & Ferguson, 2011).

The concept of fold-back developed in this project is borrowed form music amplification where fold-back is used to provide an appropriate mix of information to aid the performance of each performer in the form of music from the entire band. It is offered as a way of describing the capacity emerging technologies provide to similarly provide an appropriate mix of information for different people within educational contexts.

The fold-back project is about more than computer code. As design-based research, the project builds upon a well articulated theoretical understanding of learning that draws together key concepts from cultural-historical theory (Vygotsky, 1978), Activity Theory (Engestrom, 2001), Variation Theory (Marton & Tsui, 2004) and the concept of Communities of Practice (Lave & Wenger, 1991).

This paper will present initial design prototypes of how emerging technologies may be used to realise the capacity to provide a more expansive account of learning. The prototypes make use of the Experience Application Programing Interface, also known as the Experience API or xAPI (Advanced Distributed Learning, 2014); and the mixed methods software solution Dedoose (Dedoose, 2014). The project continues a growing interest in the interplay of design and evaluation in various aspect of educational technology in recent years (Burrows & Shortis, 2011; Cheung & Hew, 2011; Kay & Knaack, 2008; Phillips, Kennedy, & McNaught, 2012).

Method

Design-based research (DBR) is an approach that seeks to increase the impact and transferability of educational research (Anderson & Shattuck, 2012). It stresses the need for theory building alongside the development of design principles that direct and improve practice and research. This paper documents the ongoing development of prototypes of a feedback and evaluation “engine” similar in concept to the Assessment Tools for Teaching and Learning (asTTle) project led by John Hattie (2006). That is, a system that can provide teachers and institutions with data that aids educational decision making in a practical and accessible form. Data on the design progress has been kept through the project including design notes, planning papers, and observations and interviews with co-participants using the models. These data have been used to analyse the capacity of the protype designs to build an account of the experiences, understandings and skills being built in complex educational environments. The analysis is based on an evaluation framework was devised in the early stages of the project (Authors, in press).

Expected Outcomes

The project is working towards the development of a data-driven information system that can be deployed in traditional and new forms of educational engagement and outreach work. Such applications demand a robust and reliable environment that can be used to meet the needs of both development and production. The creation of a production ready xAPI platform demanded a sophisticated HTML5 Standard dashboard module that incorporated a pluggable reporting/analytics interface. The pluggable nature would enable the interface to be embedded into other web interfaces such as Moodle LMS or a Wordpress CMS. It is anticipated the project will develop systems for accounting for the activities of educational institutions and the learning that goes on within them that will provide superior information for both institutional and professional decision making. Future work will need to consider usability and cost-benefit of doing so. The current work is limited to modelling the capabilities of new technologies.

References

Anderson, T., & Shattuck, J. (2012). Design-based research: A decade of progress in education research? Educational Researcher, 41(1), 16-25. doi: 10.3102/0013189X11428813 Burrows, S., & Shortis, M. (2011). An evaluation of semi-automated, collaborative marking and feedback systems: Academic staff perspectives. Australasian Journal of Educational Technology, 27(7), 1135-1154. Cheung, W. S., & Hew, K. F. (2011). Design and evaluation of two blended learning approaches: Lessons learned. Australasian Journal of Educational Technology, 27(8), 1319-1337. Engestrom, Y. (2001). Expansive Learning at Work: Toward an activity theoretical reconceptualisation. Journal of Education and Work, 14(1), 133-155. doi: 10.1080/13639080020028747 Hattie, J. (2006). Large-scale Assessment of Student Competencies. Paper presented at the Working in Today’s World of Testing and Measurement: Required Knowledge and Skills - 26th International Congress of Applied Psychology, Athens, Greece. http://www.jisc.ac.uk/media/documents/projects/asttle_casestudy.pdf Kay, R. H., & Knaack, L. (2008). A multi-component model for assessing learning objects: The learning object evaluation metric (LOEM). Australasian Journal of Educational Technology, 24(5), 574-591. Marton, F., & Tsui, A. B. M. (2004). Classroom discourse and the space of learning. Mahwah, NJ: Lawrence Erlbaum Associates. Phillips, R., Kennedy, G., & McNaught, C. (2012). The role of theory in learning technology evaluation research. Australasian Journal of Educational Technology, 28(7), 1103-1118. Shum, S. B., & Ferguson, R. (2011). Social Learning Analytics. Available as: Technical Report KMI-11-01 Retrieved from http://kmi.open.ac.uk/publications/pdf/kmi-11-01.pdf Siemens, G. (2014). The Journal of Learning Analytics: Supporting and Promoting Learning Analytics Research. Journal of Learning Analytics, 1(1), 3-5.

Author Information

Simon N. Leonard (presenting / submitting)

University of Canberra, Australia

Robert N. Fitzgerald (presenting)