For education practitioners who design and develop learning experiences available via an online environment, it is possible for them to examine and reflect upon the process of teaching and learning by employing what may be described as tools for learning analytics. However, it has been argued that the available teaching/learning analytical toolset and support that is available to the practitioners is inadequate (Wong & Li, 2020). Viewed as the main obstacle to the development of the toolset is the absence of the design framework/theory (Mangaroska & Giannakos, 2018). In order to productively investigate the usefulness of the data generated for the purpose of analysing learning it is necessary to focus the main interpreter of such data, namely, the teacher (Hernández-Leo et al., 2019). Furthermore, there emerges a requirement for teachers to be developed (motivated and supported) as learning-data interpreters. Our epistemological hypothesis is that our work will move the teacher from a state of functional pedagogical literacy towards that of critical pedagogical literacy. We take our inspiration from Laurillard’s (2013) notion of teaching as design science, and we problematise this notion in terms of how it can be brought about - i.e. if teaching is indeed a design science than how can we develop teachers as designers.

The research we are undertaking is aimed at deductively interrogating the current use of the existing analytical toolset and the potential for expanding this use by inductively framing the process with a particular design-theoretical approach based on the meta-pedagogical theoretical construct called ‘Conversational Framework’ (Laurillard, 2001). This is a meta-pedagogical framework within which any approach can be described, explored and critically analysed. As such, it offers a pedagogically agnostic perspective and an appropriate, organising principle basis for our research purpose.

Complementing this organising principle is the means by which we aim to generate developmental momentum, that is to say, the facilitation of the growth of teachers’ analytical expertise- these fall within the province of the theoretical perspective of the Zone of Proximal Development (Vygotsky, 1978). Our aim is to arrive at a workable solution to enable the development of an adaptive learning design/analytics support for the teachers. Our approach is distinct from other work undertaken in this area as our focus is upon the means for evolving the interpretive and analytical skill of the teaching practitioners; and not merely designing the model, built on the assumption of practice and expecting the teachers to learn to use it. Essentially, we focus upon a dynamic ‘growth’ model instead of a static model which teachers are expected to ‘grow into’.

This research seeks to answer the following questions;

1. How can we diagnose the levels of teachers’ critical literacy in terms of their own pedagogical practice?

2. What scaffolding strategies can motivate the growth from the ‘diagnosed state’ (of critical literacy)?

Design Based Research Methodology is the principle investigatory means for this project which is undertaken in two phases, the first of which forms the focus of this paper. In the first phase, we investigate the limits of the existing learning analytics design through the experiences of a small group of Higher Education (HE) lecturers. This will involve exploration of their use of the current Learning Analytics toolset and of the support available to them through the Virtual Learning Environment (in this instance Canvas) in one HE institution within England. Through a series of semi-structured, one-hour interviews, we aim to explore the current expectations, opportunities, issues, barriers etc. to the greater utilisation of the existing, and the better definition of the expansion of the learning-analytical and learning-interpretative toolset and support. These questions will direct the participants’ attention to: a) what they look at/for in the analytics; b) how they make sense/interpret the data; c) how this may/may not affect/inform their actions in terms of instructional design; and d) whether they check the impact of the decisions by re-examining the analytics. The same process is repeated for all of the data views available through the ‘dashboard’. Finally, the participants’ overall reflections on their experiences and use of analytics will be gathered. Participants are also encouraged to identify the opportunities and the challenges (if any) that they encounter. Data generated from the interviews will be thematically analysed.

We anticipate that the model arising from this research will offer a tentative means by which teachers will be encouraged and empowered to grow their own interpretative, analytical and design expertise. Another expectation is that, as we progress with our research, there will be greater role for machine performed Learning Analytics, and Design Analytics in the mix; and that this will complement the overall development.

- Hernández-Leo, D., Martinez-Maldonado, R., Pardo, A., Muñoz-Cristóbal, J. A., & Rodríguez-Triana, M. J. (2019). Analytics for learning design: A layered framework and tools. British Journal of Educational Technology, 50(1), 139–152. - Joseph, D. (2004). The practice of design-based research: Uncovering the interplay between design, research, and the real-world context. Educational psychologist, 39(4), 235-242. - Laurillard, D. (2002). Rethinking University Teaching: A Conversational Framework for the Effective Use of Learning Technologies (2nd ed.). RoutledgeFalmer. - Laurillard, D. (2013). Teaching as a design science: Building pedagogical patterns for learning and technology. Routledge. - Mangaroska, K., & Giannakos, M. (2018). Learning analytics for learning design: A systematic literature review of analytics-driven design to enhance learning. IEEE Transactions on Learning Technologies, 12(4), 516–534. - Vygotsky, L. S. (1978). Mind in Society: The Development of Higher Psychological Processes. Harvard University Press. - Wong, B. T., & Li, K. C. (2020). A review of learning analytics intervention in higher education (2011–2018). Journal of Computers in Education, 7(1), 7–28.