Since the introduction and proliferation of learning management systems in K-12 schools, as part of the digital transformation in education, a huge amount of data (learning analytics) has become generally available for (re)designing and evaluating education, and for evidence-informed quality assurance (Brown & Malin, 2022). This quality assurance (QA) has become increasingly decentralized in many European countries, making schools responsible and accountable for their own quality. Schools have therefore been developing their own procedures and exploring their responsibilities in the context of QA, but the use of learning analytics (LA) data regarding learning processes often remains un(der)explored.

Up till now, LA have primarily proven their potential for QA in the context of higher education. In secondary schools, they are currently mainly used at the micro level, where they are being used by (individual) teachers to identify and tailor to learners’ specific needs. However, the potential use of LA at the school (management) level, or the ways in which schools or school teams can make optimal use of these data, remains an open and under-explored question (Ifenthaler, 2021) for researchers, practitioners and policy makers alike (Gašević et al. 2016). Moreover, there is a lack of capacity to work with these data for strategic planning and quality development in schools. Teaching staff, school leaders and middle managers are often wondering how to start off with learning analytics data in this regard, and often invoke questions from an ethical and privacy perspective. Next to these general questions, there is also the issue of competencies. Even if school staff would know how these data could inform them, there is a general lack of competencies and know-how on how to get started (Ifenthaler, Mah & Yin-Kim Yau, 2019).

The QUALAS (Quality Assurance with Learning Analytics in Schools) project therefore aims to build on the available knowledge on QA and LA to identify possibilities for enhancing the capacity of educational professionals in secondary schools in Flanders (Belgium), Ireland, Italy and Spain to make appropriate use of learning analytics for quality assurance. In order to achieve these aims, we first want to identify how LA and QA are currently being coupled and put into practice in secondary education in these different jurisdictions. Additionally, we investigate how the approaches in two of these jurisdictions (Flanders and Ireland) draw on and relate to policy and initiatives of QA and LA at a European level.

We start from European conceptualisations of quality assurance, including definitions provided by the European Commission (2018). We further rely on the distinctions drawn between external and internal evaluation, and between government-based and market-based accountability in education (Eurydice, 2015). Concerning LA, we draw on a publication by the European Commission’s Joint Research Centre on the use of learning analytics and its action list for educational stakeholders (Ferguson et al., 2016), and on the European Union’s Digital Education Action Plan. These definitions and conceptualisations are subsequently compared to national (or regional) policy texts and other grey literature concerning QA and LA, in order to answer the following research questions:

1. How is QA defined and linked to LA in the different jurisdictions?
2. How do schools deal with learning analytics data? What are LA used for?
3. What strategies for capacity building in the use of LA (for QA) in schools exist/are successful?
4. What learning management systems are most common/popular in secondary schools and why?

In this contribution, we present our findings from a grey literature analysis conducted in the educational jurisdictions of Flanders and Ireland, and situate these in relation to the European policy and research documents sketched above. This grey literature review was conducted as part of the first phase of the Erasmus + ka cooperation project QUALAS, a cooperation between Vrije Universiteit Brussel (Belgium), Dublin City University (Ireland), Universidad de Valladolid (Spain) and Instituto nazionale per la valutazione del sistema educativo di instruzione e di formazione (Italy). This first phase consisted of a rapid narrative systematic review of the existing literature on LA and its connection(s) to quality assurance in secondary education and schools. Grey literature is generally defined as: “that which is produced on all levels of government, academics, business and industry in print and electronic formats, but which is not controlled by commercial publishers.” (Paez, 2017). In our case(s), it includes: (practice-oriented) academic publications and vulgarizing texts, government reports and policy documents, and documents of LA and learning management system providers. This grey literature was incorporated based on the belief that it can make important contributions to a systematic review, because it can provide resources and data that cannot be found within commercially published (academic) literature and can thus help avoid potential (publication) bias (Paez, 2017). Given our focus on national education system contexts, our (grey) search strategy did not include consulting (international) grey literature databases. However, conference proceedings stemming from the results from our systematic search in scientific databases (Scopus, ScienceDirect, Web of Science and EBSCOhost) with a focus on the European context or the respective jurisdictions of Flanders, Ireland, Italy and Spain were included as grey literature items. Additionally, we conducted web searches looking for specific documents, reports and other publications on LA and QA in these jurisdictions, which were conducted in the jurisdictions’ respective languages. All four partner institutions conducted the analysis of grey literature found for their own jurisdictions. A template was provided by the project coordinator to ensure the reliability and validity of the analyses. The analysis protocol focused on: • Definition(s) of QA • Types and functions of LA • Level(s) of use of LA • Data use and LA within QA • Bibliometric info (year, type of publication, authors, target audience, etc.)

Including grey literature in our systematic literature review on learning analytics for quality assurance in secondary education in four European countries, and in relation to policies and conceptualisations at a European level, provided us with significant insights additional to those provided by the systematic (scientific) literature review. In this contribution, we focus on our findings in the Flemish and Irish jurisdictions, and link these to the European level. First, we will sketch the main findings for both the Flemish and Irish educational contexts: their approaches to QA, current use of LA in secondary education, and existing links between QA and LA in both jurisdictions. We then identify and discuss four main themes, arising from a comparison between the Flemish and Irish contexts based on the grey literature found in both jurisdictions and their relations to the European context: 1) LA and its relation to QA as expressions of digital optimism and the push for (post-covid) educational digitalisation for national and European recovery and resilience: tackling societal challenges through (digital) education and LA 2) LA for QA: focus on personalisation and differentiation in education 3) Digital education as covering two dimensions (REF): the educational use of digital technologies and devices (including LA) and the digital competence and professional development of educators 4) Reluctancy and fears connected to LA and QA, or educational push-back: the threat of hollowing out education and the teaching profession

Brown, C. & Malin, J.R. (eds). (2022). The Emerald Handbook of Evidence-Informed Practice in Education. Emerald Publishing Limited. European Commission. (2018). Quality assurance for school development. Guiding principles for policy development on quality assurance in school education. Retrieved from: https://education.ec.europa.eu/sites/default/files/document-library-docs/2018-wgs2-quality-assurance-school_en.pdf Eurydice. (2015). Assuring Quality in Education: Policies and Approaches to School Evaluation in Europe. Retrieved from: https://op.europa.eu/en/publication-detail/-/publication/4a2443a7-7bac-11e5-9fae-01aa75ed71a1/language-en Ferguson, R., Brasher, A., Clow, D., Cooper, A., Hillaire, G., Mittelmeier, J., Rienties, B., Ullmann, T. & Vuorikari, R. (2016). Research Evidence on the Use of Learning Analytics - Implications for Education Policy. In Vuorikari, R. & Castaño Muñoz, J. (Eds.). Joint Research Centre Science for Policy Report. doi:10.2791/955210. Gašević, D., Dawson, S. and Pardo, A. (2016). “How do we start? State and directions of learning analytics adoption”. International Council for Open and Distance Education. Ifenthaler, D. (2021). Learning analytics for school and system management. In OECD Digital Education Outlook 2021. Pushing the Frontiers with Artificial Intelligence, Blockchain and Robots. OECD Library. 161. Ifenthaler, D., Mah, D-K. & Yau, J.Y. (2019). Utilizing Learning Analytics to Support Study Success. Springer Cham. DOI 10.1007/978-3-319-64792-0 Paez, A. (2017). Gray literature: An important resource in systematic reviews. Journal of Evidence-Based Medicine,10(3), 153-240.