Student performance in Flanders has shown a steady decline, according to both national and international comparative studies (e.g. Dockx et al., 2023; Faddar et al., 2020; Flemish Government, 2025; OECD, 2023). In response, the Flemish government has implemented strategic measures for systemic quality enhancement, including the recent establishment of quality labels for both professional development initiatives and educational learning materials. This study focuses specifically on the development of these quality labels for learning materials. 

The Flemish educational landscape offers a wide array of learning materials designed to support pedagogical-didactic processes and enhance student learning. The term “learning materials” encompasses a broad range of resources, including (digital) textbooks, workbooks, teaching methods, and teacher manuals (Bellens et al., 2022). In daily practice, teachers and instructional teams rely heavily on these resources for both subject-specific content and pedagogical-didactic guidance. Within the framework of Flanders’ principle of educational freedom, developers have the autonomy to translate attainment targets and curriculum objectives into learning materials. However, there remains limited insight into the quality of these learning materials, their development processes, or their actual implementation in practice (Bellens et al., 2022; Flemish Government, 2025). 

To address this, in April 2025 the Flemish government mandated Leerpunt, an independent evidence-informed organization, to evaluate learning materials within compulsory education, providing a grant of €3.49 million for the period 28 April 2025 to 30 June 2028. This project aims to provide essential support in the development, selection, and implementation of educational resources. This paper contribution presents the results of the first part of the project, which tested both a comprehensive quality framework and a formal evaluation procedure for learning materials. Leerpunt’s strategy for quality enhancement follows two distinct tracks:  

1. A self-evaluation tool for learning materials developed by teachers and instructional teams. 

2. A formal quality label for learning materials produced by (commercial) providers. 

Although both tracks are being developed in parallel, this paper contribution focuses specifically on the second track: the formal quality label for learning materials produced by providers. 

Following a large-scale study, providers will be able to officially submit their learning materials to Leerpunt for quality assessment, starting in February 2026. The formal evaluation begins with a strategic focus on Mathematics and Dutch in primary education. Approved learning materials will also be published on Leerpunt’s website. Throughout this process, Leerpunt adopts a development-oriented perspective, prioritizing constructive feedback and continuous improvement of materials over mere compliance.  

The primary aim of the first part of the project was to develop and test a robust, standardized, and evidence-based mechanism for evaluating learning materials, with the goal of positively contributing to educational quality. This aim is divided into two central research objectives: 

RO1: The development and testing of a comprehensive quality framework across all levels of compulsory education to ensure a valid assessment of learning materials. 

RO2: The development and testing of an effective evaluation procedure (assessment protocol) to ensure a reliable and objective evaluation process. 

In a first phase, an expert committee established a scientifically grounded quality framework for the systematic screening of learning materials, including main categories, indicators and items, and developed an evaluation procedure for this purpose in collaboration with Leerpunt. In developing this framework, the expert committee built extensively upon the work of the “Kwaliteitsalliantie”, which had previously conducted a comprehensive literature review on learning materials and produced an initial version of the framework for the secondary education (Bellens et al., 2022; De Man, 2022). Based on this solid theoretical foundation, an expert committee developed a generic quality framework to map the didactic quality of learning materials across all levels of compulsory education. 

The quality framework and assessment procedure was evaluated through a phased mixed-methods research design, combining qualitative and quantitative approaches. First, in August 2025, eight focus groups were conducted (n = 53) to assess the usability and applicability of the quality framework and the evaluation procedure for screening learning materials (Patton, 2002). The focus groups followed a standardized protocol for practical organization (Morgan & Krueger, 1998). The focus groups were audio- and video-recorded and fully transcribed. The qualitative data were then analyzed using both vertical and horizontal analyses. Vertical analyses resulted in anonymized reports for each focus group, with themes identified deductively based on the quality framework and inductively from the data (Krueger & Casey, 2015; Onwuegbuzie et al., 2009). Subsequently, horizontal analysis was used to synthesize similarities, differences, and recurring patterns across the focus groups (Bazeley, 2013; Miles et al., 2014). This comparative approach informed a first revision of both the quality framework and evaluation procedure. In a second phase, a pilot study assessed the reliability of the quality framework and its evaluation procedure for screening learning materials. A total of eleven learning materials were evaluated by 37 reviewers, organized into evaluation committees of three to five members per learning material. These learning materials represented a wide range of materials including, for example, purely digital resources, primary school mathematics materials, and materials for niche subjects in secondary education. First, all reviewers independently assessed the materials and submitted their findings. Second, each committee participated in a session to reach a collective judgment. Part A of the pilot study examined the broad applicability of the quality framework and procedure across the Flemish educational context. Separate committees of three to five reviewers, including both scientists and educational professionals, evaluated eight learning materials. Part B of the pilot study investigated the extent to which different reviewers independently produced comparable judgments when screening the same learning materials. Dual committees of five reviewers each evaluated three learning materials. During and after evaluations, feedback was collected via committee reports and surveys with open and closed questions, completed by reviewers (n = 37) and providers (n = 7). Internal consistency of the quality framework was high (Cronbach’s alpha = 0.9), and inter-rater reliability was calculated using the intraclass correlation coefficient (ICC). Item–total correlations were also examined, and items with low correlations were reviewed. Based on the qualitative and quantitative findings, the quality framework and evaluation procedure were revised for a second time.

The current quality framework, consisting of 32 items, and the accompanying evaluation procedure have been fully developed and are ready for implementation. This was preceded by a thorough scientific development phase, based on an extensive literature review and supported by both qualitative and quantitative analyses, including focus groups and pilot study. These confirmed the usability, applicability, and reliability of the quality framework and evaluation procedure, leading to two revisions. The theoretical framework, structured around three main categories, indicators, and 32 items, provides a systematic way to assess effective didactic components in learning materials. The accompanying user guide supports a consistent and thoughtful application of these criteria by providing the necessary contextual information and instructions for use. The outcome is a comprehensive, scientifically grounded evaluation procedure based on the quality framework, enabling the systematic screening of learning materials. Based on this development phase, the first scientific article is currently in preparation, focusing on the development of the quality framework and the design of the evaluation procedure. Beginning in February 2026, a large-scale screening of learning materials will be conducted, with the quality framework and evaluation procedure fully implemented. Throughout this process, both the framework and procedure will be systematically refined and updated based on ongoing analyses, emerging insights, and evolving educational practices, ensuring alignment with current educational needs and assessment standards. This ongoing monitoring of the quality of the screening will directly contribute to a second scientific article, reporting on factor analyses, construct validity, inter-rater reliability, and internal consistency. This article will analyze and describe the effectiveness of the quality framework and evaluation procedure in practice, demonstrating and strengthening the scientific and practical value of the procedure.

Bazeley, P. (2013). Qualitative data analysis: Practical strategies. London, UK: SAGE Publications. Bellens, K., Byls, H., Van Assche, J., Kirschner, P. A., & Verachtert, P. (2022). Wijze Lessen voor het ontwikkelen, kiezen en gebruiken van leermiddelen in het leerplichtonderwijs: handleiding bij een kader voor didactische kwaliteit van leermiddelen. Thomas More Hogeschool. De Man, L. (2022). Naar een kwaliteitsalliantie. https://www.vlaanderen.be/publicaties/naar-een-kwaliteitsalliantie Dockx, J., Pelgrims, L., Aesaert, K., & Janssen, R. (2023). Tonen de Vlaamse peilingsonderzoeken en internationale studies dezelfde trends in leerlingprestaties?. Pedagogische Studiën, 100(4), 421-447. Faddar, J., Appels, L., Merckx, B., Boeve-de Pauw, J., Delrue, K., De Maeyer, S., & Van Petegem, P. (2020). Vlaanderen in TIMSS 2019. Wiskunde- en wetenschapsprestaties van het vierde leerjaar in internationaal perspectief en doorheen de tijd. Antwerpen: Universiteit Antwerpen. Flemish Government. (2025). Flemish tests. Retrieved November 10, 2025 Krueger, R. A., & Casey, M. A. (2015). Focus groups: A practical guide for applied research (5th ed.). Thousand Oaks, CA: SAGE Publications. Miles, M. B., Huberman, A. M., & Saldaña, J. (2014). Qualitative data analysis: A methods sourcebook (3rd ed.). Thousand Oaks, CA: SAGE Publications. Morgan, D. L., & Krueger, R. A. (1998). The focus group kit (Vols. 1–6). Thousand Oaks, CA: SAGE Publications. OECD. (2023). PISA 2025 science framework. OECD Publishing. Retrieved from https://pisa-framework.oecd.org/science-2025/ Onwuegbuzie, A. J., Dickinson, W. B., Leech, N. L., & Zoran, A. G. (2009). A qualitative framework for collecting and analyzing data in focus group research. International Journal of Qualitative Methods, 8(3), 1–21. https://doi.org/10.1177/160940690900800301 Patton, M. Q. (2002). Qualitative research and evaluation methods (3rd ed.). Thousand Oaks, CA: SAGE Publications.