Digital Transformation (DT) in Higher Education (HE) has consolidated hybrid and flexible learning as core components of institutional strategies (De Bem Machado et al., 2024). This shift towards the adoption of intentional digital models is now challenging long-standing assumptions about teaching, learning and educational quality. While technological infrastructures and platforms have rapidly expanded, pedagogical practices have often evolved more slowly, resulting in a persistent misalignment between digitally mediated learning environments and the educational needs of contemporary students.

This tension reflects a broader challenge: quality in digital education is a multidimensional and debated construct, shaped by the interaction of pedagogical, technological, and institutional dimensions (Esfijani, 2018).

In practice, institutional approaches to quality tend to privilege evaluation, accreditation, and standardisation processes at the programme or organisational level. Although these mechanisms play an important role in accountability, they frequently remain disconnected from educators’ everyday teaching practices and from the lived learning experiences of students. As a result, quality is typically perceived as an external requirement rather than as a driver capable of supporting teaching innovation and professional growth.

At the same time, HEIs are increasingly expected to respond to social and economic demands for more flexible, inclusive, and relevant forms of learning. Students seek learning experiences that support not only academic achievement but also employability, continuous professional development, and lifelong learning. High-quality course design, therefore, extends beyond issues of engagement and retention; it becomes central to fostering learner agency, self-regulated learning, and meaningful connections between academic learning and real-world challenges (Lowenthal & Davidson-Shivers, 2019). Addressing these expectations requires a pedagogical reconfiguration of teaching practices.

Within this context, the DIGITAsia project positions course quality as a central and actionable lens through which digital transformation can be meaningfully enacted. Conceived as an international strategic partnership supporting DT in HE institutions in Asia, the project recognises the educators’ role as key agents in designing, enacting, and evaluating digitally enhanced learning environments (Singun, 2025).

Education 5.0 is established as the theoretical framework (Babu, 2024), understood as a human-centred vision of education that emphasizes the critical and meaningful use of technology in support of inclusion, personalisation and social impact. It reframes digital technologies as partners in addressing complex educational challenges. This vision informs both pedagogical design choices and the criteria through which course quality is explored, aligning quality enhancement with student-centred learning, accessibility, and real-world relevance.

It is within this conceptual space that we developed the Course Quality Scorecard (CQS). Conceived as a structured yet flexible reflective tool, the CQS is designed to support educators in examining and enhancing the quality of their course designs (Xie et al., 2025). This is a response to recent calls for quality instruments that move beyond standardisation toward capability-building and professional development (McInnes et al., 2024). The CQS translates the principles of Education 5.0 into a coherent set of operational dimensions, covering flexible learning, Universal Design for Learning (UDL), student-centred approaches, data-informed design, emerging AI-enhanced practices, and the development of employability skills. The CQS thus addresses the core domains of online quality that are essential for effective digital delivery (Esfijani, 2018), while avoiding the limitations of prescriptive checklists that often fail to capture the diversity of specific disciplinary contexts (McInnes et al., 2024). 

Within DIGITAsia, the CQS is embedded in ​​a trainers’ programme for educators, along with a Capstone Project (CP) in which they redesign and pilot their courses.

Against this background, this study addresses the following research question: to what extent do educators perceive the CQS  as providing a coherent, clear, and relevant framework for supporting reflection and decision-making on course quality in alignment with Education 5.0 principles.

This study adopts Design-Based Research (DBR) (Amiel & Reeves, 2008; Anderson & Shattuck, 2012) for its capacity to address complex educational problems through iterative, collaborative, theory-informed and participatory processes (Duea et al., 2022). It unfolds across four interrelated phases: (1) analysis of the problem with practitioners; (2) development of a theory-informed CQS; (3) internal validation through expert feedback and a focus group; and (4) reflection and abstraction to inform further refinement. In Phase 1, through dialogue with educators, instructional designers, and quality staff from DIGITAsia partners, a shared challenge emerged: while digital technologies and hybrid delivery models are increasingly adopted, practitioners reported a lack of pedagogically grounded tools to support the renewal of course design practices and to reinterpret quality beyond institutional processes. This analysis highlighted the need for a course-level, educator-centred approach to quality that supports reflective practice, pedagogical innovation, and alignment with Education 5.0 principles. Phase 2 involved the design of a first version of the CQS. The development of the CQS was theoretically anchored in Education 5.0, understood as a human-centred, socially responsive vision of digital education that emphasises inclusion, learner agency, and the meaningful use of technology. To operationalise these principles at the course level, a structured review of prominent international course quality frameworks (including TELAS, EADTU E-xcellence, OLC Quality Scorecards, and Quality Matters) was conducted. The analysis examined standards, dimensions, criteria, indicators, and scoring mechanisms, with particular attention to how these frameworks address inclusivity, student-centred pedagogies, and technology-enhanced learning design. A crosswalk analysis was conducted to map correspondences, overlaps, and divergences, supported by colour-coded visualisations to identify common elements and variations in hierarchical structures. Insights from this synthesis informed the drafting of the first version of the CQS. Phase 3 consisted of the internal validation (Maina et al., 2023) via iterative cycles of implementation, testing, and refinement in practice. The initial version of the CQS, revised following expert feedback from quality staff (7) within the partnership, was first implemented within a future trainers programme. This course aimed to prepare educators and instructional designers to support DT initiatives and was structured around four modules: Education 5.0, UDL, Learning Management Systems and Learning Analytics, and course quality enhancement. Data were collected through a post-implementation focus group with seven participants (one from each project partner institution) with dual roles of educators and train-the-trainers within each institution. The focus group transcript was subjected to a structured thematic analysis.

The first version of the CQS resulting from phase one of the DBR is a structured multi-level scorecard organised into domains, criteria, and indicators. It articulates course quality across three main domains (Course Design, Program Alignment, and Teaching Practice) operationalised through a graduated scoring system (Absent, Developing, Achieved, Exemplary, Not Relevant) that supports reflective evaluation and incremental improvement. It also integrates a visual dashboard scoring system for rapid mapping of the CQS self-evaluation implementation. Educators were invited to reflect on the practical relevance and clarity of the CQS, with a specific focus on how the three domains support educators’ course design decisions, help identify professional development needs, and foster dialogue with institutions to inform training programs, support strategies, and areas for policy and organizational development. Insights from quality experts’ feedback and from educators course post-implementation focus group indicate that the CQS was positively received as a support for reflective practice and informed course design decisions. Overall, the tool was evaluated as conceptually sound and practically useful. The iterative feedback also identifies opportunities to improve the clarity and simplicity of selected indicators, as well as the need , for contextual adaptations to enhance institutional transferability. Together, these iterative results highlight the CQS as a practice-oriented tool that fosters reflective dialogue around course design, making visible teachers’ professional development needs and supporting institutions in identifying training priorities, support mechanisms, and areas for policy and organisational development. Building on these findings, the revised CQS will be implemented in the Capstone Project course for educators engaged in redesigning their own courses. Data from this second implementation will serve as external validation. This step represents the transition towards the fourth phase of the DBR, where reflection from both implementations will be synthesized to inform further refinement of the tool.

-Amiel, Tel & Reeves, T. (2008). Design-Based Research and Educational Technology: Rethinking technology and the research Agenda. Educational Technology & Society, 11(4), 29–40. https://www.j-ets.net/collection/published-issues/11_4 -Anderson, T., Shattuck, J. (2012). Design-Based Research: A decade of progress in Education Research? Educational Researcher, 41(1), 16–25. https://doi.org/10.3102/0013189X11428813 -Babu, B. V. (2024). Education 5.0: An overview. In A. Garg, B. V. Babu, & V. E. Balas (Eds.), Advances in Technological Innovations in Higher Education: Theory and Practices (pp. 168–243). CRC Press. https://doi.org/10.1201/9781003376699 -De Bem Machado, A., Sousa, M.J., Dal Mas, F., Secinaro, S. & Calandra, D. (2024). Digital Transformation in Higher Education Institutions. Springer. https://doi.org/10.1007/978-3-031-52296-3 -Duea, S. R., Zimmerman, E. B., Vaughn, L. M., Dias, S., & Harris, J. (2022). A guide to selecting participatory research methods based on project and partnership goals. Journal of Participatory Research Methods, 3(1). https://doi.org/10.35844/001c.32605 -Esfijani, A. (2018). Measuring quality in online education: A meta-synthesis. American Journal of Distance Education, 32(1), 57–73. https://doi.org/10.1080/08923647.2018.1417658 -Lowenthal, P. R., & Davidson-Shivers, G. V. (2019). Strategies used to evaluate online education. In M. G. Moore & W. C. Diehl (Eds), Handbook of distance education (4th ed., pp. 415–427). Routledge. https://doi.org/10.4324/9781315296135 -Maina, M., Guàrdia, L., Mancini, F., García-Londoño, C. (2023). Development of a model for transversal competence assessment in K-12: an internal validation study for the digital competence. Interaction Design and Architecture(s), 57, 47–64. https://doi.org/10.55612/s-5002-057-003 -McInnes, R., Hobson, J. E., Johnson, K. L., Cramp, J., Aitchison, C., & Baldock, K. L. (2024). Online course quality evaluation instruments: A scoping review. Australasian Journal of Educational Technology, 40(2), 55–75. https://doi.org/10.14742/ajet.8978 -Singun, A. (2025). Unveiling the barriers to digital transformation in higher education institutions: a systematic literature review. Discover Education, 37(4). https://doi.org/10.1007/s44217-025-00430-9 -Xie, J., Ferguson, Y., Gulinna, A., Rice, M., & Nichols, M. (2025). Modification and evaluation of an open-source rubric guiding inclusive design. Distance Education, 46(3), 452–476. https://doi.org/10.1080/01587919.2024.2383227