Abstract:

OrKa is a national quality-oriented assessment feedback tool designed to support improvement in Slovenian primary and secondary education (Cankar et al., 2022). It enables schools to access and review student achievement results from national assessments in Grades 3, 6, and 9, as well as from upper-secondary vocational and general Matura examinations. It is primarily intended for teachers and school staff involved in pedagogical and organisational decision-making. Selected analytical outputs are also accessible to external experts. OrKa aims to strengthen schools’ capacity to interpret and use results from large-scale assessments and examinations in a development-oriented manner.

The tool provides schools with systematic access to student achievement data and enables comparison with national reference distributions. It supports analyses at multiple levels, including the school, class, subject, and individual assessment item level, allowing for fine-grained, content-focused examinations of performance patterns across classes and assessment questions. Among the analytical features is value-added analysis (e.g. McPherson, 1993), which provides a school-level estimate of how much students at a given school have progressed compared to students nationwide with similar prior achievement. In addition, OrKa includes advanced, user-oriented visualisations, such as ordinal dominance graphs, which facilitate distribution-sensitive comparisons and interpretation by non-technical users.

The contribution draws on three complementary sources to present OrKa and its use in practice. First, the tool and its key analytical functionalities are illustrated through selected visual examples from the OrKa user interface. Where relevant, the contribution also describes how the use of OrKa is linked to student achievement on national assessments, without this being the primary focus of the poster. Second, available system-level usage information is used to provide insights into how the tool is used across schools. Third, annual survey data collected from school principals offer insights into user experiences, satisfaction with the tool, and reported uses of assessment feedback in school-level decision-making.

OrKa is presented as a national assessment feedback tool and discussed in terms of its relevance for school practice in Slovenia. At a system level, the contribution highlights how structured feedback from national assessments can provide schools with meaningful insights into their own performance and support the planning of pedagogical and organisational improvements. It also points to the broader potential for school leaders and teachers to make more systematic use of such feedback to inform school development processes (e.g. Schildkamp et al., 2019).

Design and approach The poster presents a descriptive, practice-oriented contribution that focuses on how the national assessment feedback tool OrKa is designed and used in everyday school practice. In addition, simple associations between programme use and student achievement are explored to provide contextual insight, without this being the primary focus of the contribution. The emphasis is on describing the forms of assessment feedback available to schools, how the tool is used across different school contexts, and how school principals perceive its role in supporting reflection and decision-making at the school level. Data sources 1. OrKa programme First, the OrKa programme is presented visually, focusing on how assessment results are displayed and used within the system (Urank et al., 2022). The poster highlights key functionalities of the tool, including overviews and distributions of student achievement at the school level, comparisons between school results and aggregated national outcomes, comparisons across years, value-added calculations, and ordinal dominance graphs. 2. Administrative and assessment data In addition, the poster draws on administrative data from the National Examinations Centre on the use of the OrKa programme, together with achievement data from national assessments in primary education and from the vocational and general Matura examinations at the end of upper-secondary education. These data are used to contextualise patterns of programme use across schools. 3. Annual survey data from school principals Third, the contribution draws on annual survey data collected from school principals across approximately 490 primary schools over a five-year period (response rates consistently 90-95%). The surveys focus on self-reported use of OrKa, perceived usefulness, satisfaction with the tool, and reported purposes of using assessment feedback. The surveys also include questions on whether and how feedback obtained through OrKa is used in pedagogical or organisational decision-making. Data analysis In addition to the visual presentation of the programme, the analysis includes basic descriptive summaries of OrKa use and survey responses from school principals. This includes simple descriptive indicators of programme use, such as frequencies and distributions, as well as descriptive statistics (e.g. means and percentages) for selected survey items. Simple analyses are used to explore associations between programme use and student achievement on national assessments.

The poster presents a nationally organised system of assessment feedback that provides primary and upper-secondary schools with structured information on student achievement from national assessments and enables comparison with aggregated national data. In this way, the system supports school self-evaluation by helping schools reflect on their results within a broader educational context. The poster introduces the main functionalities of the programme, presents basic indicators of its use, summarises school principals’ views on the tool, and showcases selected advanced features of OrKa, such as value-added analyses. It also briefly explores how the use of OrKa is related to student achievement on national assessments. Initial indications from system-level usage information and survey data suggest that the use of OrKa has increased over time and that school principals are generally satisfied with the programme. Specifically, approximately 90% of primary schools report regular use of the tool, and principals rate their satisfaction at 4.4 on a 5-point scale. At the same time, the poster highlights that schools differ in how extensively they analyse and use the available data. In this context, the role of systematic reflection and self-evaluation is emphasised as an important element of quality work at the school level. From a quality assurance perspective, the poster illustrates how structured feedback from national assessments can be made accessible and meaningful for schools. Tools such as OrKa help connect national assessment results with everyday school practice by supporting informed reflection and decision-making. The Slovenian experience highlights the value of national feedback systems that go beyond reporting results and instead support schools in actively engaging with assessment data for improvement, in line with research on school performance feedback systems (Visscher & Coe, 2003). The findings also point to opportunities for further work on embedding such tools more systematically in school self-evaluation processes.

Cankar, G., Semen, E., Knežević, M., & Urank, M. (2022). User guide for the OrKa quality assessment tool: National Assessment of Knowledge. National Examination Centre. Datnow, A., & Hubbard, L. (2016). Teacher capacity for and beliefs about data-driven decision making: A literature review of international research. Journal of Educational Change, 17(1), 7–28. https://doi.org/10.1007/s10833-015-9264-2 Mandinach, E. B., & Schildkamp, K. (2021). Misconceptions about data-based decision making in education: An exploration of the literature. Studies in Educational Evaluation, 69, 100842. https://doi.org/10.1016/j.stueduc.2020.100842 McPherson, A. (1993). Measuring Added Value in Schools. Education Economics, 1(1), 43–51. https://doi.org/10.1080/09645299300000006 OECD. (2013). Synergies for better learning: An international perspective on evaluation and assessment. OECD Publishing. https://doi.org/10.1787/9789264190658-en Schildkamp, K. (2019). Data-based decision-making for school improvement: Research insights and gaps. Educational Research, 61(3), 257–273. https://doi.org/10.1080/00131881.2019.1625716 Urank, M., Hrovat, K., Cankar, G., Semen, E., Zupanc, D., & Knežević, M. (2022). OrKa: Quality assessment tool for the National Assessment of Knowledge (Version 3.5.1) [Computer software]. National Examination Centre. Visscher, A. J., & Coe, R. (2003). School performance feedback systems: Conceptualisation, analysis, and reflection. School Effectiveness and School Improvement, 14(3), 321–349. https://doi.org/10.1076/sesi.14.3.321.15842