The objective of this paper, and the research behind it, is to investigate the effective use of descriptive proficiency scales in (international) large-scale assessments (ILSAs) for their intended purposes.

Descriptive proficiency scales (DPS) were developed as assessment tools to complement and overcome the limitations of simple numerical scores. Initially used in ILSAs such as PISA (OECD, 2001), IEA TIMSS (Mullis & Martin, 2013), and IEA ICILS (Fraillon et al., 2014), DPS enable a more nuanced understanding of students’ competencies by providing qualitative descriptions of the levels achieved.

DPS are grounded in psychometric theories such as Item Response Theory (IRT) and the Rasch Model, which position skills and item difficulty on a shared continuum (Rasch, 1980; Wilson, 2005). This approach allows each level to be associated with detailed descriptions of the knowledge, skills, and attitudes students must exhibit to be classified at that level.

Over the years, the adoption of DPS has expanded due to their utility in:

• International Comparability: providing a foundation for comparing competency levels across countries and educational contexts.
• Decision-Making Support: helping policymakers identify skill gaps and design targeted interventions (Turner, 2014).
• Personalized Learning: offering teachers actionable insights to tailor instruction to students’ needs (van der Linden, 2017).

The use of DPS has profound implications for educational design, teaching, and learning. Below are some key areas:

a) Formative feedback and professional development. DPS transform numerical scores into meaningful descriptions that support:

• Feedback for students: students gain a clear understanding of what they can do and what areas need improvement (Blömeke & Gustafsson, 2017).
• Support for teachers: DPS help teachers identify students’ strengths and weaknesses, improving lesson planning and instructional strategies (Turner, 2014).

b) Educational policy and curriculum redesign. Data derived from DPS support decision-making at various levels:

• Evidence-based policies: governments can use insights from DPS to develop targeted educational programs (OECD, 2001).
• Curriculum adaptation: curriculum designers can integrate specific competencies into educational programs to address identified gaps (Fraillon et al., 2014).

c) Promoting educational equity. DPS are powerful tools for reducing inequalities. By providing clear descriptions of competency levels, they can:

• Identify at-risk Students: students not reaching certain levels can receive targeted support (Mullis & Martin, 2013).
• Support diversity: DPS allow teaching to be tailored to the needs of diverse student groups, including those with special educational needs (Turner, 2014).

d) Transforming educational practices. DPS not only inform feedback but also drive changes in educational practices. For example:

• authentic assessment: the use of qualitative descriptions encourages the adoption of real-world assessment activities (van der Linden, 2017).
• Student-centered approaches: teachers can design activities that help students progress to higher levels (Blömeke & Gustafsson, 2017).

A preliminary search across major bibliographic repositories (Scopus, Web of Science, ERIC) using the string “descriptive proficiency scale*” OR “performanc* level* descriptor*” AND “education*” AND “intervention*” OR “practice*” yielded no relevant results. Although the importance of this tool is widely acknowledged, no research has been conducted to systematically investigate or evaluate its actual use.

This gap highlights significant opportunities for educational research and for those involved in the development of descriptive proficiency scales. DPS represent a shift from assessing “what students know” to assessing “what students can do.” Further exploration of their actual use could address current research gaps and maximize their potential in improving the equity and effectiveness of educational systems.

The research question guiding this study is: Are descriptive proficiency scales used in educational contexts (by schools and teachers, as well as students and parents)? If so, in what ways? 

The research adopts an exploratory, observational, and descriptive approach (Coggi & Ricchiardi, 2005) with a sequential mixed-methods design (Trinchero & Robasto, 2019). In the exploratory phase (qualitative), semi-structured interviews will be conducted with several experts of IEA ICILS and the heads of the international survey area at the Italian National Institute for the Evaluation of the Education and Training System (INVALSI). The aim of this phase is deepening the purposes and strategies to design and implement DPS within LSAs. In the observational phase (quantitative), a questionnaire will be administered to schools (principals and teachers) in the Italian sample of IEA ICILS 2023 (but potentially applicable to all national samples). While the ideal sampling method would target the entire population, the study will rely on a volunteer sample (Viganò, 1995) since it is not possible to oblige schools to participate in such a study. The questionnaire has the following structure and is aimed to collect: • Socio-demographic information (gender, age, length of service, teaching order, subject area, territorial macro-area, region, province, school code) • Perceptions and beliefs about standardized tests (IEA, INVALSI, OECD etc.) detected through Likert scales • Perceptions and beliefs about IEA ICILS 2023 detected through Likert scales • Use in teaching (re)design of descriptive proficiency scales (multiple choices, yes/no, and Likert scales) • Examples of the use of scales In the descriptive phase (qualitative), focus groups will be conducted with key informants, defined as schools and contexts where IEA ICILS 2023 descriptive proficiency scales have been used for instructional purposes and to modify teaching and learning practices. The study will follow this timetable: 1. Exploratory phase: January 2025. Semi-structured interviews will be conducted either online or face-to-face, recorded, and transcribed verbatim. 2. Observational phase: February 2025. Following initial contact with school principals, the questionnaire will be distributed and administered online. 3. First data analysis phase: March 2025. The qualitative and quantitative data collected in the first two phases will be analyzed. With regard to qualitative data, a thematic analysis will be conducted (Braun & Clarke, 2006; Nowell et al., 2017); with regard to the quantitative one, descriptive, bivariate, and multivariate analyses will be performed. 4. Descriptive phase: April 2025. Focus groups will be conducted in selected schools that have used descriptive proficiency scales. 5. Second data analysis phase: May 2025. Qualitative data from focus groups will be analyzed and integrated with previous findings to provide a comprehensive interpretation.

At this stage of the research, no specific results are available (they will be during the conference), but it is possible to discuss some limitations and strategies to mitigate them. Volunteer sampling, while practical in certain research contexts, has several limitations: selection bias, lack of representativeness (this limitation could reduce the external validity of the findings, making it challenging to generalize results to the entire Italian ICILS 2023 population), motivation effects, potential for skewed data. To address these potential challenges, it will be considered the following strategies: • Active recruitment • Comparison with population data • Weighting adjustments Moreover, selecting participants for focus groups is a critical step to ensure the collection of meaningful and diverse perspectives. Below are the intended criteria for participant selection: 1. Schools' use of DPS 2. Geographic representation 3. Diversity in school characteristics. 4. Teacher and principal involvement. 5. Variety of subjects and levels. 6. Engagement in educational innovation. Descriptive proficiency scales have proven to be an invaluable tool for interpreting students' competencies. However, the lack of research on their practical use in schools presents a missed opportunity to fully leverage their potential. This study addresses this gap by investigating how DPS are utilized by teachers, principals, and educational systems. The mixed-methods design ensures comprehensive insights, combining qualitative and quantitative approaches to highlight both the opportunities and challenges of implementing DPS. By engaging key stakeholders and incorporating diverse perspectives, the research underscores the transformative role of DPS in education. It advocates for broader integration of DPS to enhance feedback, inform curriculum development, and promote equitable learning opportunities. The findings aim to guide educators and policymakers in unlocking the full potential of DPS to create more inclusive and effective educational systems.

Barbaranelli, C. & Natali, E. (2005). I test psicologici: teorie e modelli psicometrici. Carocci. Blömeke, S., & Gustafsson, J.-E. (2017). Standard setting in education: The Nordic countries in an international perspective. Springer. Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2), 77-101. https://doi.org/10.1191/1478088706qp063oa Choppin, B. (1976). Developments In Item Banking. Paper given at the first European Contact Workshop held at Windsor, UK, in June 1976. Published in “Monitoring National standards of Attainment in Schools”, R. Sumner, Ed., Slough UK: NFER. Coggi, C. & Ricchiardi, P. (2005). Progettare la ricerca empirica in educazione. Carocci. Fraillon, J., Ainley, J., Schulz, W., Friedman, T., & Gebhardt, E. (2014). Preparing for life in a digital age: The IEA international computer and information literacy study international report. Springer Open. Giampaglia G. (1990). Lo scaling unidimensionale nella ricerca sociale. Ed. Liguori. Mullis, I. V. S., & Martin, M. O. (2013). TIMSS 2011 international results in mathematics. TIMSS & PIRLS International Study Center, Boston College. Nowell, L. S., Norris, J. M., White, D. E., & Moules, N. J. (2017). Thematic analysis: Striving to meet the trustworthiness criteria. International Journal of Qualitative Methods, 16(1), 1-13. https://doi.org/10.1177/1609406917733847 OECD (2001). Knowledge and skills for life: First results from PISA 2000. Organisation for Economic Co-operation and Development. OECD (2012). PISA 2009 Technical Report. Paris, France: OECD. Rasch, G. (1980). Probabilistic models for some intelligence and attainment tests. University of Chicago Press. Trinchero, R. & Robasto. D. (2019). I mixed methods nella ricerca educativa. Mondadori Turner, R. (2014). Described proficiency scales and learning metrics. Assessment GEMs no.4. Melbourne, Australia: Australian Council for Educational Research (ACER). Umar, J. (1999). Item Banking. In G. N. Masters & J. P. Keeves (Eds.), Advances in Measurement in Educational Research and Assessment, Pergamon Press, New York van der Linden, W. J. (2017). Handbook of item response theory: Applications. Chapman and Hall/CRC. van der Linden, W.J. (2018). Handbook of Item Response Theory, Volume Three: Applications. Chapman & Hall/CRC Statistics in the Social and Behavioral Sciences Series. Vigano, R. (1995). Pedagogia e sperimentazione. Vita e Pensiero. Wolfe, E. W. (2000). Equating and item banking with the Rasch model. Journal of Applied Measurement, 1(4), 409-434