Session Information
99 ERC SES 03 G, Assessment, Evaluation, Testing and Measurement
Paper Session
Contribution
The paper is part of national and international studies aimed at defining and assessing digital competencies, with particular attention to those of students at different school levels.
Digital competencies are at the top of the European political agenda, aiming to improve them for digital transformation. The European Skills Agenda 2020 promotes digital competencies and supports the goals of the Digital Education Action Plan for the development of a high-performance digital education system. The Digital Compass and the European Pillar of Social Rights Action Plan set targets to reach at least 80% of the population with basic digital skills and 20 million specialists in information and communication technologies by 2030. In Italy, the current legislation (National Plan for Recovery and Resilience) provides for the country to equip itself with a system of certification of digital competencies from 2025.
It is therefore necessary to define what is meant by digital skills and to measure them. In this perspective, INVALSI, the Italian National Institute for the Evaluation of the Education and Training System, is launching the DIGCOMP.MIS project to define a prototype model to attest digital competencies, applicable for spring 2025; reference are students of secondary II second grade but with the prospect of observing the evolution of digital skills from the end of secondary first grade to the end of secondary second grade.
The framework assumed by INVALSI and by this paper is DIGCOMP 2.2. (Digital Competence Framework for Citizens) developed by the European Commission to describe and assess the digital skills of citizens aged 16 and over.
From 2013 to today, DIGCOMP has found application in the context of employment, education, training, and lifelong learning; it has been adopted at the European level to build the Digital Skills Indicator and to monitor the Digital Economy and Society Index.
Specifically, this project deals with the definition of the levels of digital competence and the adequacy thresholds corresponding to the different school grades.
In large-scale educational surveys, the variables considered consist of skills, knowledge, or skills possessed at a stage of the school career or in a given age group, constructs not directly observable, but defined based on a theoretical reference framework and operationalized to administer standardized tests.
An outcome in terms of numerical score, however, is not directly informative of what students with a given score know and can do concerning the investigated domain; this is a limit for those interested in interpreting the results of a survey and obtaining information for interventions or teaching practices.
The attribution of an explicitly described level allows students, families, and teachers to have significant feedback, which can be integrated by the students into their perception of competence and useful for teachers teaching. Many national and international surveys combine a score result with a description of the corresponding level; similarly, INVALSI does.
The aim of the project will therefore be to define the type of target levels and their identification.
The paper aims to give an account of the first phase of the project, particularly the analysis of scientific literature and models tested or in use in other European contexts that allow linking of the elaboration of the model to the most authoritative and updated studies of national and international research; This, together with the reference to the DIGCOMP framework, allows the proposed model to be modular also given future comparative developments of digital competences surveys.
Method
The project will consist of the following phases and methodologies: • Retrieval and critical analysis of scientific literature • First definition of target-level typology and identification modalities (e.g. standard-referenced approach, descriptive proficiency levels approach) • Improvement of target levels • Level verification and remodeling, also based on INVALSI data (DIGCOMP.MIS, act. 3) • Definition of the scale at levels
Expected Outcomes
The outcome of the project will be the proposal of a model of levels descriptive of the competencies of students in terms of digital skills, corresponding to the score obtained in a standardized national test to measure them. To date, Italy does not have tools and models for this purpose. The collaboration with INVALSI and the contemporaneity of the project concerning DIGCOMP.MIS will allow to orient the research in a way that also corresponds to the application and organizational requirements of a public survey on a national scale at least. There are many predictable benefits. In the field of educational and docimological research, the project addresses original themes and will represent a useful advancement of knowledge. Professionals working in the field of school practice, teaching, and more generally training can make use of the descriptions of the level of competence achieved by the students.
References
Calvani A., Fini A., & Ranieri M. (2009). Valutare la competenza digitale. Modelli teorici e strumenti applicativi. TD-Tecnologie Didattiche, 48, 39-46. Cortoni, I. (2016). La valutazione delle competenze digitali. Analisi di un case study. Rassegna italiana di valutazione, 20(66), 7-28. Cortoni, I. & Lo Presti. V. (2014). Verso un modello di valutazione delle competenze digitali. Rassegna italiana di valutazione, 18(60), 7-23. Desimoni, M. (2018). I livelli per la descrizione delle prove INVALSI. Roma: INVALSI. Durda, T., Artelt, C., Lechner, C.M., Rammstedt, B., & Wicht, A. (2022). Proficiency level descriptors for low reading proficiency: An integrative process model. International Review of Education, 66, 211-233. https://doi.org/10.1007/s11159-020-09834-1 Griffin, P., Gillis, S., & Calvitto, L. (2007). Standards-referenced assessment for vocational education and training in schools. Australian Journal of Education, 51(1), 19-38. https://doi.org/10.1787/dfe0bf9c-en OECD (2023). PISA 2022 Assessment and Analytical Framework. Paris: PISA, OECD Publishing. Scalcione, V.N. (2022). Ambienti tecnologici di apprendimento: strumenti per la valutazione delle competenze digitali. QTimes Journal of Education, Technologies and social studies, 14(4), 171-193. Vuorikari, R., Kluzer, S., & Punie, Y. (2022). DigComp 2.2: The Digital Competence Framework for Citizens. Luxembourg: Publications Office of the European Union. Zumbo, B.D. (2016). Standard-setting methodology: Establishing performance standards and setting cut-scores to assist score interpretation. Appl. Physiol. Nutr. Metab., 41, S74–S82 dx.doi.org/10.1139/apnm-2015-0522
Search the ECER Programme
- Search for keywords and phrases in "Text Search"
- Restrict in which part of the abstracts to search in "Where to search"
- Search for authors and in the respective field.
- For planning your conference attendance you may want to use the conference app, which will be issued some weeks before the conference
- If you are a session chair, best look up your chairing duties in the conference system (Conftool) or the app.