Comparative reviews of educational reforms in Europe, and broadly, in the last twenty years or so, show a strong movement towards identifying key competencies in education (Rychen & Salganik, 2000) and implementing more competence-based curricula (European Education and Culture Executive Agency, Eurydice, 2012). These reform movements realized diverse ways and diverse innovations in teaching and learning practices, assessment tools and procedures, and school organization (European Commission, 2020). This movement is determined by European policy embodied in the Reference Framework of Key Competencies for Lifelong Learning (European Commission, 2019). The European policy is the result of several broad factors (Halasz & Michel, 2011) that could be summed up in the assumption that globalization and modernization are creating an increasingly diverse and interconnected world. In this paper, among all the key competencies that young people need in order to be successful in modern society and to contribute to the development of society, we focused on the social and civic competencies and the way how they are operationalized to be measured in two comprehensive international assessment studies ICCS by IEA and PISA Global Competence by OECD.
As stated in the Reference Framework of Key Competencies for Lifelong Learning (European Commission, 2019), civic competence is defined by the importance of knowledge of the development of, institutions that reflect, democracy, justice, equality, citizenship, and civil rights. In the IEA/ICC study, civic competence is operationalized through four content domains (Civic society and systems, Civic principles, Civic participation, Civic identities), four affective-behavioural domains (Value beliefs, Attitudes, Behavioral intentions, Behaviors) and two cognitive domains (Knowing, Reasoning and analyzing), (Schulz et al, 2022). Serbia participated in this assessment study in 2022 (OECD, 2).
Compared to IEA/ Civic Competence, OECD/PISA Global Competence is defined more broadly, as a multidimensional capacity which enables individuals to examine local, global and intercultural issues, understand and appreciate different perspectives and world views, interact successfully and respectfully with others, and take responsible action toward sustainability and collective well-being (OECD, 2018). Serbia participated in this assessment study in the PISA cycle 2018 (OECD, 2020).
The two constructs that are the focus of this study have a potentially great influence in the field of informing educational policies, defining the curriculum and assessment of educational achievements, as well as in designing pre-service and in-service programs for the professional development of teachers. The importance of these constructs is given by the fact that they are part of large international assessment studies in which a significant number of countries participate. A review of the literature, however, shows that there is a lack of research examining the complementarity of these constructs.
Based on the structural components and description of the two constructs (civic competence and global competence), although they were operationalized for measurement purposes in two different assessment studies, we assumed their complementarity. Examining this assumption is the fundamental question addressed in this paper. If the measures were distributed on one scale, the assumption of unidimensionality of the scale would be confirmed. The main intention that initiated this research work is to create a reliable instrument for evaluating the effects of an intervention program designed to encourage collaborative problem-solving in the secondary school educational context in Serbia.

Participants. Our sample consisted of 620 students from Belgrade high schools (aged 16-18). To increase the diversity of the sample, we selected both general high schools and vocational schools, as well as central city schools and schools on the suburban periphery. From each of the eleven selected schools, we sampled several classes. The sample was balanced across genders (52.6% girls) and skewed towards higher achieving students. Instruments. We selected available PISA and ICCS items according to the cognitive processes involved and difficulty level. First, we used items that target more complex processes like reasoning and applying while items saturated with specific knowledge were rarely selected. Second, we used items requiring higher proficiency levels, considering our participants were older than in the usual PISA and ICCS samples. Finally, five PISA tasks, each containing four to five items, and thirty-five ICCS items were selected. The format of items was (complex) multiple-choice, short answer or constructed response. In addition, several ICCS multiple-choice items were adapted by making them constructed response items or asking for justification to make the item more demanding. The final selection of items was arranged in ten clusters and counterbalanced across ten test booklets. Therefore, each cluster appears two times and at different places in the test booklet. Procedure. After obtaining informed consent from participants, data collection took place at schools, supervised by the school associates. Testing time was one and a half hours. Each test booklet consisted of 4 clusters, two of civic and two of science competence tasks. In this study, however, we only focus on the former. Data analysis. After data collection, we developed training materials based on PISA Global Competence and ICCS coding guides. Three coders were trained in separate sessions for each cluster. Each item was evaluated by one coder, but they resolved vague and atypical responses through mutual discussion and consultation with the supervisor. At the same time, 10% of responses were coded independently by two different coders in order to calculate intercoder reliability. Coded responses were analyzed by the IRT (item response theory) software Winsteps.

The analysis of the data obtained on a sample of 17-year-old students from Serbia confirmed our assumption about the complementarity, i.e. the unidimensionality of the two constructs. The main product and effect of this empirical finding would be the creation of a unique instrument for assessing civic competence. The possibilities of applying such an instrument are multiple and significant for education policies in Serbia, especially if you keep in mind some specifics of the education system in Serbia. Firstly, it is about a highly centralized and over-controlled system with content-based curricula, traditional teaching methods that put students in a passive position, general expectations set on the level of literate reproduction of poorly integrated facts, and lack of assessment data (Pavlovic Babic, 2020). Secondly, civic competence, like other transversal competencies, has the status of a key competence, but in teaching practice, it is represented unsystematically, sporadically and insufficiently, without unique curricula and without appropriate training of teachers, which is also indicated by the results of ICCS 2022 (Schulz et al, 2023). Therefore, assessment of civic competence would provide significant input for improving the curriculum and developing the competence of teachers in this area. In addition, the instrument could be used as the external criteria to assess the effects of various interventions in the educational system. One example of such use is the work on developing a model for supporting competencies for collaborative problem solving, which is the main goal of the project under which this research is carried out.

European Commission, Directorate-General for Education, Youth, Sport and Culture. (2019). Key competences for lifelong learning. Publications Office. https://data.europa.eu/doi/10.2766/569540 European Commission, Joint Research Centre, Sala, A., Punie, Y., Garkov, V., et al. (2020). LifeComp – The European Framework for personal, social and learning to learn key competence. Publications Office of the European Union. https://data.europa.eu/doi/10.2760/302967 European Education and Culture Executive Agency, Eurydice. (2012). Developing key competences at school in Europe – Challenges and opportunities for policy. Publications Office. https://data.europa.eu/doi/10.2797/93204 Halasz, G., & Michel, A. (2011). Key competence in Europe: Interpretation, policy formulation and implementation. European Journal of Education. 46(3), 289-306 OECD (2018). Preparing Our Youth for an Inclusive and Sustainable World The OECD Pisa Global Competence Framework. Paris: OECD. OECD (2020). PISA 2018 Results (Volume VI): Are Students Ready to Thrive in an Interconnected World? PISA, OECD Publishing. Paris. https://doi.org/10.1787/d5f68679-en Pavlović Babić, D. (2020). International Assessment Studies in Serbia Between Traditional Solutions, Unexpected Achievements and High Expectations. In H. Harju-Luukkainen, N. McElvany, & J. Stang (Eds.), Monitoring Student Achievement in the 21st Century (pp. 223-236). Springer, Cham. Rychen, D. S., & Salganik, L. H. (2000). Definition and Selection of Competencies: Theoretical and Conceptual Foundations (DeSeCo) Background Paper. Neuchâtel: DeSeCo Secretariat. Paris: OECD. Schulz, W., Ainley, J., Fraillon, J., Losito, B., Agrusti, G., Valeria, D., & Friedman, T. (2023). Education for Citizenship in Times of Global Challenge IEA International Civic and Citizenship Education Study 2022 International Report. https://www.iea.nl/sites/default/files/2023-11/ICCS2022-International-Report.pdf Schulz, W., Fraillon, J., Losito, B., Agrusti, G., Ainley, J., Damiani, V., & Friedman, T. (2023). IEA International Civic and Citizenship Education Study 2022 Assessment Framework. Springer. https://link.springer.com/book/10.1007/978-3-031-20113-4