The primary objective of this workshop is to explore how data from international assessments can be used for research related to inclusion and exclusion and thus equity in education and learning outcomes. We will focus in particular on how data from studies conducted by the IEA (International Association for the Evaluation of Educational Achievement) could provide further insights for policy and practice.

As a leading organization in the field of educational research for nearly 60 years, IEA promotes capacity building and knowledge sharing to facilitate innovation and foster quality in education. IEA studies approach the reality of educational learning outcomes in all its complexity by collecting a huge variety of background information that can be related to students’ achievement, knowledge and attitudes.

This course will introduce participants to the IEA International Civic and Citizenship Education Study (ICCS) 2016. ICCS 2016 data be published in March 2018 and provide a rich source for secondary research of outcomes related to civic and citizenship education. ICCS 2016 is IEA’s fourth study exclusively investigating civic education outcomes, following the first Civic Education Study in 1971, the Civic Education Study (CivEd) in 1999, and the first cycle of ICCS in 2009.

The course will include an overview of ICCS, covering its background, conceptual framework and design. It will present some key findings from the 2016 data collection as well as changes since 2009. Participants will be introduced to the survey material (questionnaires, released items) and database, and be provided with access paths to data sources, technical documentation, analysis guides and software tools. There will also be a presentation about available outcome variables like the civic knowledge score, attitude scales or variables regarding students’ civic engagement.

With this information, participants will formulate and discuss research questions that can be addressed with ICCS 2016 data. The instructors will be available to mentor the development of research ideas and design as well as to answer technical questions. Research questions from individual attendants will be presented to all participants in order to provide opportunities to share ideas.

No prior knowledge about large-scale international studies is required. Basic knowledge about statistical analysis is not required, but is an advantage.

Draft Agenda:

Introductory session – 10 min

   - Introduction of participants and their research interests

   - IEA – mission, studies, topics, audiences

   - ILSA – study designs, technical standards, methodological challenges and solutions

ICCS – 20 min

   - Introduction

• Background
• Main research focus, framework
• Design

   - ICCS 2016

• Results, changes since 2009
• Instruments, outcome variables and scales
• Access and availability of data files, technical documentation, analysis guides and software tools

Group work – 35 min

   - Participants will form working groups

   - Each group will receive selected questionnaire materials and information on the corresponding variables (e.g., achievement, attitude and background scales) of ICCS 2016

   - Participants will develop their own research questions that could be answered with the information collected in ICCS 2016

   - Each group presents one or two research questions and gets feedback from the instructors and the audience

Example Analysis – 20 min

   - Live demo of analysis of example research questions

   - Discussion of analysis and results

Summary & closing – 5 min

   - Remaining questions will be answered

   - Summary and conclusions

   - Invitation to advanced data analysis seminars and initiation of collaborative work

The course will begin with a brief introduction to the International Association for the Evaluation of Educational Achievement (IEA), followed by a more detailed introduction to the International Civic and Citizenship Education Study (ICCS). The introductory presentation will include information on the history of the study, its conceptual underpinning and the study design. As ICCS 2016 is already the fourth IEA study exclusively assessing civic education outcomes, the (more recent) two studies preceding ICCS 2016, namely the Civic Education study in 1999 (CIVED) and ICCS 2009, will be presented as well. Next, there will be a summary of key results from ICCS 2016. With these results being just released in November 2017, participants will get insight into the most recent findings of the most recently published international comparative study in civic education. In addition to results of the data collection in 2016, there will also be information about changes in civic education outcomes since 2009. Afterwards, there will be practice sessions that will cover most of the workshop time. Participants will be asked to think about research questions that could be answered using ICCS 2016 data. As an input for that task, participants will be provided links to the available survey material (questionnaires, assessment items already published). Further, information is given about variables and data derived from the questionnaire and assessment items, for example scale scores for latent variables, such as attitudes towards equal rights for immigrants. Then, participants will work in groups to think about and discuss possible research questions ICCS 2016 data could provide answers to. During the group work, the workshop instructors will be available to answer questions and support the groups conceptually. Each group will present their research question(s), and share and discuss these with all participants, to enable an exchange of thoughts and ideas within the group. Finally, by using (some of) the research questions developed by participants, the lecturers will conduct example analyses as a live demo. This will provide participants with first insights into methodological aspects of analyzing quantitative data from international large-scale assessments. Participants should bring a laptop. Lecturers will hand out USB sticks with published ICCS material (assessment framework, questionnaires) that will be used during the group work.

This workshop offers a unique opportunity for participants to learn about the concepts and results of ICCS 2016 as the most recent international large-scale assessment of civic and citizenship education. Participants will learn about the conceptual underpinning and design of the study as well as about the results and findings. During the practice part of the workshop, participants will be able to develop a good understanding about how to address ICCS 2016 data with appropriate research questions. Finally, participants will get insights into appropriate ways of analyzing international large-scale assessment data.

- Brese, F., Carstens, R., Köhler, H., Schulz, W., Weber, S. (forthcoming). ICCS 2016 User Guide for the International Database. Amsterdam, The Netherlands: IEA - Schulz, W., Ainley, J., Fraillon, J., Losito, B., & Agrusti, G. (2016). IEA International Civic and Citizenship Education Study 2016 Assessment Framework. Amsterdam, The Netherlands: IEA - Schulz, W., Ainley, J., Fraillon J., Losito, B., Agrusti, G., & Friedman, T. (2017). Becoming Citizens in a Changing World. International Civic and Citizenship Education Study 2016 International Report. Amsterdam, The Netherlands: IEA - Schulz, W., Carstens, R., Losito, B., & Fraillon, J. (2018). ICCS 2016 Technical Report. Amsterdam, The Netherlands: IEA