Annual Report 2014, Porto
EERA Network 9: Assessment, Evaluation, Testing and Measurement
Annual Report 2014
I. Conference programme
The conference programme of Network 9 at ECER 2014 in Porto consisted of 37 sessions with a total number of 102 papers. Five posters were presented. As a novelty and a highlight, the network programme included this year also a Network Keynote on the topic “Education in the Information Society – Assessment, Evaluation and other Challenges”, held by Tjeerd Plomp.
The sessions attracted a varied audience due to the range of different topics presented, with 10 to 20 people attending on average. Single sessions, and the keynote session in particular, attracted more than 50 participants. The Network programme and its organization once again comprised an international body of researchers, with authors from 31 countries, chair persons and discussants from 16 countries, and reviewers from 20 countries.
Of the four symposia organized with participation of Network 9, one was organized solely within Network 9 and addressed the following topic:
Towards explaining achievement: Findings from International Comparative Achievement Studies (Symposium organizers: Wilfried Bos, Martin Goy, Pekka Kupari, Tjeerd Plomp, Monica Rosén)
This symposium contained three sessions with a total number of seven papers.
Three further symposia were organized as joint ventures between Network 9 and other networks as part of cross-network activities encouraged by EERA:
Addressing the Challenges at the Interface of Curriculum Innovation and Evaluation (Symposium organizer: Caroline Long)
This symposium included three papers and was conceived as a joint endeavor to bring together researchers from Network 9 and Network 3 (Curriculum Innovation).
Assessing Language Competencies – Theoretical Considerations and Empirical Testing (Symposium organizers: Joana Duarte, Ingrid Gogolin, Martin Goy)
This symposium included four papers and was jointly organized by Network 9 and Network 31 (Language and Education).
How to Tame Monsters: Encounters between Standards and Deviants
The symposium included four papers. The symposium organizers from Network 28 (Sociologies of Education) invited Network 9 to make this a joint event in order to include the audience from both networks, which Network 9 gladly agreed to.
Furthermore, the programme of Network 9 included 28 paper sessions on the following topics (in alphabetic order):
- Analyzing Peer Group Effects
- Assessing Expectations Towards and Achievements in Vocational Education and Training
- Assessing Language and Literacy: Written Composition and Second Language Acquisition
- Assessments and Evaluations in Teacher Education
- Assessments as Opportunities for Learning and Instruction
- Assessments in Higher Education
- Comparing Large-Scale Assessments across Countries and Domains: Issues in Interpretation and Adaptation Procedures
- Competencies and Attitudes of Teachers (Session 1)
- Competencies and Attitudes of Teachers (Session 2)
- Current Issues in Assessments and Evaluations
- Developing and Evaluating Screenings and Assessments for Students with Special Educational Needs
- Developmental Perspectives on Literacy
- Developmental Trajectories of Attitudes and Competencies in the Course of Lower Secondary Education
- Evaluating Science Education
- Findings from Large-Scale Assessments: Relating Teaching and Teacher Characteristics to Student Achievement
- Findings from Large-scale Assessments: Roles of Curricula and Opportunities to Learn
- Findings from Large-Scale Assessments: Students' Socio-Cultural Background and Deviant Behavior as Challenges for Assessments and Education
- Findings from PISA: Performance in and Attitudes Towards Mathematics
- Findings from PISA: Resilience and Diversity
- Impact and Perceptions of Assessment
- Investigating Conditions of School Choice and Selection
- Investigating Regional Differences with Large-Scale Data
- Methodological Issues in Large-Scale Assessments
- Methodological Issues in Tests and Assessments
- School and Class Composition Effects in Assessments
- Students’ Perceptions, Values and Aspirations at the Transition to Upper Secondary Education
- Teachers’ Roles in Assessment Practices
- Towards Understanding Academic Resilience: Exploring the Roles of Individual and Group Level Factors
- In addition to these sessions, Networks 9 and 11 (Educational Improvement and Quality Assurance) continued their collaborations by organizing two joint paper sessions on the topics of ‘Systemic Approaches in Educational Monitoring’ and ‘Checking Effectiveness by Evaluating Schools’.
Further, one research workshop with two consecutive sessions on the topic “Using the TIMSS and PIRLS 2011 International Databases for Secondary Analysis” was held. Workshops such as these have been held for the third time now in Network 9 and are considered to be of high value and relevance for network’s goal of building the professional capacities particularly for emerging researchers. In view of advanced methods in educational research, Network 9 complements the workshops at ECER for example with spring schools on “Advanced Methods in Educational Research” for PhD students and Post-docs which have been held annually since 2011 (for details, see the section on “Network Activities” in the Minutes of the Network Meeting included below).
The poster session revealed the comparatively low amount of accepted posters which were presented in Network 9 (5 posters as compared to 11 posters in 2013). The idea of raising the profile of poster sessions at ECER by Interactive Poster Sessions, which was piloted by Network 9 in collaboration with Networks 8, 10 and 27 at ECER 2011 in Berlin, might be recommenced at ECER 2015 to attract more poster submissions.
The overall intake of submissions of proposals for symposia, papers and posters to Network 9 at the ECER main conference ranges again high among the EERA Networks, with 127 submissions before redirections, rejections and withdrawals. The number of submissions finally to be presented in Network 9 at EERA was considerably lower, which was due to the selection process in the double blind peer review applied in Network 9 and also due to a number of authors withdrawing their submissions shortly before the conference. Lack of travel funding and urgent work commitments were among the reasons given for those late withdrawals.
As in previous years, Network 9 supported the Emerging Researchers’ Conference by participating in the review process as a matter of inter-network courtesy, and also – in view of fostering ECER research culture – to offer feedback for those researchers which start out in the Emerging Researchers Group and join our network later on.
Please refer to the minutes of the Network Meeting and the different sections of the website of Network 9 for more details on the work and activities in Network 9.
II. Minutes of the network meeting
This meeting was attended by about 50 network members, and was held by the Link Convenor of Network 9, Wilfried Bos, in co-operation with his co-convenors Pekka Kupari and Monica Rosén as well as Tjeerd Plomp as emeritus convenor. Five main topics were addressed:
1. Network 9 programme overview
For the overview of the submissions in Network 9, please refer to the above sections of this report.
2. Acknowledgements of reviewers, chairpersons, and discussants
Network 9 is indebted to the commitment of the numerous chairpersons and discussants included in the ECER 2014 programme of Network 9, and to the large number of researchers who support the network by participating in the review process for the ECER conferences.
The network convenors thank all chairpersons, discussants and reviewers for volunteering their valuable time and expertise, and for contributing to the conference organization and the scientific quality of the research presented in Network 9. The names of the chairpersons and discussants are listed in the conference programme.
In order to make the effort of the reviewers visible as well, and to make the review procedure transparent, an acknowledgement of the reviewers for ECER 2014 will be added to the Network website.
3. Network activities
3.1. Capacity building and inter-network collaborations at ECER
Wilfried Bos pointed out that Network 9 promotes capacity building at ECER, for instance by recruiting experienced colleagues to act as reviewers for the Emerging Researchers’ Conference and by hosting research workshops.
Network 9 also follows EERA recommendations to increase inter-network collaborations at ECER. This year, three joint symposia and two joint paper sessions were held.
Details on these activities are provided in the overview on the Network programme (above).
3.2. Spring schools on Advanced Methods in Educational Research
Wilfried Bos gave a short retrospect on four one-week spring schools on advanced methods in educational research held from 2011 till 2014 at the Universities of Dortmund (in 2011 and 2013), Gothenburg (2012) and Oslo (2014). These spring schools are jointly organized by five European universities: Cyprus, Dortmund, Gothenburg, Oslo and Oxford, co-ordinated by Wilfried Bos (Dortmund) and Monica Rosén (Gothenburg). The schools aim to bring together European and international emerging researchers (PhD students and post-docs), train them on methodological topics, promote exchange of research and offer the opportunity for capacity building and international research collaborations.
Based on the first four spring schools being a success, Wilfried Bos announced the plans to continue these EERA spring schools in 2015. A call for proposals and a deadline for applications will be announced on the network’s website under “network activities” and through the Network 9 mailing list.
3.3. Publications
Over the past years, researchers from Network 9 have published a number of peer-reviewed special journal issues based on symposia held at ECER (the list of issues is available on the network website). A further recent publication from the context of Network 9 is an edited volume on “Educational Policy Evaluation Through International Comparative Assessments” (Eds. Rolf Strietholt, Wilfried Bos, Jan-Eric Gustafsson and Monica Rosén). Wilfried Bos encouraged the researchers present at the network meeting to continue with such publications based on work presented at ECER 2014.
4. Succession of Co-convenors in Network 9
Pekka Kupari retired as co-convenor after serving Network 9 for eight years since 2006. On behalf of the whole network Wilfried Bos thanked Pekka Kupari cordially for his longstanding and continued contributions to steering Network 9 and sharing the work of the convenors.
Four new co-convenors were proposed and accepted: Eugenio J. Gonzalez (Educational Testing Service, Princeton), Martin Goy (TU Dortmund University), Sarah Howie (University of Pretoria) and Jana Strakova (Charles University in Prague).
5. Honorary Membership for Sandra Johnson
Wilfried Bos awarded Sandra Johnson with Honorary Membership of Network 9, to acknowledge her role in establishing Network 9 and in steering the network as the first Link Convenor. Sandra Johnson (currently co-director of Assessment Europe) was one of the two founding convenors of Network 9 and served as the Link Convenor for this network for the first ten years since its inception at the European Conference on Educational Research in 1997. Sandra Johnson is the second honorary network member. In the laudation address, Wilfried Bos was joined by Tjeerd Plomp, the first Honorary Member of Network 9.