Annual Report 2013, Istanbul
EERA Network 9: Assessment, Evaluation, Testing and Measurement
Annual Report 2013
I. Conference programme
The conference programme of Network 9 at ECER 2013 in Istanbul consisted of 41 sessions with a total number of 105 papers. Eleven posters were presented. The sessions attracted a varied audience due to the range of different topics presented, with 10 to 20 people attending on average. Single key sessions attracted more than 50 participants. The Network programme and its organization again comprised an international body of researchers, with authors from 33 countries, chair persons and discussants from 12 countries, and reviewers from 18 countries.
The five symposia organized in Network 9 addressed the following topics:
- Countries’ strife of combining high achievement with equality: Success or failure? Evidence from trends over time in TIMSS and PIRLS (Symposium organizers: Kim Bellens and Sarah Gielen) The symposium included three papers.
- Towards explaining achievement: Findings from International Comparative Achievement Studies (Symposium organizers: Wilfried Bos, Martin Goy, Pekka Kupari, Tjeerd Plomp, Monica Rosén) This symposium contained three sessions with a total number of eight papers.
- Structure and determinants of multiple competencies of Grade 4 learners: Findings from PIRLS/TIMSS 2011 combined (Symposium organizer: Heike Wendt, Wilfried Bos) This symposium included nine papers in three thematic sessions.
- Students‘ engagement and performance in reading and orthography: Reciprocal relations and effects of instruction (Symposium organizers: Wilfried Bos, Inge Blatt, Martin Goy, Sarah Frahm) This symposium included five papers in two thematic sessions.
- Research and practice on A-level examinations – A European perspective on student assessment through high-stakes testing (Symposium organizer: Birgit Eickelmann) This symposium included five papers in one session.
Furthermore, the programme of Network 9 included 22 paper sessions on the following topics (in alphabetic order):
- Assessing Foreign Language Competencies
- Assessing Spelling and Written Composition
- Assessing Students' Competencies and Attitudes
- Assessing Students´ Motivations, Beliefs and Aspirations
- Assessment in Higher Education
- Assessments and Evaluation Practices in Mathematics and Science Education
- Ethical Issues in Assessments
- Findings from International Comparative Achievement Studies and their National Extensions (2 sessions):
*Issues in Equity and Inequality
*Evidence from, and in relation to, PISA
- Findings from International Comparative Achievement Studies (2 sessions):
*Relating Achievement and Attitudes to Variables on Individual, Class/School and System Levels
*Relationships in Mathematics and Science Performance
- Findings from Large-scale Assessments
- Home and School Learning Environments, Early Childhood Education and Achievement
- Issues in Developing Tests and Diagnostic Tools
- Issues in Formative and Alternative Assessments
- Issues in Test Construction and Validation
- Language Competencies in Different Cultural and Systemic Frames
- Methodological Issues in Tests and Assessments
- National and Regional Large-Scale Assessments (2 sessions):
*Methods and Findings (Part 1)
*Methods and Findings (Part 2)
- Peer Relations, Classroom Interactions, and Individual Achievement and Motivations
- Relating Students’ Achievement to Class, School and System Level Factors
In addition to these sessions, as part of cross-network activities encouraged by EERA, our network held a four sessions with other networks:
- two joint paper sessions with Network 11 (Educational Improvement and Quality Assurance) on the topics of ‘Relations of standardized tests and evaluations to performance, attitudes, qualifications and transitions’ and ‘Schools’ and students’ evaluation’
- one joint paper session with Network 13 (Philosophy of Education)
- one joint ECER Round Table with Network 22 (Research in Higher Education) on the topic ‘Assessing higher education student learning outcomes – why, how and what next in Australia, Finland and Norway
To raise the profile of poster sessions at ECER, Network 9 organized an Interactive Poster Session in addition to the general poster session. The idea to promote posters in Network 9 through interactive sessions, an active call for submissions to this format and the Waxmann Best Poster Prize seem to show first results, with 11 accepted posters this year as compared to 5 posters or less in previous years.
Further, Network 9 participated in conducting the Emerging Researchers’ Conference of ECER 2013 with a research workshop on the topic “From correlation to causation. Concepts of longitudinal research and access to large-scale databases”. Network 9 supported the Emerging Researchers’ Conference also by providing mentorship for one of the paper sessions at this preconference.
The overall intake of submissions of proposals for symposia, papers and posters to Network 9 was one of the highest in the network’s history, with 146 submissions before redirections, rejections and withdrawals (ECER 2012 saw a slightly higher number with initial 147 submissions). The number of submissions finally to be presented in Network 9 at EERA was considerably lower, which was due to the selection process in the double blind peer review applied in Network 9 and also due to a number of authors withdrawing their submissions shortly before the conference. Lack of travel funding and concerns regarding the safety in Istanbul were among the reasons given for those late withdrawals.
Please refer to the minutes of the Network Meeting and the different sections of the website of Network 9 for more details on the work and activities in Network 9.
II. Minutes of the network meeting
This meeting was attended by about 50 network members, and was held by the Link Convenor of Network 9, Wilfried Bos, in co-operation with his co-convenors Pekka Kupari and Monica Rosén as well as Tjeerd Plomp as emeritus convenor. Six main topics were addressed:
1. Network 9 convenors
Wilfried Bos announced that Pekka Kupari, long-standing co-convenor of Network 9, will retire as convenor at ECER 2014. Suggestions for successors can be sent to Wilfried Bos (<link http: _self external link in new>officebos@ifs.tu-dortmund.de).
2. Network 9 programme overview
For the overview of the submissions in Network 9, please refer to the above sections of this report.
3. Acknowledgements of reviewers, chairpersons, and discussants
Our network is indebted to the commitment of the numerous chairpersons and discussants included in the ECER 2013 programme of Network 9, and to the large number of researchers who support the network by participating in the review process for the ECER conferences.
The network convenors thank all chairpersons, discussants and reviewers for volunteering their valuable time and expertise, and for contributing to the conference organization and the scientific quality of the research presented in Network 9. The names of the chairpersons and discussants are listed in the conference programme.
In order to make the effort of the reviewers visible as well, and to make the review procedure transparent, an acknowledgement of the reviewers for ECER 2013 is added to the Network website.
4. Network activities
4.1 Spring schools on Advanced Methods in Educational Research
Wilfried Bos gave a short retrospect on three one-week spring schools on advanced methods in educational research held in 2011 and 2013 at TU Dortmund University and in 2012 at the University of Gothenburg. These spring schools are a joint project of the Institute for School Development Research at TU Dortmund University, Germany, and the Department of Education and Special Education at the University of Gothenburg, Sweden, co-ordinated by Wilfried Bos and Rolf Strietholt (Dortmund) as well as Monica Rosén and Jan-Eric Gustafsson (Gothenburg). The 2013 spring school included as additional coordinators Pam Sammons (Oxford) as well as Leonidas Kyriakides (Cyprus). The schools aim to bring together European and international emerging researchers (PhD students and post-docs), train them on methodological topics, promote exchange of research and offer the opportunity for capacity building and international research collaborations.
Based on the first three spring schools being a success, Wilfried Bos announced the plans to continue these EERA spring schools in 2014. A call for proposals and a deadline for applications will be announced on the network’s website under “network activities” and through the Network 9 mailing list.
4.2 Publications
Over the past years, researchers from Network 9 have published a number of peer-reviewed special journal issues based on symposia held at ECER (the list of issues is available on the network website). Wilfried Bos encouraged the researchers present at the network meeting to continue with such publications and to publish further special issues based on symposia held at ECER 2013.
5. Horizon 2020: EU Framework Programme for Research and Innovation
Wilfried Bos introduced the audience to the Horizon 2020 programme as an opportunity to launch applications. Within this framework, he presented a short overview of an application on “Educational Conditions of Successful Structural Change in European Metropolises and Regions”, to be submitted within the Horizon 2020 framework from the context of EERA Network 9.
6. Discussion on further network issues
6.1. Maintaining and increasing quality of research presented in Network 9
Tjeerd Plomp addressed the issue of ensuring and improving the quality of proposals. In order to give authors better guidance in developing their proposals and to give the network greater opportunities to monitor the quality of research in the network, guidelines for authors should be enhanced.
6.2. European dimension as aspect of the reviews
Rolf Strietholt (Dortmund) called the European dimension as a review criterion for judging the proposals submitted to Network 9 into question. It was argued that, given the international nature of research and the unclear definition of what the European dimension would comprise and how this should be specifically addressed in the research proposals, this criterion would upset the mean score of the review. In the discussion that followed, it was agreed that the idea of a European Research Space the proposals should refer to is an important point of reference in presentations. However, it was also decided that the issue of how a “European dimension” can be better defined and incorporated in the review procedure to avoid upsetting the decision on the scientific quality of a proposal should be raised in the group of ECER network convenors to discuss options to address the concern raised.
6.3. Keynote speakers
Rolf Strietholt (Dortmund) raised the question of keynote speakers from the context of Network 9. Wilfried Bos reported that the proposition of a keynote speaker from Network 9 for ECER 2014 was rejected. To be independent from such organizational decisions, the reasons for which were not made transparent, Network 9 will have a network keynote with a keynote speaker of the network’s choice.
6.4 Inter-network co-operations
Ingrid Gogolin pointed out the new foundation of Network 31 (Network on Language and Education) and asked if Network 9 as established network could function as a sponsor for this new network. While being cautious due to limited resources in Network 9 to support other networks, Wilfried Bos proposed that joint sessions or joint symposia could be organized to welcome this new network and foster the exchange between Networks 9 and 31.
6.5 Network website
Ingrid Gogolin praised the website of Network 9 as the best kept and most informative among the websites of EERA networks. This praise was highly appreciated, especially coming from a former EERA President.