International large-scale assessments (LSA) collect an enormous amount of data, which result in datasets containing a wealth of information for secondary analysts. In addition to achievement data from students, they also collect data from teachers, school principals and parents in the hope that this information will assist in our collective understanding of education systems and their improvement. Unfortunately, much of LSA data remains underutilized by policy makers as well as researchers, in part due to the fact that proper analysis requires the use of survey methods that are generally unfamiliar to non-technical researchers (e.g. plausible values, replicate and sampling weights). The intent of this workshop is to provide educational researchers with a set of tools to assist them in unraveling the complexities of two of the largest and most important LSA studies currently taking place: Trends in International Mathematics and Science Study (TIMSS) 2011 (Martin, Mullis, Foy, & Stanco, 2012; Mullis, Martin, Foy, & Arora, 2012) and Progress in International Reading Literacy Study (PIRLS) 2011 (Mullis, Mart, Foy, & Drucker, 2012).
This workshop has three main objectives.
The first one is to provide an overview of the objectives and theoretical frameworks of two of the LSAs conducted by the International Association for the Evaluation of Educational Achievement (IEA). These are the latest cycles of the PIRLS and TIMSS, both conducted within the same time-framework in 2011. Along these lines, the first part of the workshop will briefly review the available documentation of these two studies.
The second objective is to review the studies’ methodological complexities (survey and sample design), implications for analysis and the appropriate analysis.
The third objective of this course is to train the participants in analyzing the data from the TIMSS and PIRLS 2011 taking into account the studies’ complexities and design issues. To achieve this goal the course will provide hands-on training on analyzing data from these studies using software (provided for free by the course organizers) that handles all issues related with the analysis of large-scale assessment data. A new version of the software (IDB Analyzer), which will be released in early 2013, will be presented, introducing both improvements and new functionalities. Among these new functionalities is the use of plausible values achievement scores as explanatory variables.
This course is intended for graduate students, emerging researchers and researchers at all levels who are interested in issues concerning international LSAs, as well as secondary data analysis in reading, mathematics, and science curriculum. Participants are required to bring laptops (Windows PC) with SPSS 15 or higher and have a basic working knowledge in inferential statistics.
The workshop is planned for three hours (two sessions, 1.5 hours each). The syllabus/planned workshop activities are as follow:
Hour 1 – Introduction to the PIRLS and TIMSS 2011studies, questionnaires and data files
Hour 2 – Introduction to statistical complexities of using PIRLS and TIMSS data and overview of the IEA IDB Analyzer (IEA, 2013)
Hour 3 – Work through guided practice exercise, analysis, Q & A

IEA. (2013). IEA IDB Analyzer (version 3.0) [computer software]. Hamburg: IEA Data Processing and Research Center. Martin, M. O., Mullis, I. V. S., Foy, P., & Stanco, G. M. (2012). TIMSS 2011 International Results in Science. Chestnut Hill, MA: Lynch School of Education, Boston College. Mullis, I. V. S., Mart, M. O., Foy, P., & Drucker, K. T. (2012). PIRLS 2011 International Results in Reading. Chestnut Hill, MA: Lynch School of Education, Boston College. Mullis, I. V. S., Martin, M. O., Foy, P., & Arora, A. (2012). TIMSS 2011 International Results in Mathematics. Chestnut Hill, MA: Lynch School of Education, Boston College.