Besides their comparative nature, international large-scale assessments (ILSA) are perceived as a valuable source of information for evidence-based policy making in education, opposing the opinion-based practices. The latter relies largely on selective use of information disregarding its quality or on the untested opinions and views of groups of people or individuals which can be subjective or promoting ideological standpoints, prejudices or speculations (Gough, Tripney, Kenny, & Buk-Bege, 2011). Two important aspects of policy making are usually neglected: 1) evidence of existing problems and needs that require public intervention which is fundamental for improving the policy responses; and 2) evidence on the process of objective setting which leads to better understanding of the problem to be addressed (Sanderson, 2002). ILSA provide extensive information of the context of education: student, family, teacher and school background characteristics, attitudes and behaviours; information on the structure and organization of the educational system; and curriculum. All these sources have to be taken into account when comparing countries, analysing the differences, and borrowing policies and practices.

The objective of this symposium is to present and discuss the latest theoretical and methodological developments and issues in the field of policy making in education using ILSA data. The symposium is divided in two parts, 1) theoretical and methodological part; and b) case studies of using concrete ILSA datasets and make reliable conclusions for policy making from them. Whereas the first part focus on methodological considerations of using ILSA datasets and cross-country comparisons, the second part represents case studies of using ILSA datasets to inform policy making in education, accounting for the methodological issues.

The first part has two presentations. Often league-tables for “horse-ranking” the countries (Wiseman, 2010) are the sole policy-making source of information, turning the studies into competition instead of policy tools (Hanushek & Woessmann, 2013). One of the important “products” delivered by ILSA is a database that will permit researchers address scientific and policy-oriented questions in reliable and comparative manner (Klieme, 2013). The first presentation, entitled “Evidence-Based Policy-Making in Education: Can We Take the Horses Back in the Stable, Please?”, is focused on the underutilization of the contextual data accompanying the achievement outcome measures. Also, results from large-scale assessments are often cited in different contexts to refer to differences between groups, provide indicators of proficiency of students, make distinctions between contextual conditions, and so on. But conclusions and inferences drawn from using these data are not without some estimable amount of statistical uncertainty. These issues, as well as the methodological possibilities and limitations of ILSA data, will be discussed in the second presentation entitled “Methodological Considerations when using Large-scale Assessment Databases for Policy Analysis”.  

Two main current organizations have made the largest contributions in policy making in education: the International Association for the Evaluation of Educational Achievement (IEA) and the Organisation for Economic Co-operation and Development (OECD) (Klemencic, 2011). The data collected through IEA Trends in International Mathematics and Science Study (TIMSS) and the OECD Programme for International Student Assessment (PISA) provide large amount of data that can be utilized in the policy-making process. Therefore in the second part of symposium two case studies will be presented. The third presentation, “Examining Trends in Country Performances in International Assessments”, adds to the discussion from the first part of the symposium. The presentation focuses on the practices of reporting trends from OECD PISA cycles adding the information from the collected contextual data into the trend estimation. The fourth presentation “Is teacher experience associated with mathematics achievement?” uses data from the IEA TIMSS study, suggesting that policies related to teacher experience should be designed in a context-based fashion.

Gough, D., Tripney, J., Kenny, C., & Buk-Bege, E. (2011). Evidence Informed Policymaking in Education in Europe: EIPEE Final Project Report. London: EPPI-Centre, Social Science Research Unit, Institute of Education, University of London. Klausen, S. H. (2009). Applied Epistemology: Prospects and Problems. Res Cogitans, 6(1), 220–258. Klemencic, E. (2010). The impact of international achievement studies on national education policymaking : the case of Slovenia - how many watches do we need? In A. W. Wiseman (Ed.), The impact of international achievement studies on national education policymaking, (International perspectives on education and society (Vol. 13, pp. 239–266). Klemencic, E. (2011). Mednarodne raziskave znanja, kurikularne politike in globalno/lokalno znanje (International large-scale assessments, curricular theories and global/local knowledge). Institutum Studiorum Humanitatis, Fakulteta za podiplomski humanisticni studij, Ljubljana. Okasha, S. (2008). Filozofija znanosti: Zelo kratek uvod (Philosophy of science. A Very short introduction). Ljubljana: Krtina. Sanderson, I. (2002). Evaluation, Policy Learning and Evidence-Based Policy Making. Public Administration, 80(1), 1–22. doi:10.1111/1467-9299.00292 Wiseman, A. W. (2010). Introduction: The Advantages and Disadvantages of National Education Policymaking Informed by International Achievement Studies. In A. W. Wiseman (Ed.), The Impact of International Achievement Studies on National Education Policymaking (pp. xi–xxii). Bradford: Emerald Group Publishing Limited.