The last decade has seen wide transformations of educational governance across industrialized countries. Several authors have described how decision-making processes in education have shifted from idea based, deliberative and proactive procedures to evidence based, reactive governance (Grek 2013; Marttila 2014). This shift involves a transition from input oriented to output oriented steering processes in educational policy making. Output orientation in turn requires detailed information about the respective output, e.g. data from standardized assessments. Consequently, a main focus of recent educational reform projects, which is also promoted by the OECD PISA study and the “New Public Management Paradigm” is the targeted use of regular assessment and test-based accountability measures in schools and education systems.

It is plausible to assume that countries expanded the use of assessments for purposes of teacher and school accountability during the last 15 years and that, over time, countries became more similar with regard to their models and types of monitoring and evaluation of education. This is also assumed by Neo-institutionalist theories which predict a growing convergence of national education systems, promoted by influential international actors such as the OECD (Meyer et al. 1992; Powell and DiMaggio 1991).

However, systematic empirical evidence on changes in the prevalence and purposes of assessments is scarce. In this paper, we describe national evaluation and monitoring systems in education and their change in implementation over time. Particularly, we aim to assess, whether there are changes in the levels and purposes of accountability. We also assess the relationship between average competence levels and their impetus for more or less change in assessment policies. We propose an innovative approach for comparing assessment and accountability practices at school level by drawing on data from all available data collections of the OECD PISA study.

The strong focus on standards-based assessments and accountability also is a major point of criticism against PISA and similar testing projects. It is feared that an increase in tests at school has actually negative consequences for learning (through teaching to the test, a streamlining of curricula, etc., Lingard et al. 2013) and might create an unproductive self-sustaining cycle: The increasing importance of education triggers educational reform which in turn increases the need for testing - which again leads to reform (Baker and LeTendre 2005). To some researchers and stakeholders, this dynamic appears as an “epidemic of education policy” (Levin 1998) and a “tyranny of testing” (Mansell 2007) Some scholars identify a “global panopticism” (Lingard et al. 2013) fostered by an “infrastructure of accountability” (Anagnostopoulos 2013). Yet, systematic findings on the relationship between average competence levels and assessment and accountability practices are missing.  

The results of our paper will help to evaluate the substance of the concerns about increased standardized assessment and accountability practices.  

The OECD PISA study does not only assess competences of students in participating countries. Additional information is collected from students, parents and school heads (Kuger et al. 2016). The school level questionnaires cover aspects of assessments and evaluation in schools. In a first step we compiled all questions and items from the six available (2000, 20003, 2006, 2009 2012, and 2015) PISA school questionnaires referring to assessments, evaluation and accountability. Altogether 35 items address assessments and evaluation practices at school level. The items differ in terms of the number of data collections. For example, none of the items have been included in all data collections of PISA. Over time, the number of items on assessment and accountability increased, already indicating a growing significance of these aspects of schooling. Based on the available items, we generate indicators of evaluation and monitoring systems of the respective countries by aggregating the student-school data at the country level. Before performing the aggregation some of the items have to be recoded in order to harmonize answering categories over time, due to changing wording and answering categories between data collections (Jude 2016). The aggregated indicators measure the percentage of students in a country attending schools, which have implemented a respective policy (e.g. conducting standardized assessments 1 or 2 times a year). Such indicators, which are derived from the actual questionnaires, represent meaningful indicators of education systems because they reflect the level of actual practices, whereas formal policy statements might not “arrive” in schools. For our analyses, we restrict the sample of PISA participating countries to 20 OECD countries with valid values for all our indicators of interest. Further, we excluded indicators, which address rather traditional purposes of assessments at school, e.g. using teacher-developed tests to assess students. We grouped the remaining 22 indicators into six topical areas: Standardization, Marketization, School Accountability, Teacher Accountability, Internal Evaluation and External Evaluation. In a first step, we did descriptive analyses of mean values and standard deviations across the whole sample for every year and every indicator. In a next step, we conducted a hierarchical cluster analyses with selected indicators. The cluster analyses helped to identify groups of countries, which exhibit a similar pattern of (change in) assessment and accountability practices. In a last step the association between average competence levels and indicators of assessment and their development over time will be analyzed.

The monitoring of student`s learning progress (through testing) is a genuine feature of modern schooling. Thus, the question is not, whether countries actually make use of assessment data for purposes of selection, accountability and school improvement. The question is rather, if there are changes in these procedures going on within the respective evaluation systems. In our paper, we examine, which aspects of national evaluation systems in education can be described with data generated by the PISA studies. Our systematic analysis of all PISA school questionnaire items on assessment and accountability practices revealed that they are not aligned to theoretical classifications of accountability shemes in education. The PISA questionnaires are based on theoretical frameworks, which change between data collections, so the data collection in this topical area is not consistent. Nonetheless we were able to harmonize a number of indicators for an analyses over time. Our descriptive findings show that of 22 indicators, which have been included in at least two data collections, 21 exhibit a trend of declining heterogeneity across 20 OECD ountries, as the (relative) standard deviations decreased. As regards to the average prevalence, all but one indicator show increasing values, indicating that assessments and the use of assessments for purposes of accountability, but also the implementation of diverse evaluation procedures increased within OECD countries during the last 15 years. This finding supports the neo-institutionalist hypotheses of convergence of education system. The biggest increase can be observed for the use of assessment data for comparisons with other schools or with regional/national performance. But also several indicators for teacher education show substantially increasing trends. In a next step, we will conduct cluster analyses to identify patterns of similar countries in terms of their levels and trends of assessment and accountability practices and average competence levels.

Anagnostopoulos, D. (2013), Infrastructure of Accountability : Data Use and the Transformation of American Education, Cambridge, Mass.: Harvard Education Press. Baker, D. and G. K. LeTendre (2005), National Differences, Global Similarities: World Culture and the Future of Schooling, Stanford and Calif: Stanford Social Sciences. Grek, S. (2013), ‘Expert moves: international comparative testing and the rise of expertocracy’, Journal of Education Policy, 28 (5), 695–709. Jude, N. (2016), The Assessment of Learning Contexts in PISA, pp. 39–51. Kuger, S., E. Klieme, N. Jude and D. Kaplan (2016), Assessing Contexts of Learning, Methodolog, Cham: Springer International Publishing. Levin, B. (1998), ‘An Epidemic of Education Policy: (what) can we learn from each other?’, Comparative Education, 34 (2), 131–41. Lingard, B., W. Martino and G. Rezai-Rashti (2013), ‘Testing regimes, accountabilities and education policy: commensurate global and national developments’, Journal of Education Policy, 28 (5), 539–56. Mansell, W. (2007), Education by Numbers: The Tyranny of Testing, Politico’s Publishing. Marttila, T. (2014), ‘Die wissensbasierte Regierung der Bildung – Die Genese einer transnationalen Gouvernementalität in England und Schweden’, Berliner Journal Für Soziologie, 24 (2), 257–87. Meyer, J. W., F. O. Ramirez and Y. N. Soysal (1992), ‘World Expansion of Mass Education, 1870-1980’, Sociology of Education, 65 (2), 128. Powell, W. W. and P. DiMaggio (1991), The New Institutionalism in Organizational Analysis, Chicago: University of Chicago Press.