Data for what? Converging and diverging trajectories of Large Scale Assessment in Education: OECD versus IEA
Author(s):
Orazio Giancola (presenting / submitting) Assunta Viteritti
Conference:
ECER 2016
Format:
Paper

Session Information

28 SES 04, Data and Policies in the Fabrication of the European Space of Education

Paper Session

Time:
2016-08-24
09:00-10:30
Room:
NM-B101
Chair:
Paolo Landri

Contribution

The relevance of international space of education in grown over the years thanks to the many devices produced by large surveys that have become the new socio-technical actors at European and global level. Since the Sixties (IEA first and OCDE afetr) the big surveys produce data even more “bigger”, articulated and differentiated conditioning visions, cultures and choices in the educational field.

Large survey mobilize considerable economic resources of the countries participating, complex networks of expertise, a growing set of investigation methods (for data collection and analysis), multiplying the economic and political interests.

In the various waves of these surveys are involved hundreds of thousands people (teachers, students, school principals, families), they are subjected to hundreds of thousands of questionnaires; also it is involved a large number of experts and researchers from different disciplines (including education and pedagogy, information technology, economics, sociology). Large volumes of data are processed, stored, analyzed and used to classify individuals and countries, to show trends, similarities and highlight differences within and between countries.

While in the past century education it has been dominated by the "big theories" (the Durkheimian idea of “sacred education”, the functionalist approach, theories of human capital and rational choice ecc.) in the latter part of the century (since the 60s) reflection has shifted on "big numbers" and quantities.

The "Big data" of education take the place of theories, “talking” in their place. But the data "speak"? And what do they say? What theories are embedded in the larger survey?

In LSA-Large Scale Assessment, the two most important networks in the world are IEA and OECD. The first, born in the late '50s,  has fielded several inquiries including the most important as PIRLS and TIMSS (the first concerns the reading literacy, the second dealing with mathematics and science). The OECD education network has emerged since the nineties of the last century and now produces three major surveys: PISA (mat, science and reading), PIACC (adult literacy and numeracy), TALIS (teachers and head teachers practices). The paper presented here has the following objectives.

1) To investigate the trajectories of the two networks and analyze differences and similarities. What are the controversies, evident and hidden, involving the two major networks IEA and OECD?

2) To see closely the composition of the network OECD and IEA to analyze the methodological differences, conceptualizations, purpose, networks of actors and perspectives.

3) To investigate the cultural background which underlies the survey "LSA - Large Scale Assessment" in education. The hypothesis to be verified is that while OECD incorporates methods, conceptions, implicit theories of human capital and from the economy of education, IEA incorporates disciplinary topics related to pedagogy and docimology that preserves intent of academic research aimed to know the results of learning curricula in schools.

4) To investigate the roles and the effects of LSA in terms of comparison, ranking, classification, evidence-based trans-national policies.

5) To investigate the technological infrastructures through which the LSA surveys are carried out, the socio-technical objects produced and used, in order to reflect on symbolic and material infrastructures.

5) Finally, investigate how the LSA contribute to the construction of international space of educational. Between the various devices that have occurred over the years, such as those have acquired a hegemonic position and to what aim? These big amounts of data (the “Big Data” of education) make more solid or more problematic the various analysis in the educational field? How, and with such rhetoric, the two networks incorporate in their framework the theme of the economic crisis and the role of educational in the present historical context?

Method

The present study, in continuity with previous research (Giancola, Viteritti, 2014, 2015), is based on qualitative and interpretative research method, which will make possible to develop a descriptive analysis of the old and new databases in the educational field and to reflect on the practical and theoretical meaning of these infrastructures. Furthermore, it will be conducted interviews with national and international experts (from OECD and IEA) to gather information on the different visions of the main agencies, the research networks, the tools, and to assess whether from their point of view is an ongoing dispute or if the two models point to a convergence or a thematic breakdown of the field. So, in our hypothesis, these two big supranational entities share the educational field, competing, seeking differentiation of methods and objectives, trying to position themselves and to gain relevance in the policies. IEA is a network involving mainly academics and researchers from many countries and surveys mainly disciplinary and curricular topics. The education network in the OECD, however, has become a complex and networks that influence, supports and directs governments, policies and school policies (at various levels). In particular, this work will analyze the following topics: - The rise of databases in the educational field; - Their methodological evolution - Description and map of the various types of databases The study will try to trace a chronological map of this “databasization” process, describing the growing number of available data sources over time from the two main international organization (OECD, IEA). These issues will be dealt through an analysis of websites (presentation of data, rhetorical discourses, networks of partners for the production of data) and sources (reports and documents). An online research will also be conducted to describe and classify and mapping the various types of databases available on the Internet. The analysis on desk of the documents, websites, codebook, database available, reference literature, will produce an interpretation about: - the rhetoric and content of the communication made by the two major networks (OECD and IEA); - a representation of how the different survey/testing devices have changed over time, how they are differentiated each other both from methodologically and thematically point of view. Therefore, it will developed a detailed analysis of cultural, methods, tools, reference network polarization between the LSA – survey carried by OECD and IEA.

Expected Outcomes

The final goal of the study is therefore to produce a critical analysis of the genesis, (technical) creation/realization, and diffusion of the various typologies and uses of “big data” in education. The analysis, as shown previously, will move from a descriptive point to a more analytical level to highlight the general and specific aspects of this phenomenon, which is rapidly changing the way we look at (and into) educational systems. The main objectives are therefore: 1) produce a map of LSA in terms of network, methods, target, economic founding; 2) a reflection on the culture, on the use and dissemination of results in the two families of network; 3) a critical reflection on the effects produced (or not produced) by the two major networks: the hegemonic pervasiveness of OECD; the techno-vision of the link between education and work; the retreat of the IEA tied to topics and disciplinary with a narrow view of school disciplines. 4) Trace the incremental nature of the empirical instrumentation (individual and contextual questionnaires, tests, etc.) and how this, defining specific performative fields, creates “new” areas of competence which then impact on education systems at the national and local level (changing their practices). 4) Analyze what the data have produced over time. How the two data systems create (or does not create) comparison and benchmarks, with the effect of depoliticizing and technifying the educational field.

References

- Borer, V.L. e M. Lawn, 2013 Governing Education Systems by Shaping Data: From the Past to the Present, From National to International Perspectives, in «European Educational Research Journal», 12, 1, pp. 48-52. - Carvalho, L.M. e E. Costa 2015 Seeing Education with One’s Own Eyes and Through PISA Lenses: Considerations of the Reception of PISA in European Countries, in «Discourse: Studies in the Cultural Politics of Education», Special Is¬sue, Governing by Comparison: The Effects of International Large Scale Assessment on Education Policy and Research, 36, 5, pp. 638-646. - Dale, R. & Robertson, S. eds., 2012. Globalisation and Europeanisation in Education, London: Symposium Books. - Fenwick, T., Landri, P., (2014, a cura di), Materialities, Textures and Pedagogies, New York, Routledge. - Giancola, O. e Viteritti, A. (2014), «Distal and Proximal Vision: a multi-perspective research in sociology of education», European Educational Research Journal, 13 (1), 47-57. - Giancola, O. e Viteritti, A. (2015), «Il ruolo delle grandi survey in campo educativo. L’indagine PISA e il governo dell'educazione tramite i numeri», Rassegna Italiana di Sociologia, 3-4, 555-80. - Gorur, R. , 2014 Towards a Sociology of Measurement in Education Policy, in «European Educational Research Journal», 13, 1, pp. 58-72. - Lawn, M. & Grek, S., 2012. Europeanizing Education: governing a new policy space, Oxford: Symposium Books. - Lawn, M. e S. Grek 2013 Europeanizing Education. Governing a New Policy Space, Oxford, Symposium Books. - Lewis, S. e B. Lingard, 2015 The multiple Effects of International Large-Scale Assessment on Education Policy and Research, in «Discourse: Studies in the Cultural Politics of Education», Special Issue, Governing by Comparison: The Effects of International Large Scale Assessment on Education Policy and Research, 36, 5, pp. 621-637. - Meyer, H.D. e A. Benavot, 2013 PISA and the Globalization of Education Governance: Some Puzzles andPproblems, in H.D. Meyer e A. Benavot (a cura di), PISA, Power, and Policy. The Emergence of Global Educational Governance, Oxford, Symposium Books - OECD,2013, Exploring Data-Driven Innovation as a New Source of Growth: Map¬ping the Policy Issues Raised by «Big Data», OECD Digital Economy Contributions, No. 222, OECD Publishing. - Sanderson, I., 2002 Evaluation, Policy Learning and Evidence-Based Policy Making, in «Public Administration», 80, 1, pp. 1-22. - Savage, M., 2013 The «Social Life of Methods»: A Critical Introduction, in «Theory Culture and Society», 30, 4, pp. 3-21.

Author Information

Orazio Giancola (presenting / submitting)
Universy of Rome, "Sapienza", Italy
Universy of Rome, "Sapienza", Italy

Update Modus of this Database

The current conference programme can be browsed in the conference management system (conftool) and, closer to the conference, in the conference app.
This database will be updated with the conference data after ECER. 

Search the ECER Programme

  • Search for keywords and phrases in "Text Search"
  • Restrict in which part of the abstracts to search in "Where to search"
  • Search for authors and in the respective field.
  • For planning your conference attendance, please use the conference app, which will be issued some weeks before the conference and the conference agenda provided in conftool.
  • If you are a session chair, best look up your chairing duties in the conference system (Conftool) or the app.