Devolution and Geographies of Education: The Sse of the Millennium Cohort Study for ‘Home International’ Comparisons Across the UK

Author(s):

Chris Taylor(presenting / submitting)Gareth Rees Rhys Davies

Conference:

ECER 2014

Network:

11. Educational Improvement and Quality Assurance

Format:

Paper

Session Information

11 SES 04 B, Comparative Studies on the Quality of Education

Paper Session

Time:

2014-09-03

09:00-10:30

Room:

B232A/B Sala de Aulas

Chair:

Heidi Flavian

Contribution

Educational research has been rather slow to absorb the implications of the ‘spatial turn’ that has been so influential across the social sciences (and the humanities) in recent years (for example, Warf and Arias, 2008). To be sure, there is a long and distinguished tradition of scholarship in comparative education, which has been concerned overwhelmingly with comparisons between different states (Crossley et al., 2007). More recently, interest has begun to grow in the analysis of educational issues at the supra-national level (for example, Ozga et al., 2011). However, the more complex geographies of spatial variation within national states, whilst implicit in much educational research, have much less frequently been analysed systematically (Taylor, 2009).

This relative neglect of the regional and more local scales in the geographical analysis of educational issues is surprising. Educational outcomes (such as levels of educational attainment, for example) exhibit distinctive spatial distributions (at these scales), as well as the much better recognised differentiations between social groups (defined most frequently in terms of gender, ethnic background or social class) or distinctive institutional contexts (such as type of school attended, for instance). Indeed, these different dimensions of educational differentiation are closely interlinked.

Equally, educational policies which are responsible for shaping the availability of educational opportunities also have impacts that are differentiated at the local and regional levels. Moreover, the effects of national-level policies continue to be mediated significantly at the local level, whether through local education authorities or individual schools and colleges. Equally, however, there are complex interactions between national policy initiatives and social and economic circumstances that produce outcomes that are inevitably differentiated between different local areas and, indeed, regions. Clearly, sorting out the effects of these various dimensions of sub-national, geographical variation in education poses complex analytical issues. One innovative approach to addressing some of these issues (at least in the context of the UK) is that proposed by David Raffe and his colleagues (1999), involving what they termed ‘home international’ comparison.

What is most immediately problematic about the use of these ‘home international’ comparisons in such political contexts is straightforwardly that their methodological basis is so crude. For example, the well-established limitations of PISA in providing a satisfactory basis for evaluating the effects of systems of educational provision are nowhere acknowledged (Goldstein, 2008). Similarly, comparisons of levels of educational attainment are frequently made without recognising the difficulties in constructing comparable data-sets for the different ‘home countries’ (Rees, 2012); or taking adequate account of the effects of differences in social and economic conditions between them (Gorard, 2000). Accordingly, if ‘home international’ comparisons are to be used as a method of policy evaluation – and, potentially, policy-learning (Raffe et al., 1999) – it is essential that they are conducted on an appropriately robust methodological basis.

One approach to establishing this sort of robust methodological basis is through the conduct of ‘home international’ comparisons as what have been termed ‘natural experiments’. Hence, the adoption of a different policy in one ‘home country’ (the ‘treatment’ group) allows systematic comparison with another (the ‘control’ group), thereby enabling the delineation of the effects of the policy innovation, assuming, of course, that other differences between the two ‘home countries’ are limited or can, in some way, be allowed for in the analysis (generally, by means of statistical applications).

In this paper, we address this issue of the availability of appropriate data to make stringent ‘home international’ comparisons by drawing upon the Millennium Cohort Study (MCS) – a large-scale birth cohort study of children born across the UK during 2000-2001, with data currently available up until they are seven years old.

Method

A key limitation to any comparative study is the issue of ‘equivalence’. This is an important observation to make, particularly for policy-makers who may be concerned about the extent to which differences in the education systems are responsible for differences in educational achievement. However, the issue of equivalence in comparative geographical studies extends well beyond just accounting for the number of pupils from low-income families. Related to this is the availability of data or information relating to the characteristics of children. One of the benefits of ‘home international’ comparisons is that this reduces a number of major differences between the comparison groups that may otherwise have had to be accounted for. However, even then there is the constraint of having access to common or equivalent outcome measures. In order to address many of these issues we draw upon the first four sweeps of the Millennium Cohort Study (MCS) – a large-scale birth cohort study of approximately 19,000 children born across the UK over twelve months in 2000-01. The first four sweeps of data collection were conducted when the children were around nine months, three years, five years and seven years. As with other birth cohort studies, The MCS has the advantage of tracking individuals over time; thereby providing a more robust means of measuring the effects of educational changes than is provided. In addition, the MCS is unusual in its use of geographical criteria in the selection of its sample of respondents. It therefore provides an excellent basis on which to carry out geographical comparisons of children growing up in the different ‘home countries’ of the UK and, indeed, in different areas within these ‘home countries’. Crucially it also uses a common set of cognitive assessments for all children included in the study. We also attempt to develop the methodological basis on which ‘home international’ comparisons can be made. Here, we apply novel statistical techniques – propensity score matching – which, we argue, produce much more robust comparisons than those – beloved of politicians and the media – which are based upon ‘raw score’ measures of educational attainment. They also have some advantages over more conventional statistical approaches, especially in relation to the clarity with which results can be presented publicly.

Expected Outcomes

There would seem to be three main conclusions from this analysis. The first is that there is no single national ‘success story’, suggesting one education system in the UK is ‘better’ than another. For a start any ranking of countries depends on which measure of cognitive development is being considered. And where there are significant differences in cognitive abilities by country these are relatively small compared to the influence of other conditions that children are born in to. This leads on to the second main conclusion, that there is still a considerable need for further ‘home international’ analyses that utilise genuinely comparative and longitudinal data. The MCS provides an excellent example of this, and as the MCS cohort ages and further sweeps of data are collected this is going to become increasingly more valuable in undertaking ‘home international’ comparisons and for conducting natural experiments of particular policy initiatives in the UK. This in turn leads on to our final conclusion, that much greater consideration should be given to the desire to use simple comparisons in education for the immediate purposes of policy evaluation, formation and borrowing. Comparative education researchers have for a long time raised concerns about this (see Crossley and Watson, 2009) and queried the basis of recent international comparisons (see Sturman, 2012). But despite this, policy-makers appear to be increasingly influenced by simple comparisons of a small number, and often narrowly defined, educational outcomes. Nóvoa and Yariv-Mashal (2003) and Ozga (2012) have gone further, suggesting that comparative studies of education are now essentially political tools and a form of policy technology that undermines the intellectual scholarship that they once had.

References

Warf, B., and Arias, S. (Eds.). (2008). The Spatial Turn: interdisciplinary perspectives. London: Routledge. Crossley, M., Broadfoot, P. and Schweisfurth, M. (eds.) (2007) Changing Educational Contexts, Issues and Identities: 40 years of comparative education, London: Routledge. Crossley, M. and Watson, K. (2009) Comparative and international education: policy transfer, context sensitivity and professional development, Oxford Review of Education, 25, 5, 633-649. Goldstein, H. (2008) Comment peut-on utiliser les etudes comparatives internationals pour doter les politiques educatives d’informations fiables?, Revue Francaise de Pedagogie, 164, pp.69-76. Gorard, S. (2000) Underachievement is still an ugly word: reconsidering the relative effectiveness of schools in England and Wales, Journal of Education Policy, 15, 5, 559-573. Nóvoa, A. and Yariv-Mashal, T. (2003) Comparative research in education: A mode of governance or a historical journey? Comparative Education, 39, 4, 423-443. Ozga, J. (2012) Comparison as a governing technology: the case of PISA, Research Intelligence, 119, 18-19. Ozga, J., Dahler-Larsen, P., Simola, H. and Segerholm, C. (2011) Fabricating Quality in Education: data and governance in Europe, London: Routledge. Raffe, D., Brannen, K., Croxford, L. and Martin , C. (1999) Comparing England, Scotland, Wales and Northern Ireland: the case for ‘home internationals’ in comparative research, Comparative Education, 35, 1, 9-25. Rees, G. (2007) The Impacts of Parliamentary Devolution on Education Policy in Wales, Welsh Journal of Education, 14, 1, 8-20. Sturman, L. (2012) Making best use of international comparison data, Research Intelligence, 119, 16-17. Taylor, C. (2009) Towards a geography of education, Oxford Review of Education, 35:5, 651-669.

Author Information

Chris Taylor (presenting / submitting)

Cardiff University

WISERD

Cardiff

Gareth Rees

Cardiff University, United Kingdom

Rhys Davies