School evaluation in Sweden: a local perspective
Author(s):
Anders Hanberger (presenting / submitting) Sara Carlbaum Agneta Hult Lena Lindgren Ulf Lundström
Conference:
ECER 2016
Network:
23. Policy Studies and Politics of Education
Format:
Paper

Session Information

23 SES 04 A, Policies and Practices of Evaluation of Quality in Education

Paper Session

Time:
2016-08-24
09:00-10:30
Room:
NM-Theatre Q
Chair:
Annette Rasmussen

Contribution

Evaluation has expanded at all levels of governance as part of the broad doctrine of New Public Management (NPM) (Hood 1991; Pollitt and Bouckaert 2011). According to this doctrine, market mechanisms should be introduced to enhance efficiency and, in the context of school governance, to support competition between schools, free school choice, improved educational quality, and school effectiveness (Lubienski 2009; Lundahl 2013 et al; Merki 2011). Education systems guided by NPM and characterized by results-based management and local autonomy increasingly rely on evaluation at all levels (Mintrop and Trujillo 2007; OECD 2013). Strengthened accountability is assumed to enhance education quality and promote school development (OECD 2015; SOU 2015:22), and a combination of control- and improvement-oriented evaluation systems has been institutionalized at various levels of the school system to promote school development and enhance education quality. However, this development is contested by research claiming that the consequences of growing accountability pressure are problematic for school practice (Hoyle and Wallace 2009; Ravitch 2010). It may create multiple accountability problems, i.e. uncertainty among target groups as to which evaluation system is supposed to do what and for whom and with what authority (Schillemans and Bovens, 2011). Teachers are subjected to too much accountability that can have negative effects on professionals and education (Green 2011; Koretz 2009; Lingard and Sellar, 2013; Hargreaves 1994, Day 2002, Ball 2003, Mausethagen 2013a, 2013b).

Although evaluation is a cornerstone in local school governance it has not been studied much in this context. Local school governance refers to all the public and private school actors’ and institutions’ (e.g. education committees, opposition parties, school principals, teachers and parents) steering of local schools and education. We need more knowledge of the role and consequences of evaluation systems at the local governance level, and into how local school actors respond to these systems. What local decision makers, school providers, principals, and teachers consider relevant, useful, and actionable knowledge (Stehr & Grundmann 2012) is crucial in understanding the role of evaluation in local school governance.

 

This paper explores how local school actors in Swedish compulsory education have responded to prevailing evaluation systems and the growing accountability pressure emerging from the recentralization, marketization, and globalization of education governance. It synthesizes results from a Swedish research project (see method) and aims to improve our understanding of the role and consequences of evaluation in local school governance. It contributes with knowledge of the role and consequences of evaluation at the municipal, school, classroom, and parent/citizen levels. Special attention is paid to the value and consequences of various evaluations for local school development. A close look at evaluation in Sweden is an illustrative case as the education evaluation arena is overcrowded and the decentralised education system provides freedom of choice that actors operating in other education systems in Europe (OECD, 2015; Lawn, 2011) and elsewhere can learn from.

The paper is developed as part of a larger research project; Consequences of evaluation for school praxis –steering, accountability and school development, financed by the Swedish Research Council (2012-2015). The project explores evaluation in compulsory schools (age 13-15) in four municipalities, and this paper synthesises and discusses the results presented in detail five separate papers.

Method

The conceptual framework, developed in a separate article (Hanberger, manuscript), pays attention to the role of evaluation in three models of decentralised governance, the state model, the local government and the multi-actor model. It focuses on three main possible functions that evaluation can have in local school governance, steering, accountability and school development. It presumes that a governance model intends to steer evaluation to meet the governance models’ and governing actors’ evaluation needs, and that evaluations (performance measurements, stand-alone evaluations, synthesis reports and informal/concrete evaluations) can contribute to these functions. Evaluations may also affect governance in unintended and unexpected ways (Hanberger, 2012). Hence, the framework also accounts for constitutive effects (Dahler-Larsen, 2013) of evaluation systems, that is, to tacit or indirect effects, for example, how evaluation (systems) can shape discourses, defining what is important in education and school systems. Four medium-sized municipalities with populations of 75,000–100,000 were selected strategically to reflect differing local conditions and contextual factors that may affect education and the role of evaluation in local school governance. The municipalities differ in political majority, school performance, and share of independent schools, and eight schools were selected for in-depth interviews. The municipalities are anonymized, being referred to as “North”, “West”, “East”, and “South”. The paper is based on the analysis of documents, reports, and studies treating global and national evaluation systems, national and municipal policy documents treating school governance and evaluation, minutes from municipal education committee meetings (2011–2013), municipal websites, and 76 interviews. Four politicians from majority parties and three from opposition parties, 10 administrators (i.e. Head of the Education Department, senior administrators, and evaluation experts), five politically elected local auditors, three representatives of independent schools, eight school principals, and 43 teachers were interviewed in person or, in a few cases, by phone. In addition, an electronic questionnaire sent to teachers was used to complement the interviews with them, to obtain an overview of teachers’ experiences of evaluation in the studied municipalities. Conclusions about the functions, effects, and consequences of evaluation were generated by interpreting interviewees’ responses and various texts (e.g. policy documents, minutes, and websites).

Expected Outcomes

This study shows that multiple accountability problems emerge as a result of overlapping evaluation systems and that local decision makers set up their own evaluation systems to meet the needs of municipal school governance. Most of the evaluation systems identified in Swedish compulsory education (for students aged 13–15 years) produce quantitative data capturing measurable aspects of education, whereas data capturing other parts of the curriculum, more difficult or impossible to measure (e.g. how schools have succeeded in achieving democracy, sustainability, and solidarity objectives), are lacking. A few key performance measures are used in several systems. The identified evaluation systems induce local school actors and institutions to think and act according to the principles of NPM; these are aligned with most decision makers’ and managerial-oriented principals’ endeavours but not with those of all local school actors. This indicates that evaluations in local school governance serve to support and legitimize the applied governance model and current education policy. Stakeholder evaluations that can provide a more multifaceted understanding, including critical accounts that school actors can use for informed deliberation about the status of schools, consequences of current school policy, and where to go in the future, are not found in our case communities. The workload and accountability pressure have increased for both principals and teachers. The consequences have been the most negative for teachers, however, as external evaluations have questioned their professional competence and authority, unintentionally damaging teacher motivation. The external evaluation systems had little or no value in terms of helping teachers improve their teaching practice. Instead, teachers used their own evaluations regarding what works for various groups and students to continuously improve teaching and schools. A few school providers and principals succeeded in developing evaluations addressing the needs of teachers and were used in developing teaching and daily practices.

References

Dahler-Larsen, P. (2013). Constitutive effects of performance indicators: getting beyond unintended consequences. Public Management Review, 16(7), 969–986. Day, C. (2002). School reform and transitions in teacher professionalism and identity. International Journal of Educational Research, Vol. 37, No. 8, 677-692. Hanberger, A. (2012) Framework for exploring the interplay of governance and evaluation, Scandinavian Journal of Public Administration 16(3),9–28. Hanberger, A. (manuscript) Evaluation in Local School Governance: a framework of analysis Hoyle, Eric & Wallace, Mike (2009) Leadership for professional practice. In S. Gewirtz, P. Mahony & A. Cribb (Eds.) Changing Teacher Professionalism. International trends, challenges and ways forward, 67-80. London & New York: Routledge. Lawn, M. (2011). Standardizing the European education policy space, European Educational Research Journal, 10(2), 259-272. Lingard, B., & Sellar, S. (2013). “Catalyst data”: Perverse systemic effects of audit and accountability in Australian schooling. Journal of Education Policy, 28(5), 634-656. Lundahl, Lisbeth; Erixon Arreman, Inger; Holm, Ann-Sofie & Lundström, Ulf (2013) Educational marketization the Swedish way. Education Inquiry 4(3):497-517. Mausethagen, S (2013a). Accountable for what and to whom? Changing representations and new legitimation discourse among teachers under increased external control. Journal of Educational Change, Vol. 14, 423-444. Merki, K.M. (2011) Special issue: Accountability systems and their effects on school processes and student learning’. Studies in Educational Evaluation,37:177-179. Mintrop, H. & Trujillo, T. (2007) ‘The Practical Relevance of Accountability Systems for School Improvement: A Descriptive Analysis of California Schools’, Educational Evaluation and Policy Analysis, 29(4):319-352. OECD (2013). Synergies for better learning: An international perspective on evaluation and assessment. Paris: OECD Publishing OECD (2015) Improving Schools in Sweden: An OECD Perspective. Paris: OECD Publishing. Pollitt, C. & Bouckaert, G. (2011) Public Management Reform: A Comparative Analysis - New Public Management, Governance, and the Neo-Weberian State. Oxford: Oxford University Press. Ravitch, D. (2010). The death and life of the great American school system: how testing and choice are undermining education. New York: Basic Books. Schillemans, T. & Bovens, M. (2011) The Challenge of Multiple Accountability: Does redundancy lead to Overload? In Dubnick, M.J. and Frederickson, H.G. (Eds.) Accountable Governance. Problems and Promises (pp. 3-21). Armonk: ME Sharpe. SOU 2015:22 Rektorn och styrkedjan. Betänkande av Utredningen om rektorernas arbetssituation inom skolväsendet [The principal and the chain of command. Investigation of principals’ work situation in compulsory education]. Stockholm: Fritzes. Stehr, N. and Grundmann, (2012). How does knowledge relate to political action?, Innovation: The European Journal of Social Science Research, 25(1), 29-44.

Author Information

Anders Hanberger (presenting / submitting)
Umeå University
Applied Educational Science
Umeå
Sara Carlbaum
Umeå University, Sweden
Agneta Hult
Umeå University, Sweden
Lena Lindgren
Umeå University, Sweden
Ulf Lundström
Umeå University, Sweden

Update Modus of this Database

The current conference programme can be browsed in the conference management system (conftool) and, closer to the conference, in the conference app.
This database will be updated with the conference data after ECER. 

Search the ECER Programme

  • Search for keywords and phrases in "Text Search"
  • Restrict in which part of the abstracts to search in "Where to search"
  • Search for authors and in the respective field.
  • For planning your conference attendance, please use the conference app, which will be issued some weeks before the conference and the conference agenda provided in conftool.
  • If you are a session chair, best look up your chairing duties in the conference system (Conftool) or the app.