Success or failure? A mixed method approach to school effectiveness research
Author(s):
Maria Jarl (presenting / submitting) Klas Andersson (presenting) Ulf Blossing
Conference:
ECER 2013
Format:
Paper

Session Information

11 SES 05 JS, Assessment, Effectiveness and School Improvement

Paper Session
Joint Session with NW 26

Time:
2013-09-11
11:00-12:30
Room:
D-311
Chair:
Samuel Gento

Contribution

What processes and interventions make certain schools display good results year after year, while other schools do not manage to break a negative result development? In this article we present a mixed method approach to frame a study answering this question. The research design of the study is a comparative case study involving eight elementary Swedish schools, four of which are categorized as successful and four as failing schools.

There is the ongoing debate concerning limitations of educational effectiveness research (EER) due to traditional reliance on studies largely conducted within a single research paradigm, i.e. a quantitative or a qualitative methodological approach (Johnson & Onwuegbuzie 2004; Lincon & Guba 2000; Teddlie & Sammons 2010; Sammons 2010; Kington, Sammons, Day & Reagan 2011). Teddlie & Sammons (2010) have noted that while much school effectiveness research (SER) have used large scale quantitative methods to identify and measure differences in school effectiveness, school improvement research (SIR) have focused on generating “thick” descriptions of schools using case study approaches. However, to increase the knowledge on what makes certain schools succeed better than others it is necessary to link the paradigms.

We argue that a mixed method approach using quantitative longitudinal data to identify consistently succeeding and consistently failing schools, followed by a qualitative case study approach in which the internal organization of these schools are studied over an extended period of time is a fruitful way of feeding the two paradigms into each other. Thus, the study consists of two parts: the identification of successful and failing schools, and a process-oriented empirical study of the selected schools. 

In the paper, we present the results of the first part of the study. The case selection relied primarily on quantitative longitudinal data, Gothenburg Educational Longitudinal Database (GOLD). GOLD contains information on all individuals born between 1972 and 1992 living in Sweden at the age of 16. The successful schools were selected on the basis of continuous improvement in student outcomes over 10 years. The failing schools were selected based on continuing negative student outcome over 10 years.

Furthermore, we outline the qualitative case studies in which we plan to study the internal organization of these schools over an extended period of time (i.e. over ten years). By comparing successful and failing schools we maximize the dissemination of the data going in to the qualitative investigations. Since adequate explanations should include accounts both for causal effects and causal mechanisms, we argue for the need of a process-oriented approach in the second part of the study (George & Bennett 2004; Teorell & Svensson 2010). Following scholars in the field, we argue that case studies are superior at identifying omitted or new explanatory variables. In the search for variables explaining the dissemination of school success, close up studies of decision-making processes and patterns of action and interaction in schools is needed.

Method

The study´s point of departure is the case selection using GOLD longitudinal data. Student outcome during a ten-year period (2000-2010) is taken into account. The data displays that despite the fact the schools in Sweden are governed by the same national legislation and within the same institutional frames there exists a large variation between schools when it comes to student outcome. We describe the statistical model used to identify the top four and the bottom four of Sweden’s 2081 primary schools, i.e. schools accomplishing the highest and the lowest results in student outcome over the ten-year period (2000-2010) consistently. Describing the model we discuss contextual and individual factors of importance to student outcomes and how they were ruled out in the selection of cases by controlling for relevant variables in the statistical analysis. In the second part of the article we briefly outline the design of the qualitative case study, in which the method of process-tracing will be used to tell the story of the development of the selected schools during the ten year period.

Expected Outcomes

The outcome of the article is two-folded. First, we present the result of the longitudinal design of the case selection process. Second, we propose the process-oriented model for the study of educational effectiveness that will contribute to the ongoing debate on methodological approaches in educational effectiveness research. Finally, we discuss the implications of the findings for the development of educational effectiveness research and provide some suggestion for further research.

References

George A.L., & Bennet a. (2004) Case studies and theory development in the social sciences. BCSIA. Cambridge. Massachusetts. Johnson, R. B., & Onwuegbuzie, A. (2004). Mixed methods research: A research paradigm whose time has come. Educational Researcher, 33(7), 14–26. Kington, A, Sammons, P, Day, C and Regan, E (2011) Stories and Statistics: Describing a Mixed Methods Study of Effective Classroom Practice, Journal of Mixed Methods Research 2011 5: 103 Lincoln, Y. S., & Guba, E. G. (2000). Paradigmatic controversies, contradictions, and emerging confluences. In N. K. Denzin & Y. S. Lincoln (Eds.), Handbook of qualitative research (2nd ed., pp. 163-188). Thousand Oaks, CA: SAGE. Sammons, P. (2010). The contribution of mixed methods to recent research on educational effectiveness. In A. Tashakkori & C. Teddlie (Eds.), Handbook of mixed methods research (2nd ed.). Thousand Oaks, CA: SAGE. Teddlie, C., & Sammons, P. (2010). Applications of mixed methods to the field of educational effectiveness research. In B. P. M. Creemers, L. Kyriakides, & P. Sammons (Eds.) Methodological advances in educational effectiveness research. London: RoutledgeTaylor & Francis. Teorell, J. & Svensson, T. (2010) Att fråga och att svara. Malmö, Liber AB.

Author Information

Maria Jarl (presenting / submitting)
University of Gothenburg, Sweden
Klas Andersson (presenting)
University of Gothenburg, Sweden
University of Gothenburg, Sweden

Update Modus of this Database

The current conference programme can be browsed in the conference management system (conftool) and, closer to the conference, in the conference app.
This database will be updated with the conference data after ECER. 

Search the ECER Programme

  • Search for keywords and phrases in "Text Search"
  • Restrict in which part of the abstracts to search in "Where to search"
  • Search for authors and in the respective field.
  • For planning your conference attendance, please use the conference app, which will be issued some weeks before the conference and the conference agenda provided in conftool.
  • If you are a session chair, best look up your chairing duties in the conference system (Conftool) or the app.