The proposed presentation will answer the main questions of an evaluation study (NOESIS) conducted in Austria from 2010 to 2017. The NOESIS study monitored and documented the implementation of a new school type in secondary I (10 to 14-year-olds) in Austria, the so-called New Middle School (NMS).

This school type was implemented in the school year 2008/2009 as a reaction to the unsatisfactory PISA 2000 and PISA 2003 results of Austrian students. After four years of comprehensive primary education, students have to decide whether to follow the vocationally or the academically oriented track and this transition was seen as occurring at  too early a stage. The New Middle School (NMS) replaced the “old” lower secondary school, which had a tracking system in the core subjects (Maths, German and English) and which traditionally led to vocational education or higher vocational education. The NMS began as a school trial, but was implemented into the regular school system already in 2012/13. Although the NMS was meant to be a school for all and to replace not only the lower secondary school, but also the academic secondary school – traditionally leading to university attendance – the academic secondary school still exists.

At the same time, the NMS was more than a structural reform, but was implemented with pedagogical innovations such as collaborations between school sites, team-teaching in the core subjects, project-based learning, open learning, new time structures and an inclusive school setting in which students can be individually supported. Through these pedagogical innovations, marginalisation processes were to be reduced and transitions to Secondary II smoothened.

The NOESIS study investigated whether and how the educational goals of the New Middle School were met and addressed four main research questions:

(1) Transitions: How, in the light of the objectives (education opportunities, reduction of education deficit), can positive school and learning experiences be achieved during and after the period of schooling under investigation?

(2) School Settings: Which learning resources and options related to the local factors can students use?

(3) Capacity Building: How, from the perspectives of students and teachers, can sustainable networks be developed to strengthen the educational and social potential of the school sites?

(4) Instructional Patterns: Which learning resources and options can students use in the classroom?

A main aspect of the evaluation project was to find criteria for the measurement of the success or failure of the reform process. Unlike many other evaluation studies, which linked the success of a reform to achievement- and test-data, the NOESIS evaluation project followed a longitudinal and life-perspective philosophy. Thus, it researched success patterns of the reform in terms of students’ educational pathways and searched for conditions that enable them such pathways. In this sense, the evaluation was interested in whether the New Middle School was able to build conditions for enabling students to fulfil the goals they set  themselves at the beginning of Secondary I. In the evaluation project, students were seen as experts on their inner and out-of-school lives and so it was of interest whether the specific reforms at the school also became part of their experiences and how they faced such reforms. As such experiences are always embedded in a specific context, the evaluation also tried to focus on these contexts (context of single schools, community and areas, families, etc.) and tried to take them into account.

The presentation will deal with the main research questions and gives an overview of the main results of eight years of evaluation research.

The NOESIS study was based on a multi-method, longitudinal, multi-perspective and multi-cohort design built around four sections and main questions. (1) The transition section researched key factors for successful educational careers of students from a longitudinal perspective. Therefore three cohorts of students, their teachers and their parents (starting in 2010, 2011 and 2012) were asked to complete annual questionnaires concerning their school lives, their learning (motivation), their (academic) self-concepts and their wellbeing in school. The students were followed from the end of primary school (4th grade), through the NMS (5th to 8th grade) and up to 9th grade (the beginning of Secondary II). This study therefore involved two major transition points. (2) A closer look at the school site was part of the section “School Settings” which examined the requirements, resources and options of the different schools using elements of deliberative problem-solving methods (Fung, 2004) and combining their principles with the Delphi method (Linstone & Turoff, 1975) for initiating democratic school improvement processes (Dewey, 1916). Students, teachers, school leaders and representatives of the local authorities were asked to report on expectations and the problems they faced at their school and about possible solutions for handling such problems. The research was done in several phases involving feedback loops to students, parents and teachers. This method helped researching the interplay between the regional contexts of the schools and changes that occurred inside the schools during the school reform process. (3) Especially in the section “Capacity Building”, students were perceived as experts of their lived world and their voices were used in a peer review study to capture their perspectives for school improvement. This peer evaluation (Rudduck, 2007) followed a participatory approach and answered the question as to how, from the perspectives of students and teachers, it was possible to develop sustainable networks for strengthening the educational and social potential of the specific school. During the peer evaluation students and teachers visited a partner school and reflected on their experiences. (4) The “Instructional Patterns” section of the study was interested in the conditions under which the specific measures introduced by the school reform (e.g. no ability tracking, new forms of assessment, student-centred individualized learning) were perceived as helpful for meeting learners’ needs in heterogeneous classrooms. Drawing on “hermeneutic phenomenology” as a method for “researching lived experience” (van Manen, 1990) the research focused on students’ and teachers’ narratives in a number of case studies.

(1) How can positive school and learning experiences be achieved during and after the period of schooling under investigation? The results of the NOESIS study show that main aspects of successful transitions and trajectories are a stable network of peers within the class and school, a good and trusting teacher-student relationship, a stable class climate, high learning motivation and high academic self-concept. (2) Which learning resources and options related to the local factors can students use? The results indicate that the local surroundings of a school are very important and that cooperation between schools and the local community can help students deal with both learning inside and outside of school. One of the main findings is that every school has different needs: even when schools are located in the same town, they might face different problems and might focus on different ways to solve them. (3) How, from the perspectives of students and teachers, can sustainable networks be developed to strengthen the educational and social potential of the school sites? It turned out that for students and teachers it is very helpful to experience other school cultures. Teacher teams, especially, experienced how different pedagogical innovations like open learning were implemented by colleagues in other schools, which helped them reflect on their own teaching practice. (4) Which learning resources and options can students use in the classroom? Teachers are very helpful learning resources, especially when students feel that they have the right to participate in decision-making processes. It turned out that team-teaching is highly appreciated by students who need more support and is helpful when the two teachers work together, but it can be also seen as a distraction when students do not receive any additional support from the second teacher.

Dewey, J. (1916). Democracy and education: an introduction to the philosophy of education. New York: Macmillan. Feichter, H. J. (2012). Capacity Building: Netzwerke und Peers. Mehr als eine (An)Sammlung von Daten. In: Projektteam NOESIS: Eine Schule für alle? Zur Evaluation der Niederösterreichischen Mittelschule (S. 181-195). Graz: Leykam. Fung, A. (2004). Empowered Participation. Reinventing Urban Democracy. New Jersey, Woodstock: Princeton University Press. Geppert, C. (2017). SchülerInnen an der Bildungsübertrittsschwelle zur Sekundarstufe I. Übertritts- und Verlaufsmuster im Kontext der Neuen Mittelschule in Österreich. Budrich: Opladen, Berlin, London. Geppert, C., Katschnig, T., Knapp, M. & Kilian, M. (2014). School Choice Under the Pressure of Reform Efforts. In: Studia paedagogica, Jg. 20, H. 1/2015, S.9-28 (übersetzt in tschechischer Sprache). Geppert, C., Katschnig, T., Knapp, M., Kilian, M. & Hopmann, S. (2015). Mal was Positives von der NMS: Zentrale Ergebnisse der NOESIS-Längsschnittevaluation aus vier Jahren. In: Erziehung und Unterricht 2015, H. 3+4/2015, S. 374-383 Katschnig, T. & Geppert, C. (2016). Investigating Multiple Realities in a School Reform Process using a Multiple-Methods Evaluation Approach - A Discussion on the Basis of an Evaluation Project in Lower Austria (NOESIS). In: SAGE Research Methods Case Education. Katschnig, T. & Kilian, M. (2011): Klassenklima und Kooperation als wichtige Komponenten der innerschulischen Wahrnehmung von SchülerInnen. In: Noesis-Projektteam (Hg.): Eine Schule für alle? Graz: Leykam Verlag, S. 59-80. Katschnig, T. & Kilian, M. (2013). „Unsere Klasse ist cool“ – Eine Längsschnittstudie zur Veränderung des Klassenklimas in Niederösterreichischen Mittelschulen. In: Hopmann et al. (Hg.): Alltag und Aufbruch. Neue Mittelschulen. Wien: Leykam Verlag, S. 43-74. Kilian, M. & Katschnig, T. (2014). Wohlbefinden als Komponente schulischen Lernens - Veränderungen in der Wahrnehmung von ausgewählten Aspekten zum Wohlbefinden von niederösterreichischen MittelschülerInnen im Verlauf der Sekundarstufe I unter Berücksichtigung des Schulstandortes. In: Noesis-Projektteam (Hg.): Gute Schule bleibt verändert. Neue Mittelschulen. Wien: Leykam Verlag, S. 93-120. Kilian, M., Knapp, M., Katschnig, T., Geppert, C. u. Werkl T. (2017). School Transitions from a Longitudinal Perspective. In: Fasching, H., Geppert, C. u. Makarova, E. (Hrsg.). Inklusive Übergänge, S.287-298. Linstone, H., A. & Turoff, M. (Eds.) (1975). The Delphi Method. Techniques and Applications. London, Amsterdam, Don Mills, Ontario, Sydney, Tokyo: Addison-Wesley Publishing Company. Rudduck, J. (2007). Student Voice, Student Engagement, and School Reform. In: Thiessen, D. (Ed.). The international handbook of student experience of elementary and secondary school (pp. 587-610). Dodrecht: Springer. Van Manen, Max (1990). Researching Lived Experience; Human science for an action sensitive pedagogy. London, Ontario: The Althouse Press.