Science, Technology, Engineering and Mathematics (STEM) has become an important policy agenda in many countries around the world to increase international economic competitiveness (Kärkkäinen & Vincent-Lancrin, 2013). With the recognition that teachers are critical to successful learning (Baker‐Doyle & Yoon, 2011; Darling-Hammond, 2000; Hattie, 2011) and as schools grapple with how to introduce STEM into their curriculum (Education Council, 2015), it is becoming increasingly essential to develop processes and programs that support and sustain teacher and school change (Office of the Chief Scientist, 2016a). STEM learning centres play a vital role as part of the STEM education ecosystem (Schugurenzky, 2000; Traphagen & Traill, 2014) in providing specialist learning experiences for students and teachers to compliment school curricula. Depending on their purpose and structure, STEM learning centres can offer informal and non-formal learning opportunities and may be integrated into formal learning as part of school programming. Some STEM learning centres are part of the outreach strategy for universities, such as the University of Arizona STEM Learning Center, which engage local school students in STEM-related programs in partnership with a range of organisations. In comparison, in Italy the Fondazione Golinelli is funded privately by a philanthropic foundation since 1998, providing STEM experiences for students from early childhood through to adulthood. Different again is the LUMA Centre Finland, which is a large multi-university organisation that collaborates with private and public institutions to research, develop and implement non-formal, out-of-school and extra-curricular LUMA activities. Industry collaboration and design-based pedagogy are core foci (Aksela et al., 2021). 

In Victoria, Australia, the Tech Schools are specialised, purpose-built STEM learning centres that are hosted, owned and operated by universities or Technical and Further Education (TAFE) institutions, but funded by the Victorian Department of Education and Training (the Department). Currently consisting of a network of ten Tech Schools operating under a single Tech School ‘model’, Tech Schools are designed to provide learning programs that are developed in partnership with local industry partners to suit local contexts and needs, and are aligned to the Victorian school curriculum. Each Tech School offers different programs. Teachers are offered a range of professional development opportunities, whilst the wider community interacts through events, after school programs, and access to the facility’s resources. Like the other STEM learning Centres mentioned, Tech schools are not schools, but centres that are accessed by local secondary schools to supplement their STEM programs. 

We are conducting a longitudinal evaluation of the Tech Schools Initiative in 2019-2023. The evaluation uses a Theory of Innovation based on Jäger’s (2004) wave model of innovation, which identified three pillars of innovation: content, structure and people. This presentation will focus on one part of the evaluation: the effects of the innovative content arising through the student programs. The research question is: What differential effects do Tech School programs have for participating students and teachers? Six categories of programs were devised in order to undertake a program impact analysis: 

Category 1. Programs with industry (Industry-based technologies and involved Industry and community partners); 

Category 2. Programs focused on problem solving and design-based challenges (Design and problem-based learning); 

Category 3. Programs focused on skill building (Skill development); 

Category 4. Programs with blended delivery modes or locations (Located at the school, host, industry and community, online); 

Category 5. Programs focused on networking and deep engagement (Networking, Weeks or months in duration, Located at Tech School or industry); and 

Category 6. Programs focused on enhancing senior studies (Skill development, Career pathways, Senior school year levels [Year 11 and 12]).

The broader evaluation comprises research methods designed to capture, explore and understand the unique ways Tech Schools operate in practice, how they deliver teaching and learning that meets student and teacher needs, and how they influence the broader STEM and school ecosystem. A longitudinal four-year data collection strategy was developed to be broad, capturing data from all stakeholders (i.e., Tech School staff; partner school principals, teachers and students; industry and community partners; host representatives) from each of the ten Tech Schools; and deep as data collected to construct case studies of five Tech Schools. A suite of tools for data collection was co-designed, piloted and validated by Deakin, the Department and Tech School Directors during 2019 and 2020. The tools include surveys and interviews with each stakeholder group. The analyses have focused on outcomes for students, teachers, schools, industry and community partners and hosts; partnerships elements; and the nature of innovation occurring through the Tech Schools model. The programs have been examined in various ways in 2020, 2021 and 2022. In 2022, an analysis of programs was conducted using all student programs listed on the Tech School websites. The purpose was to identify features of programs relating to program intentions, structures (e.g., timing, location), and stakeholder involvement that might have differential impact for students and teachers and therefore point to best practice. These features were combined to form six program categories. This presentation will provide an overview of the impacts associated with six program categories and then showcase the outcomes of programs categories that represent their most valuable contribution to students and teachers. A program impact analysis used data from a student attitude survey, teacher reflection survey, student exit surveys, and student interviews. The survey items produced largely ordinal data from multiple choice/Likert scale questions. Quantitative datasets were analysed using descriptive statistics to look for varying associations between variables. Qualitative analysis of interviews included representing the espoused outcomes for the students for programs for which there was adequate quantitative data as well as data from the student interviews where students had attended those programs.

Looking across the programs, there are some common features that can have similar or different effects, depending on the program category. The presentation will show how general capabilities, designing and problem solving, technology, industry representation and connection, curriculum content connection, and the online and school delivery modes influence outcomes. The predominant features of each category that were drawn out by the data will be highlighted. Some key points that will be detailed in the presentation include: • Teacher capacity to teach STEM is most influenced when they use Tech School-devised pre- and post-lessons that prepare students for, and follow up after, the Tech School visit. • Where programs are specifically designed to represent or connect with industry or the emphasis is on careers and local industry, there is greater impact on student awareness of STEM and STEM industries, and some impact on interest in STEM studies and pathways. • Technology, design thinking, and collaboration often co-occur in programs, and the effects are generally that students and teachers are more aware and proficient with the design process, enjoy the collaboration and value its contribution to complex and novel solutions, and that technology helps students learn. The program categorisation provides a useful delineation of programs that can be offered at STEM learning centres. Understanding the effects of these for identifying best practice and where to place funding and effort in terms of program design, resourcing and delivery is useful for STEM education organisations operating outside of but integrated into the formal school structure. Tech Schools have become a valuable part of the STEM education ecosystem in the areas where they exist in Victoria because of the range of programs available and their currency to young peoples’ future, teachers’ capacity for STEM teaching, and the pathways into local STEM careers.

Aksela, M., Lundell, J. & Ikävalko, T. (Eds.) LUMA Finland. Together we are more. LUMA Centre FInland. https://www.luma.fi/en/download/luma-finland-together-we-are-more/ Accessed December 16 2021. Baker‐Doyle, K. J., & Yoon, S. A. (2011). In search of practitioner‐based social capital: A social network analysis tool for understanding and facilitating teacher collaboration in a US‐based STEM professional development program. Professional Development in Education, 37(1), 75-93. doi:10.1080/19415257.2010.494450 Darling-Hammond, L. (2000). Teacher quality and student achievement: A review of state policy evidence. Education Policy Analysis Archives, 8(1), 1-44. https://doi.org/10.14507/epaa.v8n1.2000 Education Council (2018). Optimising STEM Industry-School Partnerships: Inspiring Australia’s Next Generation Final Report. Canberra: Education Council. https://www.chiefscientist.gov.au/sites/default/files/2019-11/optimising_stem_industry-school_partnerships_-_final_report.pdf Hattie, J. (2011). Visible Learning for Teachers: Maximizing Impact on Learning. Abingdon, UK: Taylor & Francis Ltd Jäger, M. (2004). Transfer in Schulentwicklungsprozessen. Wiesbaden: VS Verlag für Sozialwissenschaften. Kärkkäinen, K. & Vincent-Lancrin, S. (2013). Sparking Innovation in STEM Education with Technology and Collaboration: A Case Study of the HP Catalyst Initiative. OECD Education Working Papers, No. 91, OECD Publishing. http://dx.doi.org/10.1787/5k480sj9k442-en. Accessed 1 November 2021 Office of the Chief Scientist (2016a). STEM Program Index 2016. Canberra: Commonwealth of Australia. https://www.chiefscientist.gov.au/sites/default/files/SPI2016_release.pdf Acessed 21 December 2021. Schugurenzky, D. (2000). The forms of informal learning: Towards a conceptualization of the field. Centre for the Study of Education and Work, OISE/UT. https://tspace.library.utoronto.ca/handle/1807/2733 Accessed 1 November 2021 Traphagen, K. & Traill, S. (2014). Working Paper: How Cross Sector Collaborations are Advancing STEM Learning. Noyce Foundation. https://smile.oregonstate.edu/sites/smile.oregonstate.edu/files/stem_ecosystems_report_execsum_140128.pdf