Evidence suggests that school-based Social-Emotional Learning (SEL) programs have a positive impact on both children's academic and social-emotional skills (Corcoran et al., 2011; Durlak et al., 2011; Cipriano et al. 2023). These programs, particularly, prove highly beneficial for students at risk of educational setbacks (Jones et al., 2011), and have proven an effective tool for inclusive education (Mitchell & Sutherland, 2020).  Notably, the impact of these programs is intrinsically tied to the quality of their implementation by educators (i.e. program implementation fidelity, Durlak & Dupre, 2008; Humphrey et al., 2018).

Despite extensive research identifying teacher and school context-related factors that influence the quality of SEL program implementation (Domitrovich et al., 2008; Durlak & Dupre, 2008; Kam et al., 2003), a consensus on which contextual factors are most pivotal or warrant the greatest emphasis remains elusive. Achieving such clarity is essential for addressing these factors with a concerted and informed approach, thus creating a more supportive context for implementing SEL programs effectively within schools and today’s inclusive reality.

In 2023, Ulla and Poom-Valickis (2023) published a systematic review and identified four categories of contextual factors that can influence implementation quality: program support, school, teacher, and student level factors. Their analysis focused on the relative importance of these factors and found that the most frequent statistically significant factors included modeling activities during coaching and teacher-coach working relationship. 

The PAX Good Behavior Game (PAX GBG, Embry et al.., 2003) is an evidence-based prevention program implemented by teachers on the classroom level, which has been recognized by the Wallace Foundation as one of 33 leading SEL programs (Jones et al., 2021). PAX GBG is an SEL program implemented worldwide (Australia, Estonia, Ireland, Sweden, United States). 

The current study takes on a qualitative approach and focuses on two distinct groups of teachers, categorized by the level of implementation quality (fidelity) of PAX GBG as assessed through an observer-rated checklist over the course of the school year in Estonia. The cases under examination consist of teachers with high fidelity and low fidelity in implementing the PAX Good Behavior Game. Those cases were selected to investigate the conditions and experiences that shape a context, determining varying degrees of positive impact on children within an inclusive classroom setting. Given that previous research exploring contextual factors influencing the quality implementation of evidence-based SEL programs has predominantly been quantitative in nature (Ulla & Poom-Valickis, 2023), the qualitative cross-case study aims to elucidate, validate, or challenge the theoretical assumptions advanced in prior research, to determine the relevant conditions for carrying out inclusion through this classroom level toolkit that is utilized worldwide. 

The study thus seeks to add to the discussion about the relevance and concep­tualization of SEL program implementation quality (Berkel et al., 2011; Dane & Schneider, 1998; Durlak, 2016) and the teacher­ and school related factors that may contribute to that (Domitrovich et al., 2008; Durlak & DuPre, 2008).

Research questions:

RQ1: What are the characteristics of high and low fidelity cases?

RQ2: In what ways do the high and low fidelity cases differ in terms of their implementation experience (including contextual factors), and how do these differences contribute to variations in the quality of implementation of the PAX Good Behavior Game program?

Sampling and data collection: In the academic year 2022/2023, all teachers participating in the PAX Good Behavior Game implementation cohort in Estonia were extended an invitation to partake in this study. The invitation requested their consent for the research team to access their implementation quality (fidelity) observation data, resulting in a potential sample of 129 eligible teachers. Remarkably, 28 teachers (constituting 22% of the population) consented to participate. To establish a selection criterion, all 129 teachers were quantitatively ranked based on their fidelity scores, derived from the observational scoring sheets, from highest (score=6.00) to lowest fidelity (score=-4,45). Subsequently, a final sample of 7 teachers was chosen, representing the top 33.33 percentile of implementation quality ranking, with scores ranging from 4.51 to 5.85. Additionally, 7 teachers were selected from the lower 20 percentile of implementation fidelity, where scores ranged from -1.00 to -2.1. Semi-structured interviews were conducted with all 14 teachers to capture their perspectives and insights on their implementation experiences. The interviews are analyzed using Qualitative Cross-Case analysis method (Miles & Huberman, 1994), as it allows to examine the similarities and differences across cases to reinforce validity, support generalizability, and promote theoretical predictions. The analysis is currently ongoing and will be finished by the time of the presentation.

The findings will either validate or refute the hypotheses derived at through previous quantitative studies (Ulla & Poom-Valickis, 2023), namely that certain contextual characteristics, such as teacher-coach alliance or modeling of program activities are related to higher quality program implementation. As Proctor et al (2011) have stated: "Qualitative data, reflecting language used by various stakeholders as they think and talk about implementation processes, is important for validating implementation outcome constructs." The results of the current study may, thus, elucidate teachers' professional development choices or personal values that may predict a high or low fidelity program implementation process. Such factors may have not been previously operationalized or hypothesized in the quantitative study designs prevalent in the current literature. The results should offer a more profound understanding of teachers' SEL program implementation experiences that could lead to more quality inclusion of students in the classroom.

Cipriano, C., et al. (2023). The state of evidence for social and emotional learning: A contemporary meta-analysis of universal school-based SEL interventions. Child Development, 94(5), 1181-1204. Corcoran, R.P., Cheung, A.C.K., Kim, E., & Xie, C. (2017). Effective universal school-based social and emotional learning programs for improving academic achievement: A systematic review and meta-analysis of 50 years of research. Educational Research Review, 56-72. Domitrovich, C. E., et al. (2008). Maximizing the implementation quality of evidence-based preventive interventions in schools: A conceptual framework. Advances in School Mental Health Promotion, 1(3), 6–28. Embry, D., Staatemeier, G., Richardson, C., Lauger, K., & Mitich, J. (2003). The PAX good behavior game (1st edn). Center City, MN: Hazelden. Durlak, J.A., Dupre , E.P. (2008). Implementation Matters: A Review of Research on the Influence of Implementation on Program Outcomes and the Factors Affecting Implementation. American Journal of Community Psychology, 31, 327-350. Durlak, J.A., et al. (2011). The impact of enhancing students’ social and emotional learning: A meta-analysis of school-based universal interventions. Child Development, 82, 405-432. Humphrey, N., Barlow, A., & Lendrum, A. (2018). Quality Matters: Implementation Moderates Student Outcomes in the PATHS Curriculum. Prevention Science, 19, 197-208. Jones, S.M., Brown, J.L., & Aber, J.L. (2011). Two-Year Impacts of a Universal School-Based Social-Emotional and Literacy Intervention: An Experiment in Translational Developmental Research. Child Development 28(2), 533-554. Jones, S.M., et al. (2021). Navigating SEL from the Inside Out. Looking Inside and Across 33 Leading SEL Programs: A Practical Resource for Schools and OST Providers. Preschool & Elementary Focus. Revised & Expanded Second Edition. Kam, C-M., Greenberg, M., & Walls, C.T. (2003). Examining the Role of Implementation Quality in School-Based Prevention Using the PATHS Curriculum. Prevention Science, 4(1), 55-63. Miles, M.B., & Huberman, A.M. (1994). Qualitative Data Analysis. Second Edition. SAGE publications. Mitchell, D., & Sutherland, D. (2020) What Really Works in Special and Inclusive Education : Using Evidence-Based Teaching Strategies. Third Edition. Taylor & Francis Group. Proctor, E., et al. (2011). Outcomes for implementation research: Conceptual distinctions, measurement challenges, and reserch agenda. Administration and Policy in Mental Health and Mental Health Services Research, 38, 65–76. Ulla, T. & Poom-Valickis, K. (2023a). Program support matters: A systematic review on teacher- and school related contextual factors facilitating the implementation of social-emotional learning programs. Frontiers in Education.