The Successful School Leadership framework (SSL) is a model of best practices for school leadership, as validated through multiple studies over a twenty-year period (Leithwood et al., 2010; Leithwood & Jantzi, 2005; Sun & Leithwood, 2015). The Successful School Leadership survey was created as a measure of perceived school leadership quality under this framework. This survey is a 22-item Likert scale with 5-point response options ranging from “not at all confident” to “very confident”. This scale intends to measure four domains of leadership: (1) setting directions, (2) building relationships and developing people, (3) (re)designing the organization to support desired practices, and (4) improving the instructional program.

                  Leithwood et al. (2023) provided strong validity evidence for this scale, reporting high reliability, measurement invariance for various person characteristics, and good person and item fit. This study also confirmed the four-factor structure and showed predictive validity evidence of high leadership scores for student achievement, even after controlling for relevant confounding variables.

Leithwood et al. (2023) showed evidence of measurement invariance for gender and education level by using the Many Facet Rasch Model (MFRM) to measure the effect of these covariates at the group level, confirming that gender and education were not predictive of scale scores. However, this method of investigating measurement invariance does not consider differences in subgroup effects at the item level. This difference in item parameters across different groups of people when holding person abilities constant is known as differential item functioning (DIF) and is a common threat to measurement invariance and fairness in scale validation (Ackerman & Evans, 1994; Roussos & Stout, 1996).

This study investigates the presence of DIF in the SSLS items for three covariates: gender, experience, and education level. We use traditional DIF detection methods evaluate these three covariates separately. However, recent interest in intersectional approaches to quantitative research (e.g., Albano et al., 2024; Bauer et al., 2021) underscores the importance of not just treating covariates as independent factors, but as more complex interdependent person characteristics. In response to this recent rise in interest surrounding intersectional approaches to quantitative research, we also use some newer, more exploratory DIF detection methods to investigate intersectional DIF in the SSLS items.

                  The data used in this study were collected as part of the research conducted in Leithwood et al. (2023). All participants in this survey were teachers in two different neighboring school districts in the southeastern United States. After filtering participants based on missing data and willingness to participate, this dataset had a total of 136 respondents. Most of the sample population (83%) identified as women, and the rest of the population identified as men. 58% of the sample population held graduate degrees, and the rest held bachelor’s degrees. 38 participants had 1-3 years of experience, 29 participants had 4-10 years of experience, 20 had 11-15 years of experience, and 48 had 15 or more years of experience.

Before evaluating SSLS items for DIF, the Partial Credit Model (PCM) was fit to the data to assess basic properties of model-data fit (Masters, 1982). We used the TAM package in R (Robitzsch et al., 2023) to fit the PCM, which is specified as ln[(P_ni (x_i=k))/(P_ni (x_i=k-1) )]=θ_n-δ_i-τ_ik Where θ_n is person n’s ability level, δ_i is the difficulty of item i, and τ_ik is the rating scale category threshold for item i. Thresholds are located on the logit scale where the probability of response k and k-1 are equal. To assess model-data fit, we calculated infit and outfit for items. Many studies use widely accepted cut points, such as 0.8 to 1.2, as “critical ranges” for infit and outfit to determine if an item is misfitting. However, these general cut points range in value across different studies and are prone to high classification error (Wolfe, 2013). In this study, we use more robust empirically derived critical values for fit statistics via parametric bootstrapping, where bootstrapped 95% confidence intervals are used to determine critical ranges for infit and outfit (see Seol (2016) and Silvia Diaz et al. (2022) for a more in-depth discussion of this method). After checking model-data fit, we began assessing DIF using Lord’s chi-square method for DIF detection in the R package lordif (Choi et al., 2011; Lord, 1980). Lord’s Chi Square tests can check for both uniform and nonuniform DIF by testing for significant differences in models that do and do not contain DIF. We assessed DIF separately for gender, experience, and education level. We then used the Partial Credit Tree model in the R package psychotree (Komboz et al., 2018; Strobl et al., 2015) to assess intersectional DIF. The Partial Credit Tree model is a polytomous Rasch Tree model that uses recursive partitioning to iteratively search for DIF in combinations of covariates. This model allows us to look for intersectional DIF within each possible covariate combination, forgoing the need to predefine intersectional focal groups, making this method more exploratory than other similar intersectional approaches to DIF detection.

The Partial Credit Model showed one item with disordered category thresholds (item 18), indicating that response categories for this item were inconsistent with the observed responses in this round of data collection. The fit analysis showed one item with substantial misfit for both infit and outfit based on the bootstrapped confidence intervals (item 8). The full paper will investigate the content of these items further to locate any potential sources of misfit. The chi-square tests for individual covariate DIF detection revealed DIF for gender and education level, but no DIF for experience. For gender, item 11 showed uniform DIF, indicating consistent differences in item parameters across the latent scale, and item 14 showed nonuniform DIF, revealing DIF that affected people differently at different locations on the latent scale. For Education, item 20 showed uniform DIF, and item 12 showed nonuniform DIF. The full paper will review these items qualitatively to locate any potential sources of item-level bias. However, the Partial Credit Tree showed no evidence of DIF for any covariate, returning a tree model with no node splits. This may indicate that the chi-square model had a level of Type I error in detecting DIF for four items across two different covariates. However, as discussed in Krist et al. (2025), imbalanced samples, like the one we have in this study, tend to lead to an underestimation of DIF and an increase in Type II error. This imbalanced sample size may also be the reason for the discrepancy in DIF detection between the two methods. This problem of imbalanced samples is somewhat unavoidable in the current context, given that most teachers in the US are women. Still, future research should explore this discrepancy further, possibly with more participants, more balanced samples, and different DIF detection models to verify DIF results.

Ackerman, T. A., & Evans, J. A. (1994). The influence of conditioning scores in performing DIF analyses. Applied Psychological Measurement, 18(4), 329–342. https://doi.org/10.1177/014662169401800404 Albano, T., French, B. F., & Vo, T. T. (2024). Traditional vs Intersectional DIF Analysis: Considerations and a Comparison Using State Testing Data. Applied Measurement in Education, 37(1), 57–70. https://doi.org/10.1080/08957347.2024.2311935 Bauer, G. R., Churchill, S. M., Mahendran, M., Walwyn, C., Lizotte, D., & Villa-Rueda, A. A. (2021). Intersectionality in quantitative research: A systematic review of its emergence and applications of theory and methods. SSM - Population Health, 14, 1–11. https://doi.org/10.1016/j.ssmph.2021.100798 Choi, S. W., Gibbons, L. E., & Crane, P. K. (2011). lordif: An R package for detecting Differential Item Functioning using iterative hybrid ordinal logistic regression/Item Response Theory and Monte Carlo simulations. Journal of Statistical Software, 39(8), 1-30. https://doi.org/10.18637/jss.v039.i08 Komboz, B., Strobl, C., & Zeileis, A. (2018). Tree-based global model tests for polytomous Rasch models. Educational and Psychological Measurement, 78(1), 128–166. https://doi.org/10.1177/0013164416664394 Krist, A. T., Wind, S. A., & Lakin, J. M. (2025). Rasch tree modeling for DIF analysis: How sample and item characteristics influence DIF detection. Applied Measurement in Education, 1–16. https://doi.org/10.1080/08957347.2025.2559811 Leithwood, K., Harris, A., & Strauss, T. (2010). Leading school turnaround: How successful leaders transform low-performing schools. San Francisco: Jossey-Bass. Leithwood, K., & Jantzi, D. (2005). A Review of Transformational School Leadership Research 1996-2005. Leadership and Policy in Schools, 4, 177-199. https://doi.org/10.1080/15700760500244769 Leithwood, K., Sun, J., Schumacker, R., & Hua, C. (2023). Psychometric properties of the successful school leadership survey. Journal of Educational Administration. https://doi.org/10.1108/JEA-08-2022-0115 Masters, G. N. (1982). A Rasch model for partial credit scoring. Psychometrika, 47(2), 149–174. https://doi.org/10.1007/BF02296272 Robitzsch, A., Kiefer, T., & Wu, M. (2023). TAM: Test analysis modules (R package version 4.0-2). https://CRAN.R-project.org/package=TAM Seol, H. (2016). Using the Bootstrap Method to Evaluate the Critical Range of Misfit for Polytomous Rasch Fit Statistics. Psychological Reports, 118(3), 937–956. https://doi.org/10.1177/0033294116649434 Silva Diaz, J. A., Köhler, C., & Hartig, J. (2022). Performance of Infit and Outfit Confidence Intervals Calculated via Parametric Bootstrapping. Applied Measurement in Education, 35(2), 116–132. Academic Search Premier. Strobl, C., Kopf, J., & Zeileis, A. (2015). Rasch trees: A new method for detecting differential item functioning in the Rasch model. Psychometrika, 80(2), 289-316. Sun, J., & Leithwood, K. (2015). Direction-setting school leadership practices: a meta-analytical review of evidence about their influence. School Effectiveness and School Improvement, 26(4), 499–523. https://doi.org/10.1080/09243453.2015.1005106