Providing high-quality teaching to all students is crucial for educational equity (Shields & Mohan, 2008). One way to support teachers in achieving this, is by offering feedback on tasks and content-related aspect of quality, for example through using observation manuals (Bell et al., 2019). However, as observation manuals hold to different theoretical frameworks, accumulation of knowledge and evidence about the quality of tasks or the content can be hindered. Thus, understanding how different manuals conceptualize and understand teaching quality is critical, particularly when choosing a manual for a specific context. A potential solution is to systematically compare results of different observation manuals applied to the same classroom teaching episodes. In this study, we applied two observation manuals to the same mathematics lesson: one based on the framework of the three basic dimensions of teaching quality (TBD; Fauth et al., 2022) and the Protocol for Language Arts Teaching Observation (PLATO; Grossman, 2015). We aim to identify overlaps and differences between the manuals’ conceptualizations, and assessment of tasks and content-related aspects of teaching quality. We explore how they rely upon certain levels of analysis of tasks and content when examining teaching quality. In the TBD framework, tasks and content-related teaching quality is conceptualized through the cognitive activation dimension (Praetorius et al., 2018), encompassing teaching quality aspects such as “focus on key content” and “challenging tasks”. Meanwhile, the PLATO manual encompasses tasks and content-related aspects such as “intellectual challenge” and “representation of content”. The lesson was scored by certified raters for each manual. All aspects are scored on a four-point scale. While the TBD manual provides one score for the whole lesson, PLATO is scored in 15-minute segments. However, both manuals were used to score the whole lesson and 15-minute segments to enable comparisons. The findings show that despite building on different frameworks, both manuals scored tasks and content-related aspects of teaching quality similarly at an overarching level over the course of the lessons, suggesting similar overarching views of teaching quality. Raters also used similar events as evidence for scoring tasks and content-related aspects, despite different conceptualizations. While based on a small video sample, this study offers a starting point for more systematic comparisons of observation manuals in the future. These results can guide researchers and practitioners in choosing appropriate manuals for their context and help improve our understanding of the possibilities and limitations of different classroom observation frameworks and manuals.

Bell, C. A., Dobbelaer, M. J., Klette, K., & Visscher, A. (2019). Qualities of classroom observation systems. School Effectiveness and School Improvement, 30(1), 3-29. Fauth, B., Herbein, E., & Maier, J. L. (2022). Beobachtungsmanual zum Unterrichtsfeedbackbogen Tiefenstrukturen (2. aktualisierte Version) [Observation manual for the classroom feedback form deep structures (2. updated version)]. Institut für Bildungsanalysen Baden-Württemberg. Grossman, P. (2015). Protocol for language arts teaching observations (PLATO 5.0). Center to Support Excellence in Teaching (CSET). Stanford University Praetorius, A. K., & Charalambous, C. Y. (2018). Classroom observation frameworks for studying instructional quality: looking back and looking forward. ZDM, 50, 535-553. Praetorius, A.-K., Klieme, E., Herbert, B., & Pinger, P. (2018). Generic dimensions of teaching quality: The German framework of the three basic dimensions. ZDM, 50(3), 407-426. Shields, C. M., & Mohan, E. J. (2008). High‐quality education for all students: putting social justice at its heart. Teacher Development, 12(4), 289-300. https://doi.org/10.1080/13664530802579843