Observation systems are argued to enable comparable analyses of key aspects instructional quality across classrooms (Klette & Blikstad-Balas, 2018). However, contextual aspects including curriculum context, teacher beliefs, subject topic and student needs are not accounted for in the designs of observation systems, while they inevitable influence scores (Bell et al., 2012) and thus pose a threat of biased or irrelevant comparisons of instructional quality (Cohen & Goldhaber, 2016). This study explores how teachers intended lesson goals might serve as context-sensitive and complimentary lenses to better understand patterns of instructional quality. Three Finnish-Swedish lower secondary mathematics teachers’ video-recorded lessons (N=9) were coded with the Protocol for Language Arts Teaching Observation (PLATO) observation system. PLATO conceptualizes instructional quality across four domains; representation and use of content, instructional scaffolding, disciplinary demand and behavior management, divided into 12 elements that are each coded on a scale of 1-4 (Grossman, 2015). Teachers’ intended lesson goals were analyzed with a framework categorizing mathematics teachers’ goals into content-, competency-, affective-, utility-, student behavior-, competence related-, and other goals (Bergqvist et al., 2010), as well as inductively. Finally, the PLATO scores were contrasted with intended lesson goals. Main findings include that the teachers’ instruction highlighted different aspect of instructional quality, and their lesson goals to some extent explained these patterns. The teachers differed in their emphasis on competency goals, one underscoring mathematical thinking and communication while the two others were concerned with procedural fluency —reflected in high scores on different PLATO elements. In addition, all teachers stressed content goals, yet PLATO did not score content presentation highly, indicating that their content explanations have room for improvement. Finally, all teachers were concerned with affective goals (e.g., fostering self-confidence), student behavior goals (e.g., fostering study routines), and utility goals (e.g., using mathematical tools), none of which are covered by PLATO but underscored in the Finnish Curriculum (Finnish National Agency for Education, 2014), suggesting a potential mismatch between analytical foci and the classroom context. Identifying learning goals and other context-specific aspects may help researchers explain and nuance interpretations of instructional quality results. For formative professional development purposes, teacher intentions can facilitate targeting aspects of observation systems that cover context-relevant aspects of instruction. This is highly relevant in a European perspective, as observation systems are increasingly used across different contexts (OECD, 2020) while there is less discussion of how the results of such comparisons can be understood.

Bell, C. A., Gitomer, D. H., McCaffrey, D. F., Hamre, B. K., Pianta, R. C., & Qi, Y. (2012). An Argument Approach to Observation Protocol Validity. Educational assessment, 17(2-3), 62-87. doi:10.1080/10627197.2012.715014 Bergqvist, E., Bergqvist, T., Boesen, J., JHelenius, O., Lithner, J., Palm, T., & Palmberg, B. (2010). Matematikutbildningens mål och undervisningens ändamålsenlighet: Gymnasiet hösten 2009. Retrieved from http://ncm.gu.se/media/ncm/forskning/kunskapsoversikt_ncm_ufm_gy.pdf Cohen, J., & Goldhaber, D. (2016). Observations on evaluating teacher performance: Assessing the strengths and weaknesses of classroom observations and value-added measures. In J. A. Grissom & P. Youngs (Eds.), Improvig teacher evaluation systems: Making the most of multiple measures (pp. 8-21). NY, USA: Teachers College Press. Finnish National Agency for Education. (2014). Grunderna för läroplanen för den grundläggande utbildningen 2014. Helsinki, Finland: Finnish National Agency for Education Retrieved from https://www.oph.fi/sites/default/files/documents/grunderna_for_laroplanen_for_den_grundlaggande_utbildningen_2014.pdf Grossman, P. (2015). Protocol for Language Arts Teaching Observations (PLATO 5.0)In. Palo Alto: Stanford University. Klette, K., & Blikstad-Balas, M. (2018). Observation manuals as lenses to classroom teaching: Pitfalls and possibilities. European educational research journal EERJ, 17(1), 129-146. doi:10.1177/1474904117703228 OECD. (2020). Global Teaching InSights: A video study of teaching. Retrieved from Paris, France: https://www.keepeek.com//Digital-Asset-Management/oecd/education/global-teaching-insights_20d6f36b-en#page3