Empirical studies have repeatedly shown that teaching quality is positively associated with students' achievement and motivation (e.g., Praetorius et al., 2018). Thus, within teacher training and education, a variety of assessment tools is used to provide (pre-service) teachers with feedback on their teaching in class. However, the instruments vary regarding their quality and there is no common theoretical and empirical basis for formative assessment across different types of schools. Consequently, a feedback form for external observations was developed and implemented as part of a broad initiative in the federal state of Baden-Württemberg in Germany. To receive an observation form that is both successfully implementable and effective, this project follows a stepwise approach of development and evaluation (Humphrey et al., 2016), using a symbiotic implementation strategy (see Parchmann et al., 2006). First, an observation form that enables formative external ratings of and feedback on teaching quality was developed. The eleven-item form is based on empirical findings and established instruments. To promote an effective and efficient implementation in school practice, an accompanying support system was supplemented, including a manual, subject-specific observation guidelines, and a training program. Second, we conducted a pilot study to examine different facets of implementation (e.g., relevance, utility, feasibility) of all developed components, as well as the reliability of the observation form. Ten experts in teacher training/teacher education participated in the study and received the training program. We assessed the facets of implementation via questionnaires at repeated measurement points during the training. To assess interrater reliability, videos showing math classes were rated before and after the training. Furthermore, we embedded repeated discussion rounds, focusing e.g., on the comprehensibility and feasibility of the form and manual, to support the co-operative and symbiotic process. The results of the questionnaires, which will be available in January, and the joint discussions are used to adapt the observation form, the manual, and the training. Third, an effectiveness study follows in April. Part of this study is again the examination of the implementation of the revised observation form and support system, and the assessment of the interrater reliability. Forty teacher trainers will participate. To take a step further into practice, the training program is offered by the trained participants of the pilot study. In the talk, we present the results concerning implementation and interrater reliability from the nearly finished pilot and the subsequent effectiveness study, as well as further implications for adjustments.

Humphrey, N., Lendrum, A., Ashworth, E., Frearson, K., Buck, R., & Kerr, K. (2016). Implementation and process evaluation (IPE) for interventions in education settings: A synthesis of the literature. Education Endowment Foundation. https://educationendowmentfoundation.org.uk/public/files/Evaluation/Setting_up_an_Evaluation/IPE_Review_Final.pdf Parchmann, I., Gräsel, C., Baer, A., Nentwig, P., Demuth, R., & Ralle, B. (2006). “Chemie im Kontext”: A symbiotic implementation of a context‐based teaching and learning approach. International Journal of Science Education, 28(9), 1041–1062. https://doi.org/10.1080/09500690600702512 Praetorius, A.-K., Klieme, E., Herbert, B., & Pinger, P. (2018). Generic dimensions of teaching quality: The German framework of Three Basic Dimensions. ZDM, 50(3), 407–426. https://doi.org/10.1007/s11858-018-0918-4