At last year’s ECER, we introduced the first English version of the “Hamburg Model of Teaching Practice and Learning Practice (HMLP)” (Augsdörfer/Casper 2018). We approached vocational teacher training at university and its perceived theory-practice-gap (Meijer, de Graaf, & Meirink, 2011; Cochran-Smith, 2005; Korthagen, 2010; Roness, 2011) in a design-based research project (DBR Collective, 2003; Reinmann, 2005). We were able to develop a working prototype of a video-based didactics seminar by referring to the HMLP as our theoretical framework. Here, learning practice is the focal point of any teaching/learning situation. In accordance with Biggs/Tang, we believe the desired level of teacher professionalization is an informed “focus on what the student does and how that relates to teaching. [This] is a student-centered model of teaching: the purpose of teaching is to support learning.” (Biggs/Tang, 2011, 20)

However, the standardized faculty survey we used to evaluate our prototype proved to be insufficient. In particular, it did not comprehensively assess our main question: How can learning settings be developed in order to tie practice and theory in meaningful, socially embedded learning practice? As shown in our last year’s contribution, our theoretical framework strongly refers to VET, work studies and higher education literature such as Wenger (2008), Schön (1983), Biggs/Tang (2011), Tramm/Casper (2018). Our research following ECER 2018 indicated that no existing evaluation instrument was fully in line with our ambitions and the pedagogical framework of the HMLP. Some established measures show a certain degree of compatibility (e. g. Kirkpatrick’s “Four Levels”, 2010) but these neither meet the VET-based theoretical roots nor the emancipatory ambitions of the HMLP.

Thus, a genuine short survey with 25 items was developed. It covers the five core concepts of the HMLP, "Teaching Practice", "Learning Practice", "(Internal) Learning Effect", “External Effects” and “Feedback”. Based on these items’ analysis, we developed a report card for teachers and lecturers, highlighting professionalization potentialities and formulating suggestions for reflection of the individual’s teaching practice. The instrument is available in both English and German and was successfully tested in several higher education settings at Universität Hamburg and Chulalongkorn University, Bangkok. That way, international applicability and generalization for non-VET/non-teacher-training settings was confirmed.

At ECER 2019, we want to present and discuss this new HMLP Survey, its theoretical framework and its test development up to date. We understand this paper as a direct follow-up to our last year’s contribution. In line with the design-based research paradigm, we assess how our prototype’s second iteration benefitted from evaluation and how this is reflected by our new survey instrument.

In general, this project follows a design-based research approach (DBR Collective, 2003; Reinmann, 2005). Iterations of didactical design and research on innovative teaching practice are understood as a continuous improvement process (CIP), alternating between phases of deductive design and inductive evaluation and generalization. Based on the design and development of last year’s prototype for a video-based VET teacher training seminar, new insights on criteria for quality education were identified and generalized. The prototype employed authentic video-taped lessons as an approach to seeing beyond the surfaces of teaching/learning situations. By reconstructing intentions, anticipations, psychological processes and planning-as-thinking/plan-as-theory, students of education could gain a deep pragmatic understanding. In order to assess the success of the prototype in specific and quality education in general, a new evaluation instrument was developed during the second iteration. It operationalizes the five core concepts of the HMLP, "Teaching Practice", "Learning Practice", "(Internal) Learning Effect", “External Effects” and “Feedback”. For each concept, four Likert-scale items were derived from the theoretical framework, with the standard Likert options “strongly disagree”, “disagree”, “neutral”, “agree”, “strongly agree”. A fifth open-text question was added for qualitative feedback for each concept. As an example, these are the items for the concept "(Internal) Learning Effect": • “I was able to follow individual interests and learning goals.” • “I am now more skilled than I was before.” • “The class challenged me to rethink some of my attitudes.” • “I gathered new knowledge, i.e. new concepts and contexts.” • “Please conclude on your personal learning effect:” In analysis, means are calculated for each item. Averages are calculated for each concept group and for the course total. In the teacher/lecturer report card, results which are lower than the total average minus one standard deviation (SD) are highlighted as “Improve?”s, while results higher than average plus one SD are highlighted as “Great!”s. That way, teachers get focused feedback relative to their individual norm, with selected areas of both best practice and improvement opportunities. For every item, suggestions for reflection are formulated, posing guiding questions and/or pointing to literature recommendations.

The HMLP survey was successfully developed in both English and German and piloted at Chulalongkorn University, Bangkok (20 responses, English, online version) and later in several courses at Universität Hamburg (56 responses, German, paper version). Until ECER 2019, we aim to generate at least twice the data for further test development. E. g., factor analysis will be carried out as soon as the necessary amount of data is available. With the given data, Cronbach’s Alpha for the five concepts was calculated and indicates sufficient internal consistency. Although the values for the English version were lower than for the German version, we attribute this to sample size limitation and expect better values for a larger sample. Sample teachers’ responses so far indicate a high usability and “catalytic validity” (Cohen et al. 2007, 139; Winter 2010, § 16): The report card, with its concise highlights and suggestions, induces change. It proves to be an eligible instrument for reflecting and promoting quality education. Considering the effects on learners of our prototype for video-based teacher training, the survey indicates a success. Eventually, the “Great!”s and “Improve?”s delivered by the HMLP Survey highlight both: its success and potential very well. For example, an outstanding agreement was achieved for “I am now more skilled than I was before.”, whereas a weaker (but still positive) point was “In exams/tests, I was able to show what I learned.” This marks a To Do for further development of the final exam. At the same time, it stresses that the course is outstandingly experienced as promoting pragmatic competencies, which was our main goal to start with. Qualitative responses underline this, especially regarding theory and practice “as necessarily linked concepts along a continuum of specifications and generalities, or cases and systems” (Augsdörfer/Casper 2018, 34).

Augsdörfer, A./Casper, M. (2018). Closing the theory-practice gap: Employing authentic video-taped lessons in vocational teacher education. In C. Nägele & B. E. Stalder (Eds.), Trends in vocational education and training research. Proceedings of the European Conference on Educational Research (ECER), Vocational Education and Training Network (VETNET) (pp. 33–43). https://doi.org/10.5281/zenodo.1319622 Benton, M. H./Casper, M./Karner, S./Tafner, G. (2016): Materialism, Subjective Happiness and Epistemic Beliefs of Students of Economics in Hamburg, Graz and Bangkok: A Cross Cultural Study and Discussion Regarding Economics Education. In: bwp@ Berufs- und Wirt-schaftspädagogik – online, Ausgabe 35, 1-20. Online: http://www.bwpat.de/ausgabe35/ benton_etal_bwpat35.pdf (13.12.2018). Biggs, J. B., & Tang, C. (2011). Teaching for quality learning at university. What the student does (4th ed.). Maidenhead: McGraw Hill Education & Open University Press. Cochran-Smith, M. (2005). Studying teacher education: What we know and need to know. Journal of Teacher Education, 56, 301–307. Cohen, L./Manion, L./Morrison, K. (2007): Research methods in education. London/New York. Design-based Research Collective. (2003). Design-based research: An emerging paradigm for educational inquiry. Educational Researcher 32(1), 5–8. Retrieved from http://www.designbasedresearch.org/reppubs/DBRC2003.pdf Kirkpatrick, D. L./Kirkpatrick, J. D. (2010): Evaluating training programs. The four levels. San Francisco: Berrett-Koehler. Korthagen, F. A. J. (2010). How teacher education can make a difference. Journal of Education for Teaching, 36, 407–423. Meijer, P. C./de Graaf, G./Meirink, J. (2011). Key experiences in student teachers’ development. Teachers and Teaching: Theory and Practice, 17, 115–129. Reinmann, G. (2005). Innovation ohne Forschung? Ein Plädoyer für den Design-Based Research-Ansatz in der Lehr-Lernforschung. Unterrichtswissenschaft, 33(1), 52–69. Roness, D. (2011). Still motivated? The motivation for teaching during the second year in the profession. Teaching and Teacher Education, 27, 628–638. Schön, D. (1983). The reflective practitioner: How professionals think in action. London: Temple Smith. Tramm, T., & Casper, M. (2018). Lernfeldübergreifende Kompetenzdimensionen: Ein gemeinsamer Gegenstand subjektorientierter curricularer Theorie und Praxis“. In T. Tramm, M. Casper, & T. Schlömer (Eds.), Didaktik der beruflichen Bildung (pp. 89–113). Bielefeld: Bertelsmann. Wenger, E. (2008). Communities of practice. Learning, meaning, and identity. Cambridge: University Press. Winter, R. (2010): Ein Plädoyer für kritische Perspektiven in der qualitativen Forschung. In: Forum Qualitative Sozialforschung / Forum: Qualitative Social Research 12 (1), Art. 7. Online: http://nbn-resolving.de/urn:nbn:de:0114-fqs110171 (04.01.2019)