This research addresses the issue of assessing schoolteachers’ knowledge on teaching practices, specifically focusing on the challenges associated with self-report measures. Although the educational community acknowledges the value of practices such as 'Blended learning' and 'Flipped classroom,' there are still gaps in evaluating teacher actual knowledge regarding these methods. Current measures encounter difficulties with open-answer formats, social desirability bias, and diverse contexts. This study suggests using the overclaiming technique (OCT) as a novel measurement tool to improve the reliability of teachers' answers.

As a part of the bigger study, I have conducted a survey with pre-service and in-service teachers in Austria (pilot phase), using OCT to assess their familiarity, detailed knowledge, and application of both real and fake teaching practices. The preliminary results indicate that 63% of participants do not overclaim, while 37% admit to having knowledge or using at least one fake practice. Logistic regression models suggest that teachers who are already working and those with higher self-efficacy are more likely to overclaim. 

Introduction and theoretical background

For decades, teacher practices and teaching methods in schools such as “Blended learning” or “Flipped classroom” have been the focus of attention. Based on a growing number of experimental studies, scientific community understands that these various practices and methods are valuable for teaching quality (Freeman et al., 2014; Karabulut‐Ilgu et al., 2018). Hence, there is a need to teach future teachers these practices in higher education institutions (Kunter et al., 2013). However, effectiveness of teacher education programs regarding specific teaching methods and practices remains under researched (Rowan et al., 2002). Studies showed that students in teacher training do obtain knowledge to a certain extent, but there is still a large room for improvement (Wilson et al., 2010). If researcher aim to gain insights in these improvements, they are faced with the challenge of how to assess knowledge gain and retention about teaching practices. Ideally such an analysis requires mass observations during lessons, which is usually too expensive or very time consuming. I propose an innovative measurement instrument, that overcomes shortcomings of existing measures on teaching practices in order to make teachers’ questionnaires more reliable.

The existing self-report measures on almost any topic (e.g., in large-scale questionnaires) pose several major challenges (Dunning et al., 2004). First, open answer formats would be too much of a work to analyse (especially regarding teaching practices). Second, in a multiple-choice format, teachers may report that they know or apply a mentioned practice regardless of truth. This challenge follows indirectly from evidence on social desirability and social pressure experienced by teachers in most countries (OECD, 2014). Third, teachers’ answers may vary drastically because of various education system’s characteristics, for example, teacher’s salary, duration of training, teacher’s position in society or working conditions. Summing up these challenges, a reasonable question arises: How the use of different teaching practices can be assessed in the most unbiased and cost-effective way possible?

As a first step in this direction, I propose to improve the existing survey procedure in a way which will allow researchers to better assess knowledge on the pedagogical practices and to cope with the potentially exaggerated self-report in teacher surveys. To reach this goal, I suggest to apply overclaiming technique (OCT) — a procedure used in other areas of education research such as ICT knowledge of school students (Vonkova et al., 2021). The use of this procedure will allow to move forward not only in an approximate understanding of the situation of mastering various teaching practices, but also in the issue of teachers’ group differences (and potential mechanisms behind them) in terms of the desire to exaggerate their knowledge.

In spring 2023, an online questionnaire was distributed to students in teacher education from four universities and five university colleges of teacher education in Austria. After data cleaning procedures we ended up with 1,643 responses which are the basis for our study. Of these participants, 67% were university students, 75% female, and 61% Bachelor students. The students were on average 26 years old (SD = 6) and 38% taught at a school during their studies. While 41% worked other jobs, 21% did not work during their studies. Of the entire sample, 40% of participants were from families in which parents had higher education. At the moment, I have data only on Austrian pre-service and in-service teachers from the pilot phase (N=1643) collecting information about the teaching practices used in everyday schoolwork, including both existing and non-existing practices (applying previously mentioned OCT). Three checkboxes for each practice are used to determine whether teachers are familiar with the practice, know it in detail, and whether they apply the practice in their work. The questionnaire includes a total of 12 practices (6 real ones and 6 fake) + sociodemographic data, several scales (self-efficacy, motivation towards teaching, well-being) and data about the school. The analysis consists of descriptive statistics in regard to the presence of fake practices in teachers' answers and a set of logistic regressions to address relational question about predictors of overclaiming behavior.

My goals are to explore integrity of teachers’ answers about practices (i.e. assessed via self-report measures); identify predictors of teachers’ integrity, including group differences, connection with their working status, socio-economic status, etc. First results suggest that 63% of the sample do not overclaim at all, i.e. do not choose any of fake practices. 37% say that they know at least one fake practice, 19% — that they know how to use at least one, and 12% — that they use at least one. For now, two main predictors of overclaiming in logistic regression models are a) “already working as a teacher” status; b) level of self-efficacy; с) teacher's sex. Other variables (such as well-being, type of educational institution or level of studies are not significant. I am going to build discussion around deeper analysis of subsamples and consideration for the future research on the matter. The study aims to explore predictors of overclaiming, considering group differences and connections with teachers' working status and socio-economic factors. The findings contribute to understanding the integrity of self-reported data on teaching practices. This has implications for refining teacher education programs and guiding future research in this domain.

References Dunning, D., Heath, C., & Suls, J. M. (2004). Flawed Self-Assessment: Implications for Health, Education, and the Workplace. Psychological Science in the Public Interest : A Journal of the American Psychological Society, 5(3), 69–106. https://doi.org/10.1111/j.1529-1006.2004.00018.x Freeman, S., Eddy, S. L., McDonough, M., Smith, M. K., Okoroafor, N., Jordt, H., & Wenderoth, M. P. (2014). Active learning increases student performance in science, engineering, and mathematics. Proceedings of the National Academy of Sciences of the United States of America, 111(23), 8410–8415. https://doi.org/10.1073/pnas.1319030111 Karabulut‐Ilgu, A., Jaramillo Cherrez, N., & Jahren, C. T. (2018). A systematic review of research on the flipped learning method in engineering education. British Journal of Educational Technology, 49(3), 398–411. https://doi.org/10.1111/bjet.12548 Kunter, M., Klusmann, U., Baumert, J., Richter, D., Voss, T., & Hachfeld, A. (2013). Professional competence of teachers: Effects on instructional quality and student development. Journal of Educational Psychology, 105(3), 805–820. https://doi.org/10.1037/a0032583 OECD. (2014). Report on Social Desirability, Midpoint and Extreme Responding in TALIS 2013: Education Working Papers. https://doi.org/10.1787/5jxswcfwt76h-en Rowan, B., Correnti, R., & Miller, R. J. (2002). What Large-Scale, Survey Research Tells Us about Teacher Effects on Student Achievement: Insights from the Prospects Study of Elementary Schools. Teachers College Record: The Voice of Scholarship in Education, 104(8), 1525–1567. https://doi.org/10.1111/1467-9620.00212 Vonkova, H., Papajoanu, O., Stipek, J., & Kralova, K. (2021). Identifying the accuracy of and exaggeration in self-reports of ICT knowledge among different groups of students: The use of the overclaiming technique. Computers & Education, 164, 104112. https://doi.org/10.1016/j.compedu.2020.104112 Wilson, C. D., Taylor, J. A., Kowalski, S. M., & Carlson, J. (2010). The relative effects and equity of inquiry‐based and commonplace science teaching on students' knowledge, reasoning, and argumentation. Journal of Research in Science Teaching, 47(3), 276–301. https://doi.org/10.1002/tea.20329