Session Information
09 SES 02 B, Assessment of Instructions and Learning
Paper Session
Time:
2009-09-28
11:15-12:45
Room:
HG, Marietta- Blau-Saal
Chair:
Birgit Eickelmann
Contribution
Student ratings have been used for many years to evaluate the performance of teachers in their classes (Stronge 1997). Despite the increasingly widespread use of Students' Evaluations of Teaching, their validity and usefulness are not undisputed (Marsh 1987). Some research papers present data and interpretations suggesting that ratings are not highly correlated to student learning. Furthermore, it is argued that they might be biased by students' interest in the subject, their liking for their teachers, by grades and the teachers' grading leniency and other aspects (Greenwald 1997). Most studies concentrate only on a limited number of potential biases. At the same time they neglect factors that should in fact have an influence on ratings, namely the teaching behaviour of the instructors being evaluated. It seems indispensable to analyse potential biases in the context of teaching behaviour as some of them like interest in the subject and the students' liking for their teacher could be a result of good or bad teaching.
This paper explores to which extent global ratings of teachers by their students are influenced by teaching behaviour as well as by other (potentially biasing) factors. It is examined if some of these factors might be the result of good or bad teaching and therefore cannot be considered a bias.
Method
The analysis is based on a survey of 2121 students at Austrian Commercial Colleges who evaluated their Accounting teachers. The questionnaire contains nine items to measure global ratings of teachers which cover overall evaluations like “is a good teacher” and “I am satisfied with my teacher” as well as global ratings concerning the acquisition of knowledge, the stimulation of interest in Accounting and of motivation to study Accounting. 42 items cover different aspects of specific teaching behaviour (like “gives explanations that I can easily understand”, “gives concrete examples to explain”, “is friendly to students” or “gives fair grades”). A wide range of potentially biasing variables completes the questionnaire. The questionnaire was carefully tested in a pretest study with 522 students at Commercial Colleges before it was used in the current main study with 2121 students at Commercial Colleges from all over Austria.
Expected Outcomes
Structural equation models show that students' global ratings mainly depend on teaching behaviour as it is perceived by the students in class. Not only do the didactic aspects of teaching behaviour have the strongest direct effect on students' global ratings but also a considerable effect on students' interest in Accounting. Aspects of teaching behaviour that reflect teachers´ social competence have almost no direct effect on global ratings but a very strong effect on students´ liking for their teachers. Yet there is an additional effect of interest and of students´ liking for their teachers on global ratings that cannot be explained by teaching behaviour and that classifies both factors as biases. The effect of most additionally examined potential biases is small and not statistically significant.
Although student ratings are not completely unaffected by possible biases, they mainly reflect teaching behaviour in class, particularly the students' perception of their teachers´ didactic skills.
References
Greenwald, A. (1997): Validity Concerns and Usefulness of Student Ratings of Instruction. American Psychologist (November 1997), pp. 1182-1186. Marsh, H. (1987): Students' Evaluations of University Teaching: Research Findings, Methodological Issues, and Directions for Future Research. International Journal of Educational Research (November 1987), pp. 253-387. Stronge, J. (ed. 1997): Evaluating Teaching. A Guide to Current Teaching and Best Practice. Corwin Press, Thousand Oaks, California
Search the ECER Programme
- Search for keywords and phrases in "Text Search"
- Restrict in which part of the abstracts to search in "Where to search"
- Search for authors and in the respective field.
- For planning your conference attendance you may want to use the conference app, which will be issued some weeks before the conference
- If you are a session chair, best look up your chairing duties in the conference system (Conftool) or the app.