Student Evaluation of Teaching (SET) is a method of teaching evaluation critical to enhancing an instructor’s ability to improve teaching, it is crucial to career success, but it is reportedly biased against women (American Sociological Association, 2019; Stark & Freishtat, 2014). Women tend to accumulate less human capital than men due to the tension between family obligations and work ambitions. Fewer human capital accumulation implies slower career advancement (Bertrand, 2017). Therefore, a fair teaching evaluation could reveal differences (if any) between men and women in teaching performance. On the contrary, biased teaching evaluation could affect women’s access to academic positions and promotion, contributing to the “glass ceiling effect,” women’s lack of access to better wages, power, and opportunities than men. 

A few literature summaries conclude that students favor male instructors due to a gender bias in SET (American Sociological Association, 2019; Stark & Freishtat, 2014). However, studies report mixed evidence of differences in SET scores by gender (Spooren et al., 2017). A major problem affecting the interpretation of findings relates to the limitations of the observational study design. The observational/correlational design do not measure instructors’ gender bias but the difference between male and women. Observational studies report a difference between male and female teachers between instructors with different levels of teaching ability (Centra, 2003; Haladyna & Hess, 1994; MacNell et al., 2015; Marsh, 1987), for instance, due to initial differences and training.

A more effective way of revealing the instructor’s gender effects on SET is through experimentation. Experiments manipulate instructors’ gender and therefore compare teachers who are only different in that attribute. Thus, experimental studies isolate the instructor’s gender effect on SET from potential confounders such as age, race, subtle differences in teaching delivery, or students’ enrollment characteristics across classrooms. 

Reviews of empirical research on instructors’ gender bias in SET summarize experimental and observational studies without addressing the severe limitations of the latter design. This study aims to comprehensively understand gender differences in faculty teaching evaluation by systematically reviewing experimental and quasi-experimental gender-bias studies in SET. Our research questions are:

1) What are the reasons or mechanisms explaining instructors’ gender differences in SET literature?

2) What does experimental and quasi-experimental peer-reviewed research say about instructors’ gender differences in SET?

3) What are the conclusions/recommendations from peer-reviewed experimental and quasi-experimental research about SET instructors’ gender bias?

A systematic literature review is a comprehensive and transparent synthesis of a relevant phenomenon. We conducted a systematic literature review following various guidelines published in the literature (Alexander, 2020; Pigott & Polanin, 2020; Polanin et al., 2019). The stages are the development of systematic review questions (these are presented in section 1), identifying our eligible studies, data collection, which comprises the literature search and screening, and analysis and interpretation. Following the UTOS framework (Units, Treatments, Outcomes, and Study design), the scope of our systematic literature review is: 1) empirical studies in higher education institutions with students as units of analysis, 2) featuring a manipulation of the instructor’s gender as the treatment, 3) using SET or equivalent as outcomes, and 4) featuring either a randomized controlled trial or a quasi-experimental design. We further narrowed the type of publications using the inclusion criteria: peer-reviewed, published in academic journals, and published in English between 2000 and 2021. We identified studies using two strategies: database search and snowballing. We included two multidisciplinary databases (EBSCO and ProQuest Central) and one education-specialized database (ERIC) in the database search. The search keywords are: “student evaluation of teaching,” “gender,” “bias,” and “experiment.” The database search occurred in July 2021. The number of articles found by each search engine was: 8.510 publications in ProQuest Central, 82.367 in EBSCO, and 39.527 in ERIC. We kept the predefined search engine priority sorting, displaying pages with 100 results each. The results from each search engine were title screened and saved using the blocks of 100 results. The title screening looked at compliance with the UTOS and inclusion criteria. We stopped keeping search results after a group of 100 articles contained no relevant publication. The snowballing strategy involves identifying relevant studies from one or a few seminal articles. Specifically, we employed the study by MacNell, Driscoll, & Hunt (2015) that matches the scope and inclusion criteria of our literature review. We used backward snowballing and forward snowballing. The search produced a total of 1153 studies for screening. We run an independent double-blind screening process for each title/abstract, reaching a Brennan & Prediger agreement coefficient between .62 and .98 for every round of 300-350 studies. We solved our inconsistencies through discussion and consensus among the principal researcher and research assistants. Finally, we identified 24 studies for full paper review.

Studies report either no theory or at least one of two psychological theories to explain why students may favor men in SET. The first leading theory is gender stereotypes. These culturally determined beliefs define attributes inherent to women and men. Gender stereotypes often define men as assertive and ambitious and women as altruists and sensitive (Chávez y Mitchell, 2019). According to theory, students automatically believe that men instructors are more stringent and colder, while female instructors are warm and altruistic regardless of their actual level of warmth (Anderson, 2010). Thus, differences in teaching evaluation by gender favor men when a SET comprises an aspect of teaching related to competence and brilliance, and SET may favor women when a SET contains aspects targeting care (Arbuckle y Williams, 2003). A smaller group of studies explain SET gender differences due to the gender expectations violations theory (Burgoon, 1995). Instructors with behaviors that depart from the expectation (stereotype) are often punished in their SET (Anderson, 2010). In addition, men are less severely punished when they challenge a gender expectation than when women do. As a summary, 10 out of 24 studies report a statistically significant effect of gender, with students favoring men in SET overall score or SET subscales (on occasions, single item responses); 6 out of 24 studies report no gender effect and no gender bias; and eight studies report mixed results favoring women in various attributes of teaching. Summaries of gender differences in SET suggest that these questionnaires should be banned entirely (American Sociological Association, 2019) or that SET is resilient to bias (Marsh, 2007). A compelling argument about the presence or lack of gender bias in SET requires a comprehensive review of the best available empirical research. Unfortunately, to our knowledge, no such systematic review exists thus far.

Alexander, P. A. (2020). Methodological Guidance Paper: The Art and Science of Quality Systematic Reviews. Review of Educational Research, 90(1), 6–23. https://doi.org/10.3102/0034654319854352 American Sociological Association. (2019, September 9). Reconsidering Student Evaluations of Teaching. American Sociological Association. https://www.asanet.org/press-center/press-releases/reconsidering-student-evaluations-teaching Bertrand, M. (2017). The Glass Ceiling. SSRN Electronic Journal. https://doi.org/10.2139/ssrn.3191467 Centra, J. A. (2003). Will Teachers Receive Higher Student Evaluations by Giving Higher Grades and Less Course Work? Research in Higher Education, 44(5), 495–518. https://doi.org/10.1023/A:1025492407752 Haladyna, T., & Hess, R. K. (1994). The detection and correction of bias in student ratings of instruction. Research in Higher Education, 35(6), 669–687. https://doi.org/10.1007/BF02497081 MacNell, L., Driscoll, A., & Hunt, A. N. (2015). What’s in a Name: Exposing Gender Bias in Student Ratings of Teaching. Innovative Higher Education, 40(4), 291–303. https://doi.org/10.1007/s10755-014-9313-4 Marsh, H. W. (1987). Students’ evaluations of University teaching: Research findings, methodological issues, and directions for future research. International Journal of Educational Research, 11(3), 253–388. https://doi.org/10.1016/0883-0355(87)90001-2 Marsh, H. W. (2007). Students’ Evaluations of University Teaching: Dimensionality, Reliability, Validity, Potential Biases and Usefulness. In R. P. Perry & J. C. Smart (Eds.), The Scholarship of Teaching and Learning in Higher Education: An Evidence-Based Perspective (pp. 319–383). Springer Netherlands. http://link.springer.com/10.1007/1-4020-5742-3_9 Pigott, T. D., & Polanin, J. R. (2020). Methodological Guidance Paper: High-Quality Meta-Analysis in a Systematic Review. Review of Educational Research, 90(1), 24–46. https://doi.org/10.3102/0034654319877153 Polanin, J. R., Pigott, T. D., Espelage, D. L., & Grotpeter, J. K. (2019). Best practice guidelines for abstract screening large‐evidence systematic reviews and meta‐analyses. Research Synthesis Methods, 10(3), 330–342. https://doi.org/10.1002/jrsm.1354 Spooren, P., Vandermoere, F., Vanderstraeten, R., & Pepermans, K. (2017). Exploring high impact scholarship in research on student’s evaluation of teaching (SET). Educational Research Review, 22, 129–141. https://doi.org/10.1016/j.edurev.2017.09.001 Stark, P., & Freishtat, R. (2014). An Evaluation of Course Evaluations. ScienceOpen Research. https://www.scienceopen.com/document/id/ad8a9ac9-8c60-432a-ba20-4402a2a38df4