Editorial summary. This is our text summary of an article published by PLOS ONE. Charts, figures, and the author’s full voice are at the original — read it there .
Editorial verdict
Methodologically rigorous. The dataset is the largest institution-wide study of its kind, and the statistical controls are appropriate — the gender and cultural bias findings in SET scores are credible, though causality cannot be established from observational data alone.
Executive summary
This article addresses the presence of gender and cultural bias in student evaluations of teaching (SET) within higher education. The authors argue that SET scores — widely used in academic performance management and promotion decisions — are subject to systematic bias against female instructors and those from non-English speaking backgrounds. Drawing on 523,703 individual student surveys across five faculties at a large Australian public university (UNSW) over 2010–2016, covering 3,123 teachers and 2,392 unique courses, the study employs a cumulative ordinal regression model with random effects to control for course and teacher variation. Key findings include: female instructors from non-English speaking backgrounds face the strongest negative bias, with local male students in Science giving them odds of a higher score approximately 42% that of male English-speaking instructors; bias is substantially reduced in course evaluations (where teachers are not directly assessed), suggesting students evaluate the person rather than the teaching; and faculties with higher proportions of minority group teachers exhibit lower bias levels, suggesting a correlation between representation and reduced bias. The study concludes that the magnitude of these biases may render SET scores unreliable as measures of teaching effectiveness for performance management purposes.
Key insights
- 1Female instructors from non-English speaking backgrounds face the most severe bias in SET scores, with odds of receiving higher scores from local male students in Science approximately 42% those of male English-speaking instructors.
- 2Bias largely disappears in course evaluations (where students assess the course rather than the teacher), suggesting the effect reflects judgements about the person rather than teaching quality.
- 3Higher proportional representation of minority groups within a faculty is correlated (approximately r=0.5) with reduced bias in SET scores, indicating that workforce composition may influence evaluation outcomes.
- 4The magnitude of gender and cultural bias in SET scores can exceed the effect of a measurable teaching effectiveness proxy (first-time course teaching), particularly in the Business faculty.
- 5No significant difference in bias was detected between undergraduate and postgraduate students, suggesting the biases are culturally ingrained rather than context-specific to the university environment.
Practical takeaways
- Institutions relying on SET scores for promotion and performance decisions are, according to this study, using a measure subject to statistically significant gender and cultural bias that may disadvantage women and non-English speaking background staff.
- The correlation between faculty-level representation of minority groups and reduced bias levels indicates that workforce diversity may function as a structural moderator of evaluation bias in academic settings.
References
- Journal of the European Economic Association (2017).Gender bias in teaching evaluations.
- Journal of Public Economics (2017).Gender biases in student evaluations of teaching.
- ScienceOpen Research (2016).Student evaluations of teaching (mostly) do not measure teaching effectiveness.
- Innovative Higher Education (2015).What's in a name: Exposing gender bias in student ratings of teaching.
- IDEA Center, Kansas State University (2011).Student ratings of teaching: A summary of research and literature (IDEA Paper No. 50).
- ScienceOpen Research (2014).An evaluation of course evaluations.
- Cogent Education (2017).Student evaluations of teaching are an inadequate assessment tool for evaluating faculty performance.
- Australian Bureau of Statistics (2001).Australian Standard Classification of Education (ASCED).
Source & Provenance
PLOS ONE
Fan et al.
Not specified
Research Study
Asia-Pacific
Original source metadata is preserved. AI analysis is generated separately.
Like this? Get the Monday Decision Brief — free, every week.
No spam, unsubscribe anytime.
Rate this article
Want the full article? Read it at the original source — free, no paywall.
Read original article