Course Evaluations: All Cost, No Benefit

Studies find racial, gender bias in student evaluations of faculty – The  Channels

The following is the result of a study by the Center for Educational Excellence at the University of San Diego–not my work–but I’m posting it and passing it along.

Known Issues with Student Evaluations of Teaching

• Student evaluations of teaching are not accurate measures of effective teaching (Esarey and Valdes 2020; Dunn, Hooks and Kohlbeck 2016; Hornstein 2017). They are measures of students’ experiences, which often centers on students’ concerns over their grades, not their learning (Becker, Bosshardt and Watts 2012).

• New meta-analyses show that students do not learn more from professors with higher student evaluation ratings (Uttle, White, and Gonzalez, 2017).
• Students who earn higher grades rate the course more favorably, and students who earn lower grades rate it less favorably (Wachtel 1998, Chambers and Schmitt 2002, Addison, et al 2006). This dynamic is called reciprocal leniency.
o Within the pattern of “reciprocal leniency,” among students who earn the same grades, those who find the class to be easier than they expected it to be rate the course more favorably, and students who found it harder than expected, rate it less favorably (Addison, et al 2006).
• Nonverbal behaviors matter much more to student evaluations than effective teaching. If the faculty member’s voice, facial expressions, mannerisms and demeanor are interpreted as warm and appealing, students view the instructor as knowledgeable and competent. This leads to lower ratings for faculty from a cultural background a student is unfamiliar with; who speak with an accent; who are perceived to hold a sexual identity a student is uncomfortable with; who have asymmetrical facial features (which comes off as worried, careless or unsympathetic), etc. (Merritt 2008).
• Attractiveness of the faculty and students’ interest in the class are the top factors associated with high scores on teaching evaluations (Murray, et al. 2020).

Gender Bias

We risk institutionalizing sexism when we use student evals in merit and promotion decisions.

• Women faculty are perceived as less competent (Baldwin and Blattner 2003). They are more likely to be referred to as “teachers” in student evaluations whereas men are more likely to be referred to as “professors” (Miller and Chamberlin 2000).

  • Students expect women faculty to be more accessible and more nurturing (Merritt 2008).
  • Gender biases against women faculty are large enough to cause more effective instructors to get lower student ratings than less effective instructors (Boring, Ottoboni and Stark 2016).

• In an identical online course, students rate male instructors more favorably, and they rated the course content more favorably (Mitchell and Martin 2018). This happens even when the students never see the instructor—but half the online class is given a female name for the professor and half are given a male name (MacNell, Driscoll and Hunt 2015).

• Female instructors are more likely to have their appearance and personality commented on in student evaluations compared to male instructors (Mitchell and Martin 2018).

Racial Bias

We risk institutionalizing racism when we use student evals in merit and promotion decisions. • Faculty of color receive lower course evaluations than their White counterparts

(Hammermesh and Parker 2005; Littleford, et al. 2010; Wallace, Lewis, and Allen 2019). 1

Center for Educational Excellence University of San Diego Fall 2020

• Students rate Black professors as less competent than their White and Asian counterparts (Ho, Thomsen, and Sidanius 2009), even before students met the professors (Bavishi, Madera and Hebl 2010).
• Black faculty (along with women) less likely to be called “brilliant” or “genius” (Storage, et al. 2016).

• Faculty tend to be rated more highly when they are “young, male, White, in the Humanities, and hold a rank of full professor” (Murray, et al. 2020).
• Faculty of color often teach “race-focused” courses and students who experience discomfort with concepts such as white privilege rate such courses lower as a way to express their frustration with the challenging content (Littleford, et al. 2010).

• Students’ gender and racial biases put women faculty of color in a “double jeopardy” because students intensely question their competence and authority (Laube, et al. 2007). Women faculty of color are more likely to be perceived as “hostile and uncaring” (Merritt 2008). This gives white males a double advantage in terms of student bias (Wallace, Lewis and Allen 2019).

Bias Against Nonnative English Speakers

• Asian mathematics instructors are evaluated lower and described by students in subtle ways that “reproduce dominant language ideology” (Subtirelu 2015).
• Mentioning an accent was one of three factors most strongly associated with low evaluation scores for tenure-track faculty (course difficulty was the strongest), meanwhile attractiveness of the faculty and students’ interest in the class were most strongly associated with high scores (Murray, et al. 2020).

• Undergraduates hold negative perceptions of international instructors, particularly TAs, before they even meet them, which can negatively affect teaching evaluations. The possession of a foreign accent “often impacts their perceived competence and effectiveness in the classroom” (Adebayo 2019; see also Manohar& Appiah 2016).

Additional Problems

• Quantitative courses are rated less favorably than other, non-quantitative courses, on average (Beran and Violato 2005, Centra 2009).
• Professors who teach quantitative courses are far more likely NOT to receive tenure, promotion and/or merit pay, and teaching awards when their performance is evaluated against common standards. They are at a substantially higher risk of being labeled “unsatisfactory” in teaching (Uttle and Smibert 2017).

• Higher level elective courses receive higher evaluations than lower level required courses (Zabaleta 2007) and courses that are “feared or despised” receive harsher ratings (Gray and Bergmann 2003).
• An early morning course can receive lower ratings than an identical course an hour later (Tobin 2017).

• Faculty who are perceived as physically attractive receive higher ratings (Freg and Webber 2009; Hammermesh and Parker 2005; Murray et al 2020).

• Extroverted instructors are rated more favorably (Gray and Bergmann 2003; Radmacher and Martin 2001).

References

Addison, William E., Best, John, and Warrington, John D. 2006. “Students’ Perceptions Of

Course Difficulty and Their Ratings of The Instructor.” College Student Journal, 40(2).

Adebayo, Comfort Tosin. 2019. “Teaching Perspective Taking in Intercultural Contexts: Encounters with International Teaching Assistants with Foreign Accents” Communication Teacher 34(1): 47-52.

Baldwin, T. and N. Blattner. 2003. “Guarding against Potential Bias in Student Evaluations: What Every Faculty Member Needs to Know.” College Teaching 51 (1):27–32.

Becker, William E., William Bosshardt, and Michael Watts. 2012. “How Departments of Economics Evaluate Teaching.” The Journal of Economic Education 43(3): 325-333.

Beran, Tanya and Claudio Violato. 2005. “Ratings Of University Teacher Instruction: How Much Do Student And Course Characteristics Really Matter?” Assessment & Evaluation in Higher Education, 30:6, 593-601.

Boring, Anne, Kellie Ottoboni, and Philip B. Stark. 2016. “Student Evaluations Of Teaching (Mostly) Do Not Measure Teaching Effectiveness.” Science Open Research DOI: 10.14293/S2199-1006.1.SOR-EDU.AETBZC.v1

Centra, John A. 2009. “Differences In Responses To The Student Instructional Report: Is It Bias?” Princeton, NJ: Educational Testing Service.

Chambers, B. A., & Schmitt, N. 2002. “Inequity in the performance evaluation process: How you rate me affects how I rate you.” Journal of Personnel Evaluation in Education, 16:103-112.

Dunn, Kimberly A., Hooks, Karen L., and Kohlbeck, Mark J. 2016. Preparing Future Accounting Faculty Members To Teach. Issues in Accounting Education 31: 155–170.

Esarey, Justin and Natalie Valdes. 2020. “Unbiased, reliable, and valid student evaluations can still be unfair. ”Assessment & Evaluation in Higher Education, DOI: 10.1080/02602938.2020.1724875

Freng, S., and D. Webber. 2009. “Turning up the Heat on Online Teaching Evaluations: Does “Hotness” Matter?” Teaching of Psychology 36 (3):189–93.

Gray, Mary, and Barbara R. Bergmann. 2003. “Student Teaching Evaluations: Inaccurate, Demeaning, Misused.” Academe 89 (5):44.

Hammermesh, Daniel and Amy Parker. 2005. “Beauty in the classroom: instructors’ pulchritude and putative pedagogical productivity.” Economics of Education Review 24: 369-376.

Center for Educational Excellence University of San Diego Fall 2020

Ho, A. K., L. Thomsen, and J. Sidanius. 2009. “Perceived Academic Competence and Overall Job Evaluations: Students’ Evaluations of African American and European American Professors.” Journal of Applied Social Psychology 39 (2):389–406.

Hornstein, Henry A. 2017. “Student Evaluations Of Teaching Are An Inadequate Assessment Tool For Evaluating Faculty Performance” Cogent Education 4:1.

Laube, H., Massoni, K., Sprague, J., and A. L. Ferber. 2007. “The Impact of Gender on the Evaluation of Teaching: What We Know and What We Can Do.” NWSA Journal 19:87–104.

Littleford, L. N., K. S. Ong, A. Tseng, J. C. Milliken, and S. L. Humy. 2010. “Perceptions of European American and African American Instructors Teaching Race- Focused Courses.” Journal of Diversity in Higher Education 3(4):230–44.

MacNell, L., Driscoll, A. and Hunt, A.N. 2015. “What’s in a Name: Exposing Gender Bias in Student Ratings of Teaching.” Innovative Higher Education 40: 291–303.

Merritt, Deborah J. 2008. “Bias, the Brain, and Student Evaluations of Teaching.” St. John’s Law Review 82 (1):235–87.

Miller, JoAnn, and Marilyn Chamberlin. 2000. “Women Are Teachers, Men Are Professors: A Study of Student Perceptions.” Teaching Sociology 28(4):283–98.

Mitchell, Kristina MW and Jonathan Martin. 2018. “Gender Bias In Student Evaluations.” PS: Political Science & Politics 51(3): 648-652.

Manohar, Uttara and Osei Appiah. 2016. “Perspective Taking to Improve Attitudes Towards International Teaching Assistants: The Role of National Identification and Prior Attitudes” Communication and Education 65(2): 149-163.

Murray, Dakota, Clara Boothby, Huimeng Zhao, Vanessa Minik, Nicolas Bérubé, Vincent Larivière, Cassidy R. Sugimoto. 2020. “Exploring the Personal and Professional Factors Associated with Student Evaluations of Tenure-Track Faculty” Plos One https://doi.org/10.1371/journal.pone.0233515

Radmacher, Sally A., and David J. Martin. 2001. “Identifying Significant Predictors Of Student Evaluations Of Faculty Through Hierarchical Regression Analysis.” The Journal of Psychology 135(3): 259-268.

Storage, D., Z. Horne, A. Cimpian, and S. J. Leslie. 2016. “The Frequency of ‘Brilliant’ and ‘Genius’ in Teaching Evaluations Predicts the Representation of Women and African Americans across Fields.” PLoS One 11 (3):e0150194–17.

Subtirelu, Nicholas Close. 2015. “She Does Have an Accent but…: Race and Language Ideology in Students’ Evaluations of Mathematics Instructors on RateMyProfessors.com” Language in Society. 44(1): 35-62.

Tobin, Roger G. 2017. “Too Early For Physics? Effect of Class Meeting Time on Student Evaluations of Teaching in Introductory Physics.” The Physics Teacher 55(5): 276-279.

Uttl, Bob, Carmela A. White, and Daniela Wong Gonzalez. 2017. “Meta-Analysis Of Faculty’s Teaching Effectiveness: Student Evaluation Of Teaching Ratings And Student Learning Are Not Related.” Studies in Educational Evaluation 54 p 22-42.

Uttl, Bob and Dylan Smibert. 2017. “Student Evaluations Of Teaching: Teaching Quantitative Courses Can Be Hazardous To One’s Career.” PeerJ 5:e3299 https://doi.org/10.7717/peerj.3299

Wallace, Sherri L., Angela K. Lewis, and Marcus D. Allen. 2019. “The State of the Literature on Student Evaluations of Teaching and an Exploratory Analysis of Written Comments: Who Benefits Most?” College Teaching, 67(1): 1-14.

Wachtel, H. K. 1998. “Student evaluation of college teaching effectiveness: A brief review.” Assessment and Evaluation in Higher Education, 23: 191-211.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s