A study published this month adds to the many concerns about judging faculty members using student evaluations of their teaching. But this one suggests that gender imbalances in departments exacerbate the issue.

Researchers found gender bias after analyzing Clemson University student evaluations of 1,885 tenure-track and non-tenure-track educators from academic year 2018–19. Studies have found gender bias before, but the new paper in Proceedings of the National Academy of Sciences describes a threat to both male and female faculty members’ futures in departments with lopsided numbers of men and women.

“In departments with gender disparities, those in the gender majority were evaluated more positively than those in the gender minority when teaching upper-level courses,” the authors wrote. “In contrast, those in the gender minority tended to be evaluated more positively than those in the gender majority when they taught lower-level courses, although not [statistically] significantly so. These patterns were not evident in departments nearer to gender parity.”

The researchers also did an experiment showing Clemson students fake faculty websites depicting male-dominated and female-dominated departments as well as those evenly divided between the two. They then asked students to predict which faculty members taught lower- or upper-level courses, imagine themselves having taken a class from the faculty members and then rate those professors’ teaching.

“In the male-dominated department condition, there was a significant two-way interaction between gender and course-level,” the researchers wrote. “Women received higher evaluations than did men for the 1,000-level course, and the inverse was true when considering the 4,000-level course.”

They wrote, “When expectations were met, we did not see any biases in teaching evaluations. However, when expectations were violated, we saw strong biases that were consistent with broader gender stereotypes. That is, women were penalized for filling the essential expert roles of teaching upper-level courses, and men were penalized for filling the interpersonal supportive roles of teaching lower-level courses.”

Women are nonetheless more likely to be harmed by this biased-against-both-genders phenomenon, the paper says.

“Because the majority of women held gender-minority status (72.6 percent of departments were male dominated), and most classes taught were upper-level courses (72.3 percent) the potentially negative impact of gender bias was greater for women than for men,” the authors wrote. “An estimated 32 percent of men and 52 percent of women at this university were potentially negatively impacted by gender bias in their teaching evaluations.”

Clemson spokespeople didn’t comment Tuesday.

Oriana Aragon, a University of Cincinnati assistant professor and the study’s lead author, said the study found these biases had small effects on evaluations, but that doesn’t make those effects unimportant.

“These effects are within that smaller range; they’re within a fraction of a point [on a five-point scale],” Aragon said. “But so, too, are the decisions being made.”

In explaining these findings—and the initial idea behind the study—the authors cite “role congruity theory.” Aragon said Alice Eagly, professor emerita at Northwestern University, developed the theory.

Drawing on it, Aragon explained that someone breaking gender expectations may not be seen as exceptional in a positive sense of that adjective. Take a woman in a firefighter’s role, Aragon said, and society “can see an incongruency of what they expect a woman to be and the traits of what that job requires.”

Citing previous research, the authors write that “the more masculine or feminine typed a workplace position is, the more the position is considered appropriate for only that gender. Thus, both men and women can be evaluated poorly when they violate an expected gender role.”

They write, “Role congruity research shows us that women can fill essential positions in female-dominated domains without being evaluated negatively. This contradicts what broader stereotypes would suggest, i.e., that men fill essential roles. In part, this contradiction can be explained by the fact that female-dominated fields such as nursing or early childhood education are viewed as more communal, that is interpersonal, nurturing and supportive. Therefore, women may be expected to fill essential positions in communal fields because women are seen as genuinely legitimate members of those fields. The same cannot be said for men. Men are penalized for taking on key positions in female-dominated fields.”

However, the authors don’t advise that colleges accept certain departments as dominated by either men or women. They cite a paper that says computer programmers were primarily women in the 1960s, and society viewed that profession differently than it does the male-dominated profession of today.

“It seems as though reaching gender parity is a way through which the cycle of gender bias can be reduced,” they write. “Until parity is achieved, departments might employ a strategy of ‘fake it until you make it’ by emphasizing the presence and achievements of both men and women within their departments. Indeed, our quick manipulation showed that even a department’s webpages could have a significant effect on how faculty are evaluated. Additionally, both male and female educators should teach lower and upper-level courses to help neutralize gender expectations.”

Also, again, until gender parity is achieved, the authors recommend that promotion and hiring committees be trained to watch out for this bias. They say universities could also provide “bias-corrected scores” and require evidence beyond semester-end teaching evaluations.

Brian Powell, another author of the study, said he’s served on a Clemson tenure and promotion review committee for six years. He said the idea for the research stemmed from a National Science Foundation ADVANCE workshop at Clemson. The NSF says ADVANCE aims to increase women’s representation and advancement in academic science and engineering careers.

“A lot of the messaging we’re looking forward to sharing with those tenure and promotion review committees is, when you’re doing the teaching evaluation, consider the environment in which that instructor is teaching,” Powell said, “and if they’re in a minority position in that environment.”