Time to re-evaluate

[adrotate banner="13"]

[feather_share show="twitter, google_plus, facebook, reddit, tumblr" hide="pinterest, linkedin, mail"]

It’s the time of year when the tables are turned and students are asked to evaluate professors.

But what many students don’t know is just how important course evaluations are at McMaster. The Silhouette investigated the complex world of course evaluations, including the research showing systemic gender bias by students in general and some of the big changes coming including the publication of course evaluations results on the student portal in Mosaic.

McMaster is unique in many ways that don’t make university rankings.

Our faculty is non-unionized, meaning faculty are represented through the McMaster University Faculty Association.

Our institution also prides itself on its investment in teaching and learning.

McMaster’s commitment to teaching has been reinforced with the Strategic Mandate Agreement and Patrick Deane’s letter to the community, Forward with Integrity.

“I think [the SMA and FWI highlighting the importance of teaching] is sending a message to the community, which I think Patrick probably intended to do, to students that teaching is important to us and that they’re important and we take them seriously. It’s also a message to professors that the university takes teaching seriously,” said Rafael Kleiman, president of MUFA.

One way this commitment to teaching manifests is in the weight accorded to teaching evaluation. Teaching evaluation is comprised of peer evaluation and student evaluation.

As the policy on assessment of teaching outlines, factors range from teaching awards given to answers to feedback surveys.

One example of these is the teaching awards by the McMaster Student Union.

“The way teaching awards work is that you nominate your professor and once we receive an ‘x’ number of nominations it triggers an evaluation in that class, and that’s the score we use to consider who gets the award,” said Mina Karabit, coordinator of the MSU Teaching Awards Committee 2014-2015.

Both the Teaching Awards Committee and the student evaluation of professors face the same problem: participation rates.

For course evaluations, student participation rates have been extremely low since McMaster switched from paper to online evaluations.

The Teaching Awards Committee experienced a similar problem when they piloted an online only model for evaluations of teaching assistants.

When they go to classes with paper forms, they usually receive a 100 percent response rate. However, when they tried doing this online, one tutorial only had a 22.5 percent response rate.

However, Karabit acknowledges the many advantages of online evaluations. “There’s so many advantages to putting it online, it takes less time and you can do a lot more analytic stuff with the data a lot faster.”

Given the weight of teaching evaluations in tenure, promotion, and salary, the low response rate is concerning.

“One of the biggest concerns that faculty see is the low response rate,” said Kleiman. “If the response rate is low, it’s hard to know if it’s representative.”

Karabit says if reviews were done midterm, like some professors currently do, students might be more engaged.

“I’ve had a couple of courses where profs are now introducing their own version of online evaluations halfway through the term, as a means to gauge how the first half went and then changing it up for the second half if need be, or if the evals say everything is great then keep going. That’s just a professor-driven incentive, but it was really cool because at that point I think the class was more engaged because we felt we could actually make a possible change for the second half and the professor actually cares.”

Given the low response rate, it is unsurprising that the Provost has struck a teaching evaluation committee, led by the AVP Faculty Susan Searls-Giroux. MSU VP Education Rodrigo Narro Perez represents students on this committee, and the incoming VP Education will replace him in the spring.

Among many questions, the committee is examining the gendered bias that has been documented in the literature.

Studies have shown that female professors are ranked more harshly by students than their male counterparts.

Lilian MacNell is one of the researchers of a study from the University of North Carolina that showed when male and female profs lied about their gender in a course they taught online, students would mark them lower when they presented themselves as female.

“I think really the most important thing to take away from [my research] is that if you are considering these evaluations at your university, as you mentioned at your university they are really important, I think keeping that in mind from the administrative perspective is more important than for the instructors. I think if you ask female professors, if they ask themselves ‘what can I do to solve this?’ or ‘what can I do to overcome this?’ I think that’s… putting the solution on them when they aren’t the cause of the problem,” said MacNell.

Another study of RateMyProfessor.com also showed that female professors tend to be marked on more superficial criteria than their male colleagues.

“There are lots of variables that influence evaluations—the kind of course it is. Students favour lab experiences over lecture, for example. But yes, the literature does suggest that there is a ‘tax’ if you will for those faculty who deviate—in physical appearance—from dominant representations of cultural authority, which is still white and male,” said Susan Searls-Giroux, AVP Faculty, who chairs the committee examining evaluation of teaching.

This data leads some to ask whether course evaluations are given too much weight or whether the data should be adjusted.

Regardless of how this problem is addressed, there will be more transparency with the move to Mosaic as some answers to course evaluation questions will be published through the student portal.

“Typically the overall question that is asked to each student, that we are mandated by Senate policy to ask each student, is ‘overall what is your opinion of the effectiveness of this instructor’, and I believe the statistical data that comes from this question will be available to students,” explained John Bell, IT director in the Faculty of Humanities.

“We were involved in updating and upgrading that policy, mostly that was just to have it posted on the Internet instead of the library, so it would be much more accessible,” said Kleiman.

“We’ve heard though that as much as that happens, previously they weren’t accessed very much—we’d actually like them to be, I think there’d be a lot more conversation if they were accessed more and there’d be a lot more conversation about how they are being used and we could participate in that.”

Overall, students should realize how seriously course evaluations are taken.

“We take those evaluations seriously, they’re used throughout all of our processes, and the students have more influence than they might think. But they can only have that influence if they exercise it through these mechanisms,” said Kleiman.

[feather_share show="twitter, google_plus, facebook, reddit, tumblr" hide="pinterest, linkedin, mail"]

[adrotate banner="12"]