Best Professors List

Yesterday, I discussed a movement to make parts of the course evaluations done by students available to prospective students. Basically an in house RateMyProfessor-like service. During that discussion, I noted that there was an additional movement to establish a 'top 30% teachers' list for students to use in choosing their courses. I am against a 'top 30% teachers' list and want to defend that position here.

The idea for a 'top 30%' list seems to come, at least in part, from the Vice Provost for Faculty and Academic Affairs. The list is based on a University of Illinois List of Teachers Rated Excellent, where the Vice Provost was a faculty member (information from the Minnesota Daily). As an aside, one thing I see much too frequently from the University of Minnesota administration is the 'well everyone else is/was doing it' defense. 'Everyone else does it' does not constitute a data driven justification.

Of course having a 'top 30%' list does provide more information to prospective students trying to plan which courses they will register for (FYI: 'for which courses they will register' sounds stupid, so I'm ending that sentence with a preposition deal with it). But this raises a number of questions that need to be dealt with. Including, is the information meaningful in a way the university intends, or at least how I hope they intend? How do we determine the 'top 30%' of teachers? What are the potential ramifications of such a list?

Is the information meaningful in a way the university intends?

This is a key problem in my opinion. I expect the university wants to identify the individuals that best educate the students, generate passion for the material, and promote students well prepared for their subsequent courses or life outside of college. Does anyone believe that students can effectively evaluate whether the instructors have effectively disseminated knowledge or taught the tools necessary to gain and assess knowledge? Do the students know whether or not they are well prepared for their advanced courses? I doubt it.

In fact numerous studies have clearly demonstrated that course evaluations reflect the predicted grade the student thinks they are getting and little else. Based on this, those courses where most students get A's are more likely to have instructors rated as excellent than courses that show a greater distribution of grades. This is not an absolute of course.

Smaller classes are reviewed more favorably than larger classes, which gives an advantage to instructors teaching upper level courses compared to instructors teaching large survey courses. 

How do we determine the 'top 30%' of teachers?

Furthermore, why 30%? Why not 25% or 33%? Regardless how do we establish who the top 30% are? I have my evaluations for the two courses I taught last semester. One was a course for second year students, which I taught for the first time; one was a course for fourth year and graduate students that I developed and have taught for six years.

The surveys list the following 'Core Items' that are scored from: 1 (Strongly disagree) to 6 (Strongly Agree). I'll simply give the mean and standard deviations for my two courses for each Core Item.

1. The instructor was well prepared for class.
• 5.85 ± 0.36 and 5.94 ± 0.23
2. The instructor presented the subject matter clearly.
• 5.89 ± 0.31 and 5.56 ± 0.60
3. The instructor provided feedback intended to improve my performance.
• 5.19 ± 0.77 and 5.17 ± 0.76
4. The instructor treated me with respect.
• 5.74 ± 0.44 and 5.41 ± 0.84
5. I have a deeper understanding of the subject matter as a result of this course.
• 5.67 ± 0.54 and 5.61 ± 0.83
6. My interest in the subject matter was stimulated by this course.
• 5.07 ± 0.98 and 5.17 ± 1.38

To make this more meaningful I'll convert those means to %s and an A-F grade scale.

1. 97.5 (A) and 99.0 (A)
2. 98.2 (A) and 92.7 (A)
3. 86.5 (B) and 86.2 (B)
4. 95.7 (A) and 90.2 (A)
5. 94.5 (A) and 93.5 (A)
6. 84.5 (B) and 86.2 (B)

In both courses my overall (all 6 criteria combined from the letter grade) GPA is a 3.67. Four A's and two B's for each course, which seems pretty damn good. Does this put me in the top 30%? It seems likely, but does this mean I am an excellent teacher or maybe I simply have a great rapport with the students or maybe both. What if these scores are not sufficient to be in the top 30%? Does this mean I am not an excellent teacher? Should I make changes to my courses to improve these scores? Note: I am not suggesting making changes to improve the course, but simply to improve these scores. 

I am still guessing on how the top 30% is determined, maybe the powers that be will simply choose item #6 or not use any of these Core Items but one or more of the 'student release data' discussed yesterday. This was never clearly established.

What are the potential ramifications of such a list?

As I already noted this kind of scoring system may cause some instructors to change their teaching and/or courses to improve their score. Why would they do this? Well not all instructors are tenured or even on a tenure track. These kind of scoring systems are easy for administrators to look at and say 'Ohhh a number with a decimal point that must be important,' regardless of whether or not the number conveys meaningful information. It is also important to realize that the number of students in a classroom directly reflects the money coming into the department offering the course. If there are two sections of the same course taught by different instructors and one is packed and the other unable to fill all the seats based on these scoring systems, which instructor will likely suffer most during yearly evaluations with their departmental chair/dean.

Will there be competition between colleges? If the 30% is a university-wide figure, then problems can emerge. There are already some serious issues with grade inflation here in Minnesota as observed at many universities across the country. However, the grade inflation is not uniform. The College of Biological Sciences and College of Science and Engineering have much less grade inflation than the College of Food, Agriculture, and Natural Resource Sciences (a post for another day). Based on the evidence that instructor evaluations correlate with student grades, then colleges with more grade inflation should on average have better course evaluations. How does this tell which instructors are the 'top 30%'?


As it stands, I do not see the value added to a 'top 30%' list anymore than the release of some course evaluation data. The point seems to be one of treating the students as customers. I understand that we as faculty and staff have responsibilities and duties to our students and the community as a whole. I am not espousing a view where students are fortunate to hear me pontificate on microbiology. But universities are not corporations like Burger King trying their best to please and appeal to the customer more than McDonald's (aka University of Wisconsin).