Submitted by Fan Zhang
Beran, T. N., & Rokosh, J. L. (2009). The consequential validity of student ratings: What do instructors really think?.Alberta Journal of Educational Research, 55(4).
Felton, J., Koper, P. T., Mitchell, J. and Stinson, M. (2008), “Attractiveness, easiness and other issues: student evaluations of professors on Ratemyprofessors.com”, Assessment&Evaluation in Higher Education, Vol. 33 No. 1, pp. 45-61.
Summary of Article
The Consequential Validity of Student Ratings: What do Instructors Really Think?
The purpose of this study was to examine the consequential validity of student ratings, according to instructors at a major Canadian university. Results indicate that most instructors reported concerns about the SRI.
Poor design of the instrument (70%)
The survey produced a limited amount of useful information. While there are items in the study that are targeted at specific areas of instruction, yet, these items may not be precise enough for instructors to determine how to improve these areas.
Many instructors indicated that the items were general and not applicable to their style of teaching or course design.
Procedural difficulties (56%)
Many instructors find that the SRI is administered too frequently, and resulting in “student rating fatigue.”
Myth-based issues (31%)
Some instructors consider the USRI to be an unfair measure as it is purely a “popularity contest” or believe that giving out higher grades will result in better SRI scores.
Ratings are biased (29%)
Many instructors believe students’ evaluations to be biased by several factors, including course difficulty, instructor popularity, grading leniency, prior student interest, and class size, although research has consistently shown that most such background characteristics have a negligible effect on student evaluation.
Negative effect on instructors/instruction (11%)
A number of instructors reported feeling that the student rating procedure leads instructors to lower their standard to avoid receiving low ratings.
This study revealed the importance of the consistency between what instructors consider to be quality teaching and the measures used to assess them.
Attractiveness, easiness and other issues: student evaluations of professors on Ratemyprofessors.com
Ratemyprofessors.com is a website with the motto ‘Where the students do the grading.’ It is not affiliated with any institution of higher education or accrediting agency. Since 1999, it has received nearly six million postings rating more than 750,000 instructors at more than 6000 schools.
At the time of this study, students can voluntarily rate their professor at the website based on easiness, helpfulness, clarity, overall quality, and hotness. It is worse noting that today, there are only two categories that are being highlighted on the website, and they are the level of difficulty and overall quality.
This study included data from 6852 professors from 369 institutions in and the United States and Canada. They found that there is a significant positive correlation for Quality and Easiness (0.62), and they found professors with high Easiness scores usually have student comments regrading a light workload and high grades. The authors of this article claim, based on these findings, they think these self- selected evaluations from Ratemyprofessors.com cast considerable doubt on the usefulness of in-class student opinion surveys for purposes of examining quality and effectiveness of teaching.
- Do you think SRI can be an accurate representation of the quality of a course?
- 2. What changes will you make so the SRI can be more helpful to the growth of an educator?
- 3. Do you think websites like “rate my professor” influence how professors teaching today?