Monthly Archives: October 2016

The Role of Perceived Race and Gender in the Evaluation of College Teaching on RateMyProfessors.com (2010)

1 Star2 Stars3 Stars4 Stars5 Stars (No Ratings Yet)
Loading...

Submitted by Christina Barnett

Article Reference

Reid, L. D. (2010). The role of perceived race and gender in the evaluation of college teaching on ratemyprofessors.com. <em>Journal of Diversity in Higher Education</em>, 3, 137-152. doi:10.1037/a0019865

Article DOI

10.1037/a0019865

Summary of Article

 

The present study examined the effects of perceived faculty race and gender on student evaluations of teaching (SETs). Prior research indicates that an instructor’s gender imposes different standards of competencies for women than men, as well as less favorable reviews for women instructors in traditionally masculine academic disciplines (Kierstead, D’Agostino, & Dill, 1988). Prior research also shows that instructors of color are often evaluated as less intellectually competent (Ho, Thomsen, & Sidanius, 2009). Although gender is one of the most researched demographic characteristics in relation to student evaluations, there is little empirical research about how an instructor’s race may influence student evaluations, or about how impactful one’s intersectional identity, inclusive of both race and gender, could be when students complete evaluations. The present study collected and evaluated ratings for every faculty member listed on ratemyprofessors.com at the top 25 liberal arts colleges according to the 2006 U.S. News and World Report rankings. Student coders used publically available photographs to evaluate race, and used the name as well as gender pronouns within the reviews to determine the gender of the instructor. Data from the study were collected from 5,630 faculty ratings (3,079 White; 142 Black; 238 Asian; 130 Latino; and 128 Other race faculty). Instructors’ SETs were evaluated using ratemyprofessors.com’s rubric which includes a global rating of overall instructor quality, as well as ratings of easiness, helpfulness, and clarity. Results indicated that instructors of color were evaluated more negatively than White faculty in overall quality, helpfulness, and clarity, but were rated higher in easiness. Cluster analyses results indicated that gender differences within racial groups were non-existent except for Black instructors, where Black male instructors were considered easier than Black female instructors, and Black female instructors were perceived as more clear than Black male faculty. Although results were limited by several factors including the instructors’ inability to self-identify demographic information, the narrow scope of the SETs, and measurement error from the site itself, this study is important in recognizing that stereotype-based expectancies may actually be what’s being assessed during student evaluations of faculty of color.

 

Discussion Questions

  1. 1. Does VCU’s SRI capture the meaning of effective teaching?
    2. Do you believe SRIs are representative of teaching effectiveness? Are they an accurate indication of who an instructor is? 
  2. 3. Can students discern teaching excellence?
    4. What personal biases influence your rating of instructors?  
  3. 5. The text states that biasing factors account for "so little of the variance in ratings that they can be ignored with little risk." Do you believe that this this is true when race of the instructor is taken into consideration?

The Role of Perceived Race and Gender in the Evaluation of College Teaching on RateMyProfessors.com (2010)

1 Star2 Stars3 Stars4 Stars5 Stars (No Ratings Yet)
Loading...

Submitted by Landon D. Reid

Article Reference

Reid, L. D. (2010). The role of perceived race and gender in the evaluation of college teaching on ratemyprofessors.com. <em>Journal of Diversity in Higher Education</em>, 3, 137-152. doi:10.1037/a0019865

Article DOI

10.1037/a0019865

Summary of Article

 

The present study examined the effects of perceived faculty race and gender on student evaluations of teaching (SETs). Prior research indicates that an instructor’s gender imposes different standards of competencies for women than men, as well as less favorable reviews for women instructors in traditionally masculine academic disciplines (Kierstead, D’Agostino, & Dill, 1988). Prior research also shows that instructors of color are often evaluated as less intellectually competent (Ho, Thomsen, & Sidanius, 2009). Although gender is one of the most researched demographic characteristics in relation to student evaluations, there is little empirical research about how an instructor’s race may influence student evaluations, or about how impactful one’s intersectional identity, inclusive of both race and gender, could be when students complete evaluations. The present study collected and evaluated ratings for every faculty member listed on ratemyprofessors.com at the top 25 liberal arts colleges according to the 2006 U.S. News and World Report rankings. Student coders used publically available photographs to evaluate race, and used the name as well as gender pronouns within the reviews to determine the gender of the instructor. Data from the study were collected from 5,630 faculty ratings (3,079 White; 142 Black; 238 Asian; 130 Latino; and 128 Other race faculty). Instructors’ SETs were evaluated using ratemyprofessors.com’s rubric which includes a global rating of overall instructor quality, as well as ratings of easiness, helpfulness, and clarity. Results indicated that instructors of color were evaluated more negatively than White faculty in overall quality, helpfulness, and clarity, but were rated higher in easiness. Cluster analyses results indicated that gender differences within racial groups were non-existent except for Black instructors, where Black male instructors were considered easier than Black female instructors, and Black female instructors were perceived as more clear than Black male faculty. Although results were limited by several factors including the instructors’ inability to self-identify demographic information, the narrow scope of the SETs, and measurement error from the site itself, this study is important in recognizing that stereotype-based expectancies may actually be what’s being assessed during student evaluations of faculty of color.

 

Discussion Questions

  1. 1. Does VCU’s SRI capture the meaning of effective teaching?
    2. Do you believe SRIs are representative of teaching effectiveness? Are they an accurate indication of who an instructor is? 
  2. 3. Can students discern teaching excellence?
    4. What personal biases influence your rating of instructors?  
  3. 5. The text states that biasing factors account for "so little of the variance in ratings that they can be ignored with little risk." Do you believe that this this is true when race of the instructor is taken into consideration?

Qualtrics for Surveys, Evaluations, Etc.

1 Star2 Stars3 Stars4 Stars5 Stars (No Ratings Yet)
Loading...

Submitted by Cathryn Richmond

For my technology project, I explored the possibility of using Qualtrics for online assessments.  Qualtrics has a number of online training videos available that explore basic features and survey building which allow you to explore the utility of the platform before creating an account: https://www.qualtrics.com/support/survey-platform/getting-started/survey-platform-overview/

Qualtrics appears quite user-friendly and has a number of useful features.  There are a wide variety of question types, including multiple choice, free text, visual analog scales, Likert scales, etc. and validation can be added to reduce data entry errors – for instance, questions can only be available for data entry when a previous field is answered a specific way (e.g. "If yes, explain" would only be available if the previous question was answered "Yes"), fields can be required to be completed prior to moving to the next page, etc.  In addition, the surveys themselves can be modified in several ways, including requiring passwords, requiring all questions be answered in a single setting, etc.

Perhaps the most exciting part in my opinion was the analytic features.  Data can be automatically downloaded into a number of formats including SPSS and Excel, and there are a number of tools to assist with analysis such as cross tabulation (will calculate p-values and chi-square statistics based on your data, for example), the ability to exclude specific response options, and easy report generation.  A particularly exciting feature is the ability to programatically search free-text responses – Qualtrics allows the user to select certain words or combination of words to search for in the responses to any open text fields, which is extremely useful for qualitative data such as faculty evaluation comments.

I highly recommend Qualtrics, and creating an account is free!  https://www.qualtrics.com/free-account/

The Voices Project: Reducing White Students’ Racism in Introduction to Psychology

1 Star2 Stars3 Stars4 Stars5 Stars (No Ratings Yet)
Loading...

Submitted by Cathryn Richmond

Article Reference

Nordstrom, A. H. (2015). The voices project: Reducing white students’ racism in introduction to psychology. Teaching of Psychology, 42, 43-50.

Article DOI

10.1177/0098628314562524

Summary of Article

<strong>Background</strong>
<ul>
<li>APA guidelines: “…diversity issues should be infused throughout the undergraduate learning goals and outcomes.”</li>
<li>Racism in general, colorblind racism, White fear: Less racial awareness, cultural sensitivity, more racist attitudes, more resistance to diversity education, lower self-esteem, worse college adjustment</li>
<li>Racism defined as “a form of prejudice that provides Whites with social, cultural, and economic privileges and disadvantages people of color (non-Whites).”</li>
<li>Predominately White institutions: Less exposure to other groups, lack of understanding and exploration of racial development and identity</li>
<li>Intervention based on Contact Theory: “Direct intergroup contact would reduce students’ stereotypes and prejudice of racial groups toward whom they have negative attitudes or unfamiliarity.”</li>
</ul>
<strong> </strong>

<strong>Methodology</strong>
<ul>
<li>Two Intro to Psychology courses taught by the same professor [Intervention (n=26) vs. Control (n=17)]</li>
<li>Measured attitudes and beliefs at the beginning and end of semester
<ul>
<li>Attitudes towards Asians Scale</li>
<li>Modern Racism Scale
<ul>
<li>“Discrimination is no longer an issue and that Blacks have received undeserved influence and privileges”</li>
</ul>
</li>
<li>Oklahoma Racial Attitudes Scale Revised
<ul>
<li>Conflictive Racial Justice subscale: “Helping racial minorities will undermine Whites”</li>
<li>Reactive Racial Justice subscale: Acknowledgement of White privilege</li>
</ul>
</li>
<li>Prejudice Against Hispanics Scale</li>
<li>Islamophobia Scale</li>
</ul>
</li>
<li>Intervention: “The Voices Project (TVP)”
<ul>
<li>Activities:
<ul>
<li>Interviewing a member of a social group with which they have unfamiliarity or have negative attitudes towards about the person’s experience being Muslim, Asian American, African American, or Hispanic</li>
<li>Attend a cultural event of the same group</li>
</ul>
</li>
<li>Assignment:
<ul>
<li>Report on events in class as they occurred</li>
<li>Write a 5-7 page memoir in first-person of the interviewee’s life</li>
</ul>
</li>
<li>Control
<ul>
<li>“Read a memoir of bipolar disorder, break a social norm [or] analyze commercials on television to identify stereotypes”</li>
</ul>
</li>
</ul>
</li>
</ul>
<strong> </strong>

<strong>Results</strong>
<ul>
<li>Significant improvement in TVP group compared to control
<ul>
<li>Decrease in negative attitudes and increase in positive attitudes</li>
<li>Not restricted to the group membership of the interviewee</li>
<li>Largest effects for Muslims, Hispanics, and conflictive racial justice</li>
</ul>
</li>
</ul>

Discussion Questions

  1. What are potential explanations for the varying effects of the intervention on the different scales?
    o Significant effect on conflictive racial justice but essentially no effect at all for reactive racial justice
    o Most dramatic effect on Islamophobia
    o Very little effect on the Modern Racism scale
  2. Do you think this intervention would have been equally effective for other racial or ethnic groups? In non-psychology courses? At a different timepoint (e.g. senior year, high school)? If it were done via Skype in an online course?
  3. How could we incorporate diversity exposure into our 300+ person Intro to Psych courses? What are other ways we might incorporate diversity and ideally exposure in our courses?

Making students’ evaluations of teaching effectiveness effective: The critical issues of validity, bias, and utility

1 Star2 Stars3 Stars4 Stars5 Stars (No Ratings Yet)
Loading...

Submitted by Brad Pierce

Article Reference

Marsh, H. W., & Roche, L. A. (1997). Making students' evaluations of teaching effectiveness effective: The critical issues of validity, bias, and utility. <em>American Psychologist</em>, <em>52</em>(11), 1187-1197.

Article DOI

10.1002/wps.20238

Summary of Article

The article argues for using a multidimensional approach to instructor feedback using specific items rather than using more general, global items (e.g., “The instructor was effective at teaching the course material”). The multidimensional approach would assess a number of areas. For example:
<ol>
<li>Learning/Value</li>
<li>Instructor Enthusiasm</li>
<li>Organization/Clarity, Group Interaction</li>
<li>Individual Rapport</li>
<li>Breadth of Coverage</li>
<li>Examinations/Grading</li>
<li>Assignments/Readings</li>
<li>Workload/Difficulty</li>
</ol>
More general items are influenced by student subject interest, expected grade, course workload, class size, and level of course (student year). Rating systems that use a multidimensional approach with more specific items provide more utility in determining strengths and weaknesses and are less susceptible to student biases, moods, and context.

Discussion Questions

  1. Are student ratings of instruction measuring the instructor’s effectiveness, or simply student’s perception of the instructor’s effectiveness?
  2. How influential should student ratings of instruction be when evaluating an instructor’s effectiveness?
  3. 3. Are there unintended consequences to using student ratings of instruction? The book lists a few (may decrease morale, teachers change their style, grade inflation).

Google Form for Guest Lecturing

1 Star2 Stars3 Stars4 Stars5 Stars (No Ratings Yet)
Loading...

Submitted by Dana Schreiber

For my technology activity, I chose to create a google rating form/instructor evaluation form that I can use for guest lecture opportunities. I have never created a google form and was interested to get more experience. I found it extremely easy to create the form and like the options that google provides regarding formatting questions. I like that the form is user friendly and also that it is very easy to share. This form can easily be adapted if I need to update/revise it in the future. While it can be helpful to get paper feedback after a guest lecture, I like having an online version that I can provide students with as well. I plan to use google forms in the future as a resource to get student feedback in various domains and am glad I got the opportunity to learn more about this resource. Here is a link to the form that I created in case anyone is interested: https://docs.google.com/forms/u/1/.

 

 

 

Active Learning Not Associated with Student Learning in a Sample of College Biology Courses

1 Star2 Stars3 Stars4 Stars5 Stars (No Ratings Yet)
Loading...

Submitted by Dana Schreiber

Article Reference

<span style="font-weight: 400;">Andrews, T. M., Leonard, M. J., Colgrove, C. A., & Kalinowski, S. T. (2011). Active learning not associated with student learning in a random sample of college biology courses. </span><i><span style="font-weight: 400;">CBE-Life Sciences Education</span></i><span style="font-weight: 400;">, </span><i><span style="font-weight: 400;">10</span></i><span style="font-weight: 400;">, 394-405.</span>

Article DOI

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3228657/

Summary of Article

<span style="font-weight: 400;">This article investigates the role of active learning in student learning of natural selection in Introduction to Biology courses across colleges and universities in the United States. Previous work has suggested that active learning improves student learning compared to traditional lecture style classes. However, most research has examined classes that are instructed by professors with science education research experience. The goal of the current study was to examine the use of active learning in college-level biology courses using instructors that represent a broader population of college professors. Researchers randomly selected instructors of college-level introduction to biology courses. The researchers measured active learning by quantifying how often instructors use active learning per week (i.e., never, once per week, once per class, more than once per class) and measured learning with pre and post tests that included both a multiple choice test and an open ended question test. Learning gains were measured by effect size (Cohen’s </span><i><span style="font-weight: 400;">d</span></i><span style="font-weight: 400;">). Results indicated that active learning, as used by typical college biology instructors, was not associated with greater learning gains. The study did find, however, that certain types of active learning approaches may be beneficial in regards to student learning (i.e., addressing student misconceptions through active learning strategies). The researchers concluded that more training and resources are needed for instructors in order learn how to effectively use active learning approaches. While active learning is a useful strategy to incorporate in large lecture classrooms, it requires a thorough understanding, commitment to, and ability to execute strategies in order for them to be effective. </span>

Discussion Questions

  1. The article found that active learning was NOT associated with student learning in Introduction to Biology courses and concluded that one reason is because instructors are using active learning ineffectively. How can instructors learn to use active learning can in a more effective way? Are there barriers to using active learning in college classrooms?
  2. The article states that in order for active learning to be effective, instructors must use a constructivist approach (idea that students construct knowledge by incorporating new ideas into an existing framework). What are ways that instructors can implement this approach? How can instructors use active learning to effectively handle misconceptions (preexisting ideas that students have about a topic)?
  3. Are there factors that the study did not take into account that may be impacting results? Is this study generalizable to other fields/topics of study (i.e., Psychology) and other classroom environments (i.e., smaller classes, upper level college courses)?