Assessment and Feedback

Week Seven Activities (due March 27th):

We have some background and resources on assessment and feedback this week, followed by activities that parallel the work you did regarding course content:

  1. Do an inventory of your current assessment and feedback strategies.
  2. Create a new assessment for your course.
  3. Write and share a blog post about #1 and #2.

students taking test
A, B, C, or D?

These days, if your idea of assessment and feedback is a multiple choice exam with a grade, there are products from commercial vendors and automated “adaptive learning” software that will streamline that work…and render faculty practically obsolete. But, as you know, good teaching involves feedback that facilitates learning (not just a grade) and assessments that are authentic (not just memorization). Those important tasks are not well-automated by an algorithm; they’re more high-touch, than high-tech. This week, we consider a variety of issues related to such feedback and assessment.

Assessment Is Not the Same as Grading

As Walvoord and Anderson note,  “Grading must be understood as a process that includes: Identifying the most valuable kinds of learning in a course, constructing exams and assignments that will test that learning, setting standards and criteria, guiding students’ learning, and implementing changes in teaching that are based on information from the grading process.”  Assessment and feedback are part that larger process.

While grading can be the result of assessment, not all assessments are graded.  These sorts of distinctions matter greatly as you plan your course.  For example, generally speaking:

  • An “assessment” is something given to students to do (e.g. an assignment), on which they will receive “feedback” in some form.  In this way, the topics of assessment and feedback are connected but distinct.
  • There are a wide variety of assessments.  One important difference is between formative and summative assessments, each of which has a distinct purpose.
    • Formative assessments are designed to give you some insight into the progress and quality of student learning (and identify any areas of difficulty) early enough so that students can make adjustments.  Feedback–which may or may not include a grade–is especially crucial here to give a students a sense of how they are doing and to offer ways of improving their learning.  Formative assessments are often not graded or have very low stakes grades that “count” much less than summative assessments.  (For example, Andrew Miller writes about his journey moving away from grading formative assessments.)
    • Summative assessments are designed for you to judge student learning at the end of a unit or course against the standards and expectations you’ve established.  These are typically high-stakes, graded assignments where feedback beyond a grade is often minimal. Rubrics can sometimes be useful here in providing an explanation and justification of the grade, while minimizing the time needed by the instructor.

Many factors can influence the type and mix of assessments used, including the subject area, the level of students (first-year undergrads or PhD students?) and class size. But, regardless of context, there are some basic considerations to keep in mind.

Assessments Should Reflect Your Course Goals and Be as “Authentic” as Possible

Earlier, you spent some time reviewing and revising the goals for your course.  We encouraged you to think beyond just content-related goals to highlight what students would learn to do; the skills they would acquire as a result of your course.  As biologist Steven Brewer notes, “To know what students are learning, you need to look at what students are doing.”  So coming full circle, your assessments need to accurately reflect those goals. Otherwise, as the classic comedy bit points out, you might as well have a Five Minute University.

The crowd laughs knowingly here because no one remembers much of what they’re taught in college.  Why?  In part, because we’re talking about the short-term memorization of information regurgitated via multiple-choice tests. Your course goals are more than the memorization of content, so make sure you are assessing learning accordingly.

One approach is to focus on “authentic assessment” which, as psychologist Jon Mueller puts it, is “A form of assessment in which students are asked to perform real-world tasks that demonstrate meaningful application of essential knowledge and skills.”  (Here’s a useful overview of this approach.)  In contrast, often students complete assignments that they don’t care about, while faculty dread the process of reviewing and evaluating these assignments. Part of authentic assessment is avoiding such “disposable assignments and intellectual Stairmasters,” where neither the student nor the instructor really cares about the result.

Instead, projects in which students get to show what they’ve learned in ways that tap into their interests and creativity are more likely to generate engagement on their part, as well be more interesting for you to provide feedback and evaluate.  Such assignments are more likely to produce unique and interesting results, rather than repetitious and predictable content.

Know Why You are Doing an Assessment

Creating good assessments involves being clear about your goals for the course and identifying types of evidence you need to show student achievement of those goals. These might include:

  • Product: Do students generate high quality products? Is creativity apparent in their solutions?    
  • Process: Do students’ dialogue meet your expectations and standards? Do they engage in thoughtful discussion of issues? Do their comments imply understanding?
  • Application: Do students apply important concepts from your discipline in a way that shows understanding?
  • Questions: Are they asking good, thoughtful questions, revealing engagement with course material?
  • Recall: Are their responses to quizzes and exams correct? (Remember, there are many variations to offering quizzes and exams that may produce more meaningful learning.)

And don’t forget that students can usefully be involved in assessing themselves.  Here are some basics on self-assessment.

With some thought, you can incorporate creative assessments to encourage students to actively engage with the substance of your course and to help them achieve your course goals.

15851441460_8395cbb046_z
As part of a competition, student teams design programmable robots to follow commands, traverse a simulated Mars terrain and retrieve objects–a very different sort of learning experience.

Asking Questions

Part of good assessment often involves asking good questions.  Good questions vary, of course, and depend upon the context within which they are being asked and the purpose to which they are directed. Sometimes we want students to ask good questions about relatively generic issues, such as questions about a research article or questions about a source. Other times, the questions will be tailored very specifically to the content at hand.

Even in the latter case, though, basic question frameworks can be used to encourage critical thinking. For example, a “4 C’s” frame (from this) can be applied to a video (or readings) asking students:

  • Connections: What connections do you draw between the video and other concepts from this course?
  • Concepts: What do you think are the most important or key concepts or ideas in the video that are worth holding on to?
  • Challenge: What ideas, positions, or assumptions in the video do you want to challenge or argue with?
  • Changes: What changes in attitudes, thinking or action are suggested, either for you or for others?

Such questions encourage students to get beyond superficial reactions (e.g. what they “like” or found “interesting”) and start thinking about material more critically (making assessments, seeing connections, considering implications, and the like). These sorts of basic framing questions are infinitely adaptable and can be customized to fit your specific course content.

Assessing Connected Work

Finally, if one of your goals is teaching lifelong learning skills, you may want to assess how students are working in an open, connected, online environment.  VCU Education student, Laura Gogia MD PhD, has just finished a dissertation about digital assessments.  Laura is now a VCU research fellow of Learning Innovation and Student Success.  Her work focuses on emerging digital pedagogies, course design, and student assessment in higher education.  Details of her work can be found here and, below, she talks about how and why assessing student connections could be relevant for your course.

Her dissertation site includes handy materials for faculty such as language for learning objectives and a rubric for assessing student blogging.

Feedback

Feedback is a corollary to assessment and a key element of teaching.  Some ways to provide feedback in an online class include:

  • Blog Comments. The comment feature on student blogs is a natural for feedback.  The limitation here is that most themes only allow comments at the bottom of a post, making it difficult to comment on specific sentences without quoting. However, you can activate plugins that allow commenting on individual paragraphs for more specificity.  Also, these comments are public and may help other students besides the one for whom they were written.
  • Word Processing Comments. More detailed comments on a student’s writing is more appropriately done in with the comments feature in Google Docs or Word. This can also give you some privacy, if there are some issues you don’t want to comment on publicly.  One strategy is to have students draft work here, get feedback from you, and then copy/paste their essay in a blog post to make them visible to others and available for further feedback.
  • Video feedback.  Another way some instructors provide feedback on almost any sort of student work is to create simple screencasts that capture audio comments while the instructor points to portions of the student assignment.  Audio and visual work together and students can replay comments.
  • Miscellaneous Tools.  As we’ve noted earlier there are tools that can assist in producing other sorts of feedback including:
    • Hypothes.is can be used for close commenting on / annotating any online text, including student work.
    • Videonot.es, which can be used to provide feedback on student-created videos.
  • Rubrics.  Rubrics can be a useful way to both structure an assessment in advance–giving students clear expectations for the assignment–and to provide quick and easy feedback on the areas of strength and weakness in an assignment. This can be especially useful in larger classes. But rubrics can also be overly detailed, overused, and, intellectually fraudulant–leading to considerable controversy.  Still, there is a mountain of material on rubrics you can explore, including:

Format doesn’t matter, though, if the substance of the feedback is not useful.  See Seven Keys to Effective Feedback for some helpful advice.  It was written for K-12 but is mostly applicable to higher education as well.

Also, feedback can also come from other students, as with blog comments.  This serves the dual purpose of both providing one student with feedback while having another student exercise some critical thinking and writing skills. There are a number of ways–including the RISE approach— to help students (and instructors) learn to write more useful feedback.

RISE_infographic-peer

Activity #1: What are Your Current Assessments and Feedback Mechanisms?

Review your syllabus and make an inventory of the types of assessment and feedback you use. Make a list in a Google Doc put it in the shared OLE folder labelled “Assessment and Feedback Inventories.”  For each item on your list be sure to:

  1. Note the type and form of assessment. For example:
    • multiple-choice quiz or test,
    • short-answer quiz or test,
    • reflection responses, short essays, research papers,
    • blog posts
    • videos
    • presentations
    • projects
    • other
    • Is this work done in public (presentations, blog posts, etc) or private?  Why?
  2. Note the primary purpose of this assessment:
    • To what course goal(s) is this linked?
    • What are you looking to learn from this assessment?
  3. Note the type of feedback each assessment receives.
    • Is the feedback from you alone?  Do students provide some feedback and comment?
    • What form does the feedback take?  (written, audio, video?)
    • Is the feedback public (so others might benefit) or private?
    • What is the primary purpose of the feedback?

Now that you have a snapshot of how you approach assessment and feedback for the course as a whole, go on to Activity #2.

Activity #2: Design a New Assessment

After you’ve completed the first activity, create an assessment for some part of your course using an approach you have not tried before.  For example, if you typically rely on essays, try a different type of assignment–perhaps a video or some other project.  Or plan to use a novel type of feedback mechanism. Be sure you include instructions to students and an explanation of how you will be providing feedback/grading.

Add this new assessment to the Google Doc you created in Activity #1

Activity #3: Share Your Learning in a Blog Post

Write a blog post reflecting on what you’ve learned from these activities. (Be sure to check the proper category on your post.) In particular:

Existing:

  • Did you find that your existing assessments accurately reflected the type of learning you are seeking from students?
  • Upon reflection, do they warrant the time you invest in providing feedback and (perhaps) a grade?
  • Why did you choose these particular assessments?
  • Are there any parts of your course where assessment and feedback are a special challenge? Why?

New:

  • Why did you choose to create this particular new assessment?
  • What are you hoping to achieve with it?
  • What challenges might it pose for you or your students?

As you are working through these activities, take a look at–and comment on–on your colleagues’ Google Docs and blog posts. What do you think about these assignments?  Do they seem engaging?  Are they explained clearly?  Does the evaluation framework seem clear and fair? Does anything surprise you? Did you like and adapt an idea from one of them?  Do you have a suggestion that might help strengthen it?