There is no beta: Instructional experimentation in live classes

bridge collapsing

Working in the medical sciences field it stands to reason that a lot of my work would be best fit to publish in academic journals focused on those particular disciplines. The challenge is that these journals set publishing expectations from a scientific point of view. Work should be based on a valid experiment, with each question carefully designed to avoid bias and ensure validity and replicability. There is great value in that approach when working on scientific processes but, my work is on learning in science and learning is a messy process. Working with samples too small to prove anything, no control groups, and outside pressures augmenting the experiment are realities in instructional research. Most importantly, my “subjects” are not volunteers, they are students seeking knowledge. Doing anything to put their learning at risk only in the name of scientific progress is dangerously irresponsible.

Instructional experimentation takes place in real time. There is no test group; there is a first group. Scientific journals may not see the validity in this approach but, they should appreciate the necessity for its use. Despite the amount of preparation and thought put into an instructional experiment, risk factors persist. This does not mean that instructional innovation should be abandoned but, rather there  be a higher level of care and awareness throughout the process. As you prepare your next instructional experiment I offer the following thoughts:

See the experiment through – I have invested countless hours into the development of learning objects, lessons, units, and courses only to have the lead faculty member on the course abandon the experiment after as little as one class. “It didn’t work.” “Students were frustrated.” “The technology was a disaster.” All of these uttered concerns were likely true and they should have been expected. Instructional experiments often start out ugly. There are a lot of factors at play and typically some element of preparation was missed. That is a lesson learned, not a signal to abandon ship.

…but not at the expense of the students – Persistence is a virtue unless it becomes an act of stubborn arrogance. There will likely be some rough patches when implementing change. The key is to determine if it can be overcome or if it is truly affecting how students learn. This is not science. If the experiment is failing you do not always need to see it the end just to have a complete set of data. It is often very evident when the experiment has shifted from challenging to doing harm.

Aim to learn not to succeed – We all would like to write about our successes in the classroom and how one change made a profound impact on all students. Idealistic thinking for sure, but it is not often the reality of instructional experimentation. Most instructional experiments I have been involved with produce modest results at best. The results are rarely statistically significant but, they are significant in other ways. Successful instructional experiments should provide the educational scientist with a better understanding of how learning takes place given the conditions that they set forth in their class. The results may not be valid but, the lessons learned from them are invaluable.

Everything is iterative -Did you think you were done after your first experiment? This is a point that science and education can both agree with. Your first experiment is just that, your first. Now take what you’ve learned and build the next experiment. Great instruction comes from experimentation, from failure, from evaluation.

Share your experience -Maybe your instructional experiment is not valid. Maybe it won’t get published because your design was flawed. These are not reasons to silence your journey. Education is not a science. It involves far more misses than hits. However, we learn from those misses, both our own and those of others. Share your experience through peer discussions, blogging, or presenting at a workshop. It wasn’t a failure, it was an experiment in real time and you lived to tell about.

Leave a Comment