Why it's so hard to evaluate teaching methods

Few studies offer conclusive evidence

There is no standard protocol for evaluating teaching approaches in higher education, meaning faculty must look elsewhere to determine what works and what doesn't, Angela Carbone writes for The Conversation. 

Even the most skilled instructors face uncertainty when designing a curriculum. They wonder whether adding online learning will improve student outcomes, whether to allow students to bring their own devices to class, or whether the lecture model is really as outdated as everyone says it is.

Instructors need to know which methods will improve several metrics of student success, including:

  • Motivation and inspiration;
  • Study approaches;
  • Employability; and
  • Grades.

Unfortunately, says Carbone, tens of thousands of studies attempt to assess teaching approaches, but their results are often limited to specific circumstances—and may even appear to contradict each other entirely.

"For every study that says a change is better—for example, introducing social media into the classroom—there will be another that argues the opposite," writes Carbone.

There's also no set standard of methodology, which can make it difficult to draw broad conclusions from narrow studies.

Research in the physical sciences, for instance, often focus on facts and data, while studies in the social sciences often focus on developing an understanding over a long period of time.

So where do instructors get their answers about student success?

Because of the discrepancies in the field, Carbone says the closest an instructor will come to determining best practices is by looking at a meta-study, which presents a systematic review of research synthesized into main conclusions.

Carbone offers several examples of meta-studies that synthesize broad research into key conclusions.

Related: Engage faculty to make a lasting change in student outcomes

One such synthesis is the Blended Learning in Higher Education study from the Association for Information Systems (AIS), which looked at the best practices for blending online and face-to-face components. AIS found that the most effective mix depends on the following criteria:

  • Professional development support available at the institution;
  • The instructor's experience using technology for work;
  • How willing the instructor is to test new approaches;
  • Technical support available at the institution;
  • The types of students enrolled in the class;
  • The students' technology accessibility;
  • The students' outside commitments;
  • The students' campus accessibility; and
  • The type of course being taught.

An additional meta-analysis, which speaks most directly to the question of best practices for teaching, is the Australian Government's Office for Learning and Teaching's (OLT) review of teaching quality.

That meta-analysis found that the following dimensions determine teaching quality:

  • How personable the teacher is;
  • Whether the teacher is performative;
  • How well the teacher creates interactions;
  • How much the teacher motivates students;
  • Whether the teacher's demands are realistic;
  • If the teacher has an international perspective;
  • How well the teacher helps students find meaning;
  • Whether the teacher uses effective assessment processes; and
  • Whether the teacher develops autonomy among students.

(Carbone, The Conversation, 10/17).

How to identify—and support—your most innovative instructors

  • Manage Your Events
  • Saved webpages and searches
  • Manage your subscriptions
  • Update personal information
  • Invite a colleague