Sixty institutions try a new solution to an old question: Are students learning?

New multi-state program 'holds a lot of promise'

The results of a pilot initiative attempting to measure student learning in a consistent way are in, Doug Lederman reports for Inside Higher Ed.

The Multi-State Collaborative to Advance Learning Outcomes Assessment is a joint project led by the Association of American Colleges and Universities (AAC&U) and the State Higher Education Executive Officers (SHEEO).

Historically, people have fallen into two schools of thought on measuring student learning:

1. Faculty grading does not say much about what students learn; or
2. Standardized tests define learning too narrowly and are unrelated to classroom lessons.

The collaborative, however, "holds a lot of promise in grappling with these issues in a new, different way," says Corbin Campbell, assistant professor of higher education at Columbia University's Teachers College.

How it came about

ACC&U developed "essential learning outcomes" and then enlisted hundreds of faculty members to create rubrics to gauge those benchmarks in the Valid Assessment of Learning in Undergraduate Education (VALUE) program. Essentially, they created a set of expectations of what every student should be able to do. 

ACC&U and SHEEO then partnered on the collaborative, involving 60 institutions across nine states. About 7,000 samples of student course work were shared with 125 VALUE-trained professors, who independently scored them. This year, three more states will join.

The graded work came from the participating colleges, which each sent in a representative sample from students who had completed at least three-fourths of their course work.

Findings

The results were not promising, Lederman writes.

  • Fewer than a third of samples from four-year colleges earned a three or four (out of four) for "using evidence to investigate a point of view or reach a conclusion."
  • About two-thirds of two-year student work earned a less than a three or four for their use of evidence and sources.
  • More than half of four-year college work and about two-thirds of two-year college work earned less than a three of four for drawing "appropriate conclusions based on quantitative analysis of data."

The findings are similar to other studies, says Richard Arum, New York University professor of sociology and education. "This is as close to a consensus as one ever sees in social science about the nature and character of this problem," he says.

The collaborative may be one of "multiple indicators" of student learning eventually embraced by colleges, universities, and policy makers, he says.

Looking forward

Some participants say they want to see submissions of assignments completed by students who have completed less than 75% of their course work because the current standard narrows the scope to the most successful students. It excludes others who may have dropped out or switched schools, Campbell says.

She says she also wants to see skills compared over time.

"If you don't have some kind of comparison of change, showing what they could do when they came in and when they left," she says, "it may do exactly what the rankings do: reinforce the reality that great students produce great work, and great institutions have great students" (Lederman, Inside Higher Ed, 9/25).

Thoughts on the story? Tweet us at @eab_daily and let us know.


  • Manage Your Events
  • Saved webpages and searches
  • Manage your subscriptions
  • Update personal information
  • Invite a colleague