Why hasn't better data led to better decisions?

Kristin Tyndall, editorKristin Tyndall, editor

As college leaders faced the seismic changes wrought by the recession, a solution emerged: Get more data.

In recent years, colleges have made enormous strides in collecting data, making it transferrable, and building sophisticated analytics and institutional research teams.

Despite these advances, campus leaders continue to report that they don't feel confident that they and their teams are making data-informed decisions. And in terms of building a data-informed culture across campus, faculty remain skeptical of assessment efforts.

To find out why colleges aren't seeing the results they expected from their data efforts, I reached out to Apoorva Shah, an EAB expert on colleges' use of data with our Academic Performance Solutions team.

Why hasn't more data helped college leaders feel confident they're making better decisions?

As it turns out, the institutions with the most advanced analytics tools are not necessarily the institutions that are most sophisticated at using them to make decisions.

What happened on a lot of campuses is we end up with a situation where individual deans and chairs and senior leaders are all trying to make better decisions, but they're all going about it in their own way.

So you end up with a kind of data anarchy?

Exactly. You've got a lot of people making small decisions at the institution, and people are using their own dashboards or their own metrics or no dashboards at all. Some of your deans and chairs feel really strongly about their own tools while others say they feel like they're flying blind.

In this situation, it's easy to get distracted by arguing about what each data point means and whether it's important. For example, if we're deciding whether a program needs an additional faculty member, should we be looking at the rising faculty workload? Headcounts in course sections? Peer benchmarks of faculty-to-student ratios? Did we choose the right peers for those benchmarks? It becomes a slippery slope.

Sounds like all this data hasn't really changed what happens when a team gets together to make a decision?

No, that use of data doesn't really help drive conversations and decisions on campus forward. Provosts tell us their teams spend a lot of time debating the data itself rather than making decisions, and whoever shouts the loudest often wins.

In fact, this kind of data culture can lead to a lot of miscommunication and harm. Your faculty, deans, and chairs may not understand the context about why decisions are made. And in turn, they're not going to feel like they have the resources they need to make their own decisions in a way that aligns with the broader campus priorities.

How can colleges break out of this cycle?

You can't start with data for its own sake.

Step one is to figure out what are your institutional goals and what are you going to look at to measure your progress against those goals. You've got to build a common language around: What should we be looking at?

That's when the data becomes most useful, and when it is aligned to specific goals and objectives, you can create a true cultural shift.

Tell me a little more about that process. In your experience, what's the most effective way to start influencing culture?

In my work with Academic Performance Solutions, I’ve found that the campus leaders who are most effective at creating change are those who can empower their deans and chairs with a common understanding of the institution's goals, foster agreement about how they're going to meet them, and provide the tools they need to actually make those decisions in the moment.

My other recommendation is that measuring progress in a vacuum is difficult. To really understand how your institution is performing, you need better context and calibration using data from outside your organization. The campus leaders I've worked with tell me that peer benchmark data has been a critical tool for driving change. 

Learn more: How to equip your deans and chairs with the resources they need to make data-informed decisions

  • Manage Your Events
  • Saved webpages and searches
  • Manage your subscriptions
  • Update personal information
  • Invite a colleague