Institutional incentives are giving way to less reliable scientific research, Paul Smaldino writes for The Conversation.
A great deal of research has some serious shortcomings, argues Smaldino, assistant professor of cognitive and information sciences at the University of California, Merced.
Some examples include:
- Published findings that do not replicate when repeated;
- Published findings that have smaller effects than purported;
- Studies conducted with small sample sizes;
- Correlations claimed in data that do not really exist; and
- Hypotheses developed after the fact.
According to Smaldino, the problem is that scientists are rewarded for measures of success that don't necessarily point to good research methods, such as:
- Number of peer-reviewed journals in which a study appears;
- Status of the journals in which studies are published; and
- Rate at which a researcher's work is cited by others.
"Publications, particularly in high-impact journals, are the currency used to evaluate decisions related to hiring, promotions, and funding," Smaldino explains. "Studies that show large and surprising associations tend to be favored for publication in top journals, while small, unsurprising or complicated results are more difficult to publish."
Administrators seek to deepen industry relationships to keep basic research afloat
This system that values visibility over accuracy in research reflects the principle that "when a measure becomes a target, it ceases to be a good measure," Smaldino writes. And as the cycle of shoddy science continues, he argues, more flawed research is produced.
Fortunately, Smaldino says, there are some efforts to reduce the power of institutional incentives, such as promoting:
- Open data;
- Publication of negative results;
- Holistic evaluations;
- Best practices; and
- Scientific integrity.
"There will always be researchers committed to rigorous methods and scientific integrity," Smaldino writes. "But as long as institutional incentives reward positive, novel results at the expense of rigor, the rate of bad science, on average, will increase" (Smaldino, The Conversation, 9/20).
Which colleges are winning at research? It depends on who you ask.
Next in Today's Briefing
Tight facilities budgets force difficult decisions