SPOTLIGHT: There are structural reasons for the low quality of much scientific research.

BIG PICTURE: We tell children that science is a special source of knowledge. Scientific findings get tested by open and vigorous debate, we say, and can therefore can be relied upon.

But that isn’t true most of the time. As Richard Harris explains in Rigor Mortis: How Sloppy Science Creates Worthless Cures, Crushes Hope, and Wastes Billions, there’s “little funding and no glory involved in checking someone else’s work.”

The very thing that makes science special actually rarely happens. Thousands of scientific papers are submitted to journals each year. Those describing an intriguing new discovery are most likely to be published. No one demands that work be successfully replicated before publication. In Harris’ words, this means that

errors often only become evident years later, when a popular idea is finally put to the test with a careful experiment and suddenly melts away. A false lead can fool whole fields into spending years of research and millions of dollars of research funding chasing after something that turns out not to be true.

The IT world has long embraced “bug bounties.” Companies as diverse as Volkswagon, Nintendo, and Starbucks offer significant financial incentives to outsiders who can identify weaknesses in their computer code. Nothing like that exists in academic publishing.

Harris’s book explains that much medical research suffers from easily avoidable pitfalls: flawed design, dubious ingredients, poor lab techniques, or botched data analysis. In the United States alone, these shortcomings cost taxpayers an estimated $28 billion a year. That’s a lot of wasted resources.

TOP TAKEAWAY: Scientists, like everyone else, respond to incentives. In Harris’ words, many of our finest minds inhabit a world of “perverse incentives that discourage them from probing deeply enough to find out whether their exciting ideas are actually wrong.”

LINKS:

Rigor Mortis: How Sloppy Science Creates Worthless Cures, Crushes Hope, and Wastes Billions Richard Harris

• the $28 billion in annual wasted medical research is discussed here
• in this 2015 editorial, the editor of The Lancet medical journal declares that “much of the scientific literature, perhaps half, may simply be untrue.”
• see my previous commentary, Is science really self-correcting?

please support this blog

→ Receive posts via e-mail by signing up on the right side of this page, above – or by following this blog on Facebook and Twitter.
→ Download or e-mail a PDF of this post by clicking the Print button under Share This below – then select the blue arrow beside PDF at the bottom left.