SPOTLIGHT: A few weeks ago, The Economist ran an article titled Some science journals that claim to peer review papers do not do so. It not only missed the elephant in the room, it advocated unjust hiring practices.

BIG PICTURE: Peer review is a tool. Used by the scholarly publishing industry, it serves their needs. A couple of knowledgeable individuals are invited to critique a piece of research. Taking into account the feedback of those individuals, the journal then decides to publish the research or not.

In a report written two years ago, I referred to peer review as cargo cult science. Exhaustive attempts to prove it raises research quality have come up empty. On the other hand, its shortcomings (inconsistency, superficiality, subjectivity, inability to detect fraud, tribalism) are legion.

Over the years, academia has mythologized peer review. In that universe, it isn’t a mundane editorial tool, but a sacred process by which unpublished research gets elevated and anointed and transformed into ‘real’ work you can point to when applying for grants and promotions. If a series of pre-publication steps are followed, goes this cargo cult thinking, sound science is the end result.

One of the dirty little secrets of this mythology is that journals have always been at liberty to define peer review however they please. External oversight, enforcement, and minimum standards have never been part of the picture.

Recently, certain individuals and organizations have begun compiling blacklists of journals they suspect are only pretending to conduct peer review. Astonishingly, The Economist thinks this is a good idea. Even worse, it thinks universities should be “vigilant about checking candidates’ publication histories” against these blacklists when considering who to hire.

That would be profoundly unjust. A blacklist compiled in secret by anonymous people should not determine anyone’s job prospects.

There’s nothing open or transparent about these lists. We’re told one organization “employs 65 criteria to determine whether a journal” should be added to those “it believes are guilty” (my italics). However, a spokesperson is “reluctant to go into details.”

The rest of us have no way of evaluating the competence of these blacklisting efforts, or the fairness of their decision-making processes. Are journals given a chance to defend themselves? Who knows. The scope for errors and abuse is mind boggling.

TOP TAKEAWAY: The Economist thinks people should be denied a job because journals that published their work have landed on a shadowy blacklist. Whether the job applicant has even heard of the blacklist, and whether the journals are guilty or innocent is apparently immaterial.

LINKS:

Making Nature: The History of a Scientific Journal Melinda Baldwin

• Economist article, Some science journals that claim to peer review papers do not do so
• my 2016 paper: Peer Review: Why Skepticism is Essential
• Melissa Baldwin, Did Isaac Newton need peer review?
• in her book, above, Baldwin points out that “Nature published some papers without peer review up until 1973″

please support this blog

→ Receive posts via e-mail by signing up on the right side of this page, above – or by following this blog on Facebook and Twitter.
→ Download or e-mail a PDF of this post by clicking the Print button under Share This below – then select the blue arrow beside PDF at the bottom left.