academic freedom Intellectual freedom Intelligent Design Peer review

At Nature: Change how we judge research. Hmm…

Spread the love
What’s hot? What’s not?/Niklas Bildhauer, Wikimedia

From Stephen Curry, professor of structural biology and assistant provost for equality, diversity and inclusion, at Nature:

The San Francisco Declaration on Research Assessment (DORA) … Conceived by a group of journal editors and publishers at a meeting of the American Society for Cell Biology (ASCB) in December 2012, it proclaims a pressing need to improve how scientific research is evaluated, and asks scientists, funders, institutions and publishers to forswear using journal impact factors (JIFs) to judge individual researchers.

Most agree that yoking career rewards to JIFs is distorting science. Yet the practice seems impossible to root out. In China, for example, many universities pay impact-factor-related bonuses, inspired by unwritten norms of the West. Scientists in parts of Eastern Europe cling to impact factors as a crude bulwark against cronyism. More worryingly, processes for JIF-free assessment have yet to gain credibility even at some institutions that have signed DORA. Stories percolate of research managers demanding high impact factors. Job and grant applicants feel that they can’t compete unless they publish in prominent journals. All are fearful of shrugging off the familiar harness.

“Fearful of shrugging off the familiar harness”? Anyone who knows anything about harnesses will realize that the whole point of a harness is that the harness-ee cannot shrug it off.

Anyway, decision-makers hear lots of soaring rhetoric but feel no pressing need. Smart money doesn’t take the risk.

I favour concise one- or two-page ‘bio-sketches’, similar to those rolled out in 2016 by the University Medical Centre Utrecht in the Netherlands. These let researchers summarize their most important research contributions, plus mentoring, societal engagement and other valuable activities. This approach could have flaws. Perhaps it gives too much leeway for ‘spin.’ But, as scientists, surely we can agree that it’s worth doing the experiment to properly evaluate evaluation. More.

But what would protect the researcher who submits the suggested bio-sketch from becoming a target for political reasons that are unrelated to research quality? Think Jordan B. Peterson or Gunter Bechly. Or anyone who sounds like a risk for blowing the whistle on corruption. The fate of whistleblowers is already often grim.

Curry’s idea is interesting but the flaw is that, very often, the reason for silencing the person is to silence the ideas themselves. That is what matters most in the long run. A background in diversity grievance politics is not the best position from which to see that problem.

See also: Lonely are the brave: What happens to whistleblowers Retraction Watch offers watch-yer-back advice for whistleblowers

Human evolution 2018: Not only upended icons but suspicious relics

From Technology Review: How science gets morphed into propaganda


Does it matter in science if no one can replicate your results?

One Reply to “At Nature: Change how we judge research. Hmm…

  1. 1
    polistra says:

    The correlation between fashion and quality has always been low, in any field from science to literature to technology to music.

    Probably not a negative correlation … occasionally the best material is popular … but certainly a weak correlation.

    The best thing we can do is to insure that unpopular work is PRESERVED in a form that will be readable and findable two generations later, after current fashion is finally recognized as destructive and deadly.

Leave a Reply