Intelligent Design Peer review

Get rid of the science paper?

Spread the love

At the The Guardian, here’s a thought from a prof at King’s College. He begins with an overview of all the problems, then,

The solution to all these problems is the same as the answer to “How do I organise my journals if I don’t use cornflakes boxes?” Use the internet. We can change papers into mini-websites (sometimes called “notebooks”) that openly report the results of a given study. Not only does this give everyone a view of the full process from data to analysis to write-up – the dataset would be appended to the website along with all the statistical code used to analyse it, and anyone could reproduce the full analysis and check they get the same numbers – but any corrections could be made swiftly and efficiently, with the date and time of all updates publicly logged.

This would be a major improvement on the status quo, where the analysis and writing of papers goes on entirely in private, with scientists then choosing on a whim whether to make their results public.

Stuart Richie, “The Big Idea: Shoud we get rid of the science paper” at The Guardian (April 11, 2022)

Could we try it for five years and see?

6 Replies to “Get rid of the science paper?

  1. 1
    Querius says:

    Interesting idea about transparency, but . . .

    What I think is missing is a method of collecting/tagging the proposed notebook websites. Perhaps the transparency should also include commentary and a modified slash-dot rating system for several factors: clarity, methodology, statistical analysis, and significance.

    However, the following problem still remains: There will be a spectrum of notebooks that range from

    a. Politically correct notebooks that faithfully support nuances of the mainstream narrative
    b. A flood of complete idiot-whacko and computer-generated papers numbering in the billions

    I’m not sure that the cure is better than the malady.

    -Q

  2. 2
    Fasteddious says:

    Agree with Querius @1.
    To maintain any sort of science exchange medium that is useful to scientists requires some sort of gatekeeper. Today we have “peer review” (which may be little more than “name recognition?) and we’ve seen how that is falling apart. There are probably numerous other ways to guard the gate, and doubtless many have been tried in various venues. Indeed, there are probably papers about science journal gatekeeping! But that just raises the bar to defining what makes a good gatekeeper; which would be as open to ideological mischief as any individual gatekeeping approach.
    I don’t have a solution, but the one offered in this article seems unlikely to work.

  3. 3
    polistra says:

    Transparency of intermediate results is a terrible idea. Trial and error requires a certain amount of looseness and privacy. If every try and every misspelled word and every correctible math error must be seen by everyone in the world, students and profs will find ways to avoid showing the intermediate work. The net result will be the same as it is now, except for a whole lot more wasted effort.

    Proposals for transparency ALWAYS end up driving real work farther underground.

    Worst of all, a truly fresh idea will be seen early by competing teams with more resources, who will then steal and frontrun it.

  4. 4
    Querius says:

    Polistra @3,

    Good point. Unfortunately, there’s no amount of skullduggery and fraud that some people are willing to stoop to get themselves published. Even the obscuring language and convoluted expression in most scientific papers is simply puffery in the name of science.

    What might be interesting is the concept of “living” documents that result in a series of corrected versions, addenda, or retractions as challenges, new information, and additional studies are published.

    Yeah, probably also unworkable.

    -Q

  5. 5
    polistra says:

    Yup, there’s plenty of skulduggery and spying. But opening up the early stages of research would make the spying too easy.

  6. 6
    kairosfocus says:

    Simple, both and. Publish the paper with a link to the notebook. BTW, these exist as an electronic version of the old fashioned lab notebook. Look up, Anaconda Project for Python. KF

Leave a Reply