Because it pays. From physicist Sabine Hossenfelder at BackReaction:
To the end of producing popular papers, the best tactic is to work on what already is popular, and to write papers that allow others to quickly produce further papers on the same topic. This means it is much preferable to work on hypotheses that are vague or difficult to falsify, and stick to topics that stay inside academia. The ideal situation is an eternal debate with no outcome other than piles of papers.
You see this problem in many areas of science. It’s origin of the reproducibility crisis in psychology and the life sciences. It’s the reason why bad scientific practices – like p-value hacking – prevail even though they are known to be bad: Because they are the tactics that keep researchers in the job.
It’s also why in the foundations of physics so many useless papers are written, thousands of guesses about what goes on in the early universe or at energies we can’t test, pointless speculations about an infinitude of fictional universes. It’s why theories that are mathematically “fruitful,” like string theory, thrive while approaches that dare introduce unfamiliar math starve to death (adding vectors to spinors, anyone?). And it is why physicists love “solving” the black hole information loss problem: because there’s no risk any of these “solutions” will ever get tested. More.
Well said. Hossenfelder worries that “if we in academia don’t fix our problems soon, someone else will. And I don’t think we’ll like it.”
Well, maybe it won’t happen that way. For one thing, many problems cannot be fixed by outsiders, meddle as they may. Traditions can simply decline, whether or not outsiders meddle.
The type of reform Hossenfelder seeks really must come from within.
Here’s one suggestion for an opening question: Is it true that human consciousness is an illusion? If so, what reform would make a difference?
See also: If this is science, yes we do hate it