Freeman Dyson (1923–2020) explains:

I find it a miracle. I mean, I don’t pretend to understand it and I think it is absolutely marvellous that nature somehow thinks like a mathematician, that was what James Jeans said that… that God appears to be a mathematician. And it is astonishing that somehow all these weird mathematical ideas which we have invented for purely aesthetic reasons, essentially just as works of art, as intellectual constructions, turn up then unexpectedly to be used in nature. There’re so many examples of this, of course. Of course the classic case was differential geometry which was invented by Gauss for very practical purposes, just for projecting maps from the spherical earth onto a plane, onto a piece of paper, so he invented this differential geometry as a way of representing curved surfaces on a flat plane. And then 50 years later Riemann applied that to a description of space and conjectured that space itself might actually be curved, but it was still sort of purely an intellectual hypothesis without any kind of physical basis. And then another 50 years later it turned out to be the essential tool for Einstein to understand gravitation. It is in fact what Einstein used for general relativity. So it’s built… it’s built deep into the structure of space-time.

No wonder New Atheism petered out. Ultimately, maybe they couldn’t argue with the math.