Uncommon Descent Serving The Intelligent Design Community

Crowdsourcing: A new way of finding solutions faster, using large amounts of data

arroba Email

Mapping Dark Matter
In “From Bondi to the big bucks: the 28-year-old who’s making data science a sport” (The Age, November 4, 2011), Asher Moses reports on how it’s catching on:

“To date Kaggle has crunched data on dark matter, predicting which used cars are likely to be bad buys, improve the World Chess Federation’s official chess rating system, and predicting the likelihood that an HIV patient’s infection will become less severe, given a small dataset and limited clinical information,” Kaggle claims.

NASA’s dark matter competition was actually won by a British glaciologist from the University of Cambridge, Martin O’Leary. The solution was a mathematical model for the tiny distortions in images of the galaxy, thought to be dark matter.

NASA, the European space agency and others had been working on the problem for 10 years but O’Leary found the solution in a week and a half. The effort garnered him a mention on the White House website, which compared him to Albert Einstein and Isaac Newton.

Hmmm. There is a certain “bubblicious” quality to all this, but one factor may help explain it: Freelancers working solo or in tandem for large rewards may perform better than employees working within a corporate bureaucracy, simply because they experience fewer barriers to a solution that works.

Put another way: O’Leary didn’t have hundred people over him in a dark-matter bureaucratic hierarchy, where even getting his solution considered would be 90% of the battle.

Indeed, it is simply a fact of life that, at many firms, the admin assistant could tell the boss ways to trim expenses and boost productivity. But her low place in the hierarchy means that she will be – at best – indulged, not listened to … she is “having one of those days, you know.”

Wait till crowdsourcing hits the streets.


Leave a Reply