Uncommon Descent Serving The Intelligent Design Community

Tabby’s Star — on the “extraordinary evidence” claim

Share
Facebook
Twitter
LinkedIn
Flipboard
Print
Email

If one watches the TED talk by Astronomer Tabetha Boyajian, one will notice that she begins with Sagan’s aphorism on “extraordinary” claims. This inadvertently reveals just how significant epistemological concerns are in scientific undertakings.

Accordingly, for follow up, I post a corrective:

The issue in knowledge is not extraordinary evidence (an assertion that invites selective hyperskepticism) but instead adequate warrant so that claimed knowledge is indeed warranted, credibly true (and so also reliable). END

PS:  It seems I need to add a clip I just made and annotated from a UKG paper on envisioning future scenarios for RW purposes, to illustrate a point on risk vs uncertainty i/l/o planning horizons — though, frankly, a U-UBSE (unknown unknown, black swan event) can hit you short term with v. little warning:

In for a penny, in for a pound. Let me add on the window of opportunity for change challenge i/l/o Machiavelli. This ties to how, often, after an over-the-cliff event, we can all indulge Monday Morning Quarterbacking — and to how hard it can be to be open minded enough to recognise weak, noise-beset signals of likely unwelcome realities and build enough of a critical mass to act in good time. and, statisticians and scenario modellers, I am looking straight at you:

See why I believe in war games, role-playing, BAU/ALT reflection and good old ZOPP?

Comments
Headlined: https://uncommondescent.com/mathematics/tabbys-star-3-the-business-of-dealing-with-black-swans/kairosfocus
January 19, 2018
January
01
Jan
19
19
2018
01:38 AM
1
01
38
AM
PDT
F/N: Let me clip Barsch as a public service for those dipping a tentative toe in the frigid, shark-infested waters of a chaotic black swan-prone environment:
Statisticians lament how few business managers think probabilistically. In a world awash with data, statisticians claim there are few reasons to not have a decent amount of objective data for decision making. However, there are some events for which there are no data (they haven’t occurred yet), and there are other events that could happen outside the scope of what we think is possible. The best quote to sum up this framework for decision making comes from the former US Defense secretary Donald Rumsfeld in February 2002: “There are known knowns; there are things we know we know. We also know there are known unknowns; that is to say we know there are some things we do not know. But there are also unknown unknowns – there are things we do not know we don’t know.” Breaking this statement down, it appears Mr. Rumsfeld is speaking about Frequentism, subjective probability (Bayes) and those rare but extreme events coined by Nassim Taleb as “Black Swans”. . . . . Rumsfeld seems to be saying we can guess the probability of the “known knowns” because they’ve happened before and we have frequent data to support objective reasoning. These “known knowns” are Nassim Taleb’s White Swans. There are also “known unknowns” or things that have never happened before, but have entered our imaginations as possible events (Taleb’s Grey Swans). We still need probability to discern “the odds” of that event (e.g. dirty nuclear bomb in Los Angeles), so Bayes is helpful because we can infer subjective probabilities or “the possible value of unknowns” from similar situations tangential to our own predicament. Lastly, there are “unknown unknowns”, or things we haven’t even dreamed about (Taleb’s Black Swan). Dr. Nassim Nicholas Taleb labels this “the fourth quadrant” where probability theory has no answers. What’s an illustration of an “unknown unknown”? Dr. Taleb gives us an example of the invention of the wheel, because no one had even though or dreamed of a wheel until it was actually invented. The “unknown unknown” is unpredictable, because—like the wheel—had it been conceived by someone, it would have been already invented. Rumsfeld’s quote gives business managers a framework for thinking probabilistically. There are “known knowns” for which Frequentism works best, “unknown knowns” for which Bayesian Inference is the best fit, and there is a realm of “unknown unknowns” where statistics falls short, where there can be no predictions. This area outside the boundary of statistics is the most dangerous area, says Dr. Taleb, because extreme events in this sector usually carry large impacts . . .
I add, that's where you need creative strategic thinkers who can suss out subtle signs and bring them to the table. Onward, I would lay out a game tableau and/or decision tree for a multi-player, multi-turn game against known players AND "nature" involving not only known patterns but potential high impact outliers brought to the table by the futurists. Then, we can proceed to a scenario based analysis on the outcome patterns of business as usual vs credible alternatives including black swans as well as internal-to-dynamics catastrophes. All of this is cast in the context of strategic decision-making. How we apply this to science is to imagine ourselves sitting on the board of a big-ticket journal, and deciding on what to publish; where, your journal also has a research grant budget so it can help shape the path of research. (High status institutions in Science do indirectly influence research funding.) Then, factor in the prudential principle of least regret on the downside and biggest windfall on the upside. Now, recommend your bets on an imaginary pool of grant funds for research and your recommendations on what to publish, knowing that this is likely to shift the balance of grant funding for future turns. What articles and proposals would you entertain, why? What SHOULD you be doing to hedge, to minimise likelihood of big regrets on both the downside and the up-side? Now, imagine you are the interested public, voting for the pols who provide the Journal's grant pool and influence priorities in part influenced by the journal's publications. Which pols will you most likely vote for, why? Which SHOULD you vote for, why? See the emergence of a problematique of thorny interacting, mutually reinforcing problems that can easily, blindly reinforce policies and ideologies that amplify exposure to destructive black swans? Now, look back at the OP and particularly the appended charts. Tell us if a lightbulb goes off. KFkairosfocus
January 19, 2018
January
01
Jan
19
19
2018
01:06 AM
1
01
06
AM
PDT
MR, you were repeatedly directed to Taleb and the black swan concept. If in a chaotic age you do not have an inkling about radical uncertainty with high potential impact and the need to respond to adequate warrant for the unwelcome, you need to come in out of the rain. KFkairosfocus
January 18, 2018
January
01
Jan
18
18
2018
01:51 PM
1
01
51
PM
PDT
I'm afraid I can't extract any signal from this noise, so I guess I'll leave you to it.mullers_ratchet
January 18, 2018
January
01
Jan
18
18
2018
11:52 AM
11
11
52
AM
PDT
JS, it should be clear that there are limits to statistical, probabilistic approaches such as Bayesian and likelihood approaches in general;which were raised by MR in response to the epistemological corrective highlighted in the OP. If you are not familiar with Taleb's Black Swan book and linked discussions, I suggest that you may find it useful to do some reading. Though in fact my own knowledge base on this comes from issues in strategic planning, finance, macroeconomics and sustainable development, also philosophy. All, i/l/o events on July 18, 1995 and thereafter, when a discontinuity in history utterly and radically reshaped the history of this nation, utterly invalidating all confident projections that were made hitherto -- and exposing the folly of business as usual and linked dismissiveness in the years that followed, ultimately costing people their lives. Well do I recall hearing radio discussions by ministers of government on how those entertaining visions of catastrophe were exhibiting neurotic behaviour and even notifying the public of plans for "subversives" -- the people who were trying to warn that we were dealing with a highly dangerous unknown and that optimistic probability estimates were ill-founded. The evidentialist fallacy has serious consequences. KFkairosfocus
January 17, 2018
January
01
Jan
17
17
2018
11:56 PM
11
11
56
PM
PDT
MR, with all due respect what I cited from Taleb and summarised from my own background is highly relevant to the "extraordinary claims" assertion by Sagan et al, and to the putting up of Bayesian analysis as a scientific cover for ideologically driven selective hyperskepticism. The point is, again, that the cases where we can construct a plausible probability distribution and use probability revision techniques or likelihood analysis are the nicely-behaved cases. Far too often we have to deal with uncertainty which opens up Taleb's Black Swans and Boyd's broken OODA loop. Rare, high impact, unexpected events are often an utterly dominant fact of life that shatters extrapolations and nicely calculated models. In that light, it is utterly imprudent to disregard adequate warrant for a result because the conclusion it points to is unwelcome and perceived as "extraordinary" leading to ill-advised imposition of arbitrarily biased hyperskepticism and usually groupthink to back it up. This obtains for ordinary life, for actuarial mathematics, for the court room, for strategic management, for sustainable development planning, for finance and investment, for programmes and projects, for macroeconomic planning, for government generally, for national security and the military, for engineering and for science. KFkairosfocus
January 17, 2018
January
01
Jan
17
17
2018
11:47 PM
11
11
47
PM
PDT
Well, JSmith, claiming FTL travel is an extraordinary claim outside of science fiction. And demonstrating FTL travel would be extraordinary given our current understanding. Claiming blind and mindless processes produced the variety of life starting from some simple replicators is an extraordinary claim. And given what we know about living organisms demonstrating such a thing would indeed be extraordinary.ET
January 17, 2018
January
01
Jan
17
17
2018
07:00 PM
7
07
00
PM
PDT
KF at 44 and 45, maybe I am clueless, but what does any of this have to do with whether or not extraordinary claims require extraordinary evidence. Too many of these threads get dragged off on bizarre tangents.JSmith
January 17, 2018
January
01
Jan
17
17
2018
04:51 PM
4
04
51
PM
PDT
Are you OK? This is not related to the topic of discussion at all, as far as I can tell.mullers_ratchet
January 17, 2018
January
01
Jan
17
17
2018
04:30 PM
4
04
30
PM
PDT
PPS: More from Taleb:
THE FOURTH QUADRANT: A MAP OF THE LIMITS OF STATISTICS (2008) Statistical and applied probabilistic knowledge is the core of knowledge; statistics is what tells you if something is true, false, or merely anecdotal; it is the "logic of science"; it is the instrument of risk-taking; it is the applied tools of epistemology; you can't be a modern intellectual and not think probabilistically—but... let's not be suckers. The problem is much more complicated than it seems to the casual, mechanistic user who picked it up in graduate school. Statistics can fool you. In fact it is fooling your government right now. It can even bankrupt the system (let's face it: use of probabilistic methods for the estimation of risks did just blow up the banking system). The current subprime crisis has been doing wonders for the reception of any ideas about probability-driven claims in science, particularly in social science, economics, and "econometrics" (quantitative economics). Clearly, with current International Monetary Fund estimates of the costs of the 2007-2008 subprime crisis, the banking system seems to have lost more on risk taking (from the failures of quantitative risk management) than every penny banks ever earned taking risks. But it was easy to see from the past that the pilot did not have the qualifications to fly the plane and was using the wrong navigation tools: The same happened in 1983 with money center banks losing cumulatively every penny ever made, and in 1991-1992 when the Savings and Loans industry became history. It appears that financial institutions earn money on transactions (say fees on your mother-in-law's checking account) and lose everything taking risks they don't understand. I want this to stop, and stop now . . .
--> BTW, we may be going through a Black Swan right now; with the discovery of a major architecture flaw in many families of microprocessors.kairosfocus
January 17, 2018
January
01
Jan
17
17
2018
03:53 PM
3
03
53
PM
PDT
MR, on Sept 10, 2001, what was the predicted likelihood of what happened the next day and changed the course of history? November 21, 1963? December 6, 1941, June 27, 1914? Just to pick one class of explosive break-points. (cf. here: https://www.quora.com/What-are-some-of-the-most-important-black-swan-events-in-history ) I am pointing out that likelihood ratios or Bayesian probability revisions or other approaches do not map uncertainty as opposed to risk very well, and that the Black Swan concept is a very important counter-weight to the notion that we have a handle on things down to a probability distribution. You raised issues on this line, I have responded and pointed to why although expert elicitation and the like can be useful, scenario based examination of possibilities under radical uncertainty and the capacity to be critically aware, intuitively open minded, deeply insightful and more built up through such are even more important. As in, John Boyd has a very serious set of points. And yes I am looking far beyond nice neat comparative hyps and likelihood ratios etc. That is to show in outline that Bayesian approaches have problems and beyond that this is actually a side-track from the demonstrable agenda of selective hyperskepticism in pursuit of ideological imposition. Or have you forgotten who wrote the book, The Demon-Haunted World? Where also, FYI, troll tolerance has gone to zero, if you think that clever insubstantial dismissive talking points are going to cut it. If you have substantial concerns, raise and address them. If not, don't take steps that will put you in the same box with the animus-driven trolls. KF PS: Taleb:
What we call here a Black Swan (and capitalize it) is an event with the following three attributes. First, it is an outlier, as it lies outside the realm of regular expectations, because nothing in the past can convincingly point to its possibility. [ --> see the problem of ideologically loaded question-begging evidentialism that makes a crooked yardstick the standard of straightness and accuracy, in the teeth of adequate evidence?] Second, it carries an extreme 'impact'. Third, in spite of its outlier status, human nature makes us concoct explanations for its occurrence after the fact, making it explainable and predictable. I stop and summarize the triplet: rarity, extreme 'impact', and retrospective (though not prospective) predictability. A small number of Black Swans explains almost everything in our world, from the success of ideas and religions, to the dynamics of historical events, to elements of our own personal lives. [The Black Swan, ch 1. Notice, ability to actually spot and use subtle signs would be enormously relevant, but the problem is we characteristically don't spot them or take them seriously. On Dec 7, 1941, the IJN raid was spotted on radar and it should have been known that radar had saved Britain the previous year, but the watch supervisor insisted it had to be an expected flight of B17's. I trust this helps you see why I corrected Sagan as in the OP.]
kairosfocus
January 17, 2018
January
01
Jan
17
17
2018
03:23 PM
3
03
23
PM
PDT
Yes, duller, you are sorry. And just because you are too dull to understand the answer doesn't mean it hasn't been given. In comment 37 you talked about evidence that was never previously provided. So if you want to eat word soup start with yours.ET
January 17, 2018
January
01
Jan
17
17
2018
01:57 PM
1
01
57
PM
PDT
I'm sorry, this is a word soup that doesn't answer the question at all.mullers_ratchet
January 17, 2018
January
01
Jan
17
17
2018
01:01 PM
1
01
01
PM
PDT
PS: Cf OP addendum on U-UBSE and envisioning uncertainty.kairosfocus
January 17, 2018
January
01
Jan
17
17
2018
05:26 AM
5
05
26
AM
PDT
MR, imposing a Bayes Theorem/Likelihood grid may be applicable to some situations but in fact the issues originally in view and in view as applied today are much broader. One key facet is the fuzzy border between risk and uncertainty. Risk deals with unknowns sufficiently understood to have some sort of reasonable probability estimation. Uncertainty deals with unknowns that we may be aware of but which are intractable and worse yet the unknown unknowns that we cannot see for any one of many reasons. In short the Bayesian/Likelihood model you have tried to impose to make Sagan's version of Cliffordian evidentialism seem more plausible and thus apparently reasonable and responsible -- when manifestly it is often used to impose ideologically driven hyperskeptical blinkers and soft nihilist power games -- simply does not fit the relevant domain. KFkairosfocus
January 17, 2018
January
01
Jan
17
17
2018
05:19 AM
5
05
19
AM
PDT
MR @ 37: I could see one thinking that a claim "out of the ordinary" would require evidence that "violates the ordinary", with an expectation that "the ordinary" is well understood and explained. However, it isn't. Simple example: The precession of Mercury. A readily observable phenomenon for centuries; explained often, maybe even to the satisfaction of many, never actually understood. Then comes along the extraordinary claim of General Relativity; a very counterintuitive little piece of mathemagic; to explain this and some other rather "ordinary" evidences.LocalMinimum
January 17, 2018
January
01
Jan
17
17
2018
04:40 AM
4
04
40
AM
PDT
duller:
Given those two pieces of evidence, do you think both claims are equally likely?
What evidence?ET
January 16, 2018
January
01
Jan
16
16
2018
04:54 PM
4
04
54
PM
PDT
I guess I'll ask you the same question as I did Joe in 34. Given those two pieces of evidence, do you think both claims are equally likely? Also, do you accept the "extraordinary evidence" phrase describes the application of Bayes Theorem (and that Bayes Theorem is true).mullers_ratchet
January 16, 2018
January
01
Jan
16
16
2018
04:50 PM
4
04
50
PM
PDT
MR, kindly stop trying to muddy a clear issue. It is a well known standard of empirical science that experimental results should be confirmed by multiple parties, preferably using diverse techniques. That criterion was never met. Observational sciences cannot repeat experiments so have weaker warrant. And, reconstructions of the remote, unobserved past of origins have yet weaker warrant. This is of course not far from a summary of Newton's rules. Of course, this has been too often suppressed in recent times and those who point it out have been unjustly pounced on in the media etc and treated as pariahs. KFkairosfocus
January 16, 2018
January
01
Jan
16
16
2018
04:21 PM
4
04
21
PM
PDT
muller- You have issues. You just spewed nonsense about what kairosfocus posted. Just own up to that fact. For the record, I tend not to believe anyone. Trust has to be earned. Someone tells me they flipped a coin and it came up heads I would ask them if that meant it was time to change their undies. Someone tells me they ran an experiment and found FTL travel and I would ask to see the experiment run again and again.ET
January 16, 2018
January
01
Jan
16
16
2018
04:14 PM
4
04
14
PM
PDT
OK Joe. Someone tells you they tossed a coin and it came up heads. Someone else tells you they ran an experiment that showed faster than light travel is possible. Do you believe each claim equally?mullers_ratchet
January 16, 2018
January
01
Jan
16
16
2018
04:08 PM
4
04
08
PM
PDT
muller:
So… because the evidence wasn’t very strong and the prior probability was so low you dismissed the result.
That doesn't follow from what you quoted. Do you have moral issues?ET
January 16, 2018
January
01
Jan
16
16
2018
04:05 PM
4
04
05
PM
PDT
I should point out, the "extraordinary evidence" phrase is just an informal way of describing an application of Bayes Theorem. I think even ID is probably not going to deny elementary mathematical theorems!mullers_ratchet
January 16, 2018
January
01
Jan
16
16
2018
04:03 PM
4
04
03
PM
PDT
DS, the original reports were tentative, and were soon withdrawn. There was no serious reason for a follow-up. If there had been confident reports and confirmation, that would be opening up a different context. KF
So... because the evidence wasn't very strong and the prior probability was so low you dismissed the result. But if the evidence was stronger (more "confident" or "confirmed") you would be more open to changing your view. Almost as if a priori improbable claims require stronger evidence than others...mullers_ratchet
January 16, 2018
January
01
Jan
16
16
2018
04:01 PM
4
04
01
PM
PDT
Dick:
Suppose someone were to assert that Sagan’s claim (actually Hume’s) that extraordinary claims require extraordinary evidence is itself an extraordinary claim.
Then they would have to support it. If Sagan/ Hume didn't present anything to support their claim then it can be ignored.ET
January 16, 2018
January
01
Jan
16
16
2018
03:57 PM
3
03
57
PM
PDT
Suppose someone were to assert that Sagan's claim (actually Hume's) that extraordinary claims require extraordinary evidence is itself an extraordinary claim. Would they be wrong, and, if not, what's the extraordinary evidence that supports Sagan's claim?Dick
January 16, 2018
January
01
Jan
16
16
2018
03:53 PM
3
03
53
PM
PDT
DS, the original reports were tentative, and were soon withdrawn. There was no serious reason for a follow-up. If there had been confident reports and confirmation, that would be opening up a different context. KFkairosfocus
January 16, 2018
January
01
Jan
16
16
2018
01:22 PM
1
01
22
PM
PDT
UB, yup. KFkairosfocus
January 16, 2018
January
01
Jan
16
16
2018
01:20 PM
1
01
20
PM
PDT
some sort of (e)mbedded mathematics, geometry or language certainly would be adequate.
Language requires a very specific physical organization. This organization is not only physically identifiable, but is exclusively found only in language systems -- with one exception. It is also the physical organization that enables the gene system. So, the very thing that would make a SETI claim of 'intelligent action' irrefutable, is found at the very origin of the living cell. And the double standards suddenly appear.Upright BiPed
January 16, 2018
January
01
Jan
16
16
2018
12:23 PM
12
12
23
PM
PDT
Where you stand depends on where you sit when it comes to applying the adjective, I suppose. KF's point that Sagan intended it to rationalize hyperskepticism is spot on to anyone paying attention. That said, personally, I find the design in biology and the fine tuning of the cosmos that have been uncovered since Sagan's demise to be extraordinary evidence for God. So strong, that his materialist heirs have been forced to extraordinary lengths to argue against it. A multiverse hypothesis that will forever remain impossible to verify experimentally? Extraordinary!jstanley01
January 16, 2018
January
01
Jan
16
16
2018
12:08 PM
12
12
08
PM
PDT
1 2

Leave a Reply