Climategate: And finally, the source codes!

An explanation for declining to release the data in response to applications filed under the Freedom of Information Act (FOIA) has already been offered:

CRU data accessibility. From the date of the first FOI request to CRU (in 2007), it has been made abundantly clear that the main impediment to releasing the whole CRU archive is the small % of it that was given to CRU on the understanding it wouldn’t be passed on to third parties. Those restrictions are in place because of the originating organisations (the various National Met. Services) around the world and are not CRU’s to break. As of Nov 13, the response to the umpteenth FOI request for the same data met with exactly the same response. This is an unfortunate situation, and pressure should be brought to bear on the National Met Services to release CRU from that obligation. It is not however the fault of CRU. The vast majority of the data in the HadCRU records is publicly available from GHCN (v2.mean.Z).

There is a comment further down from a member of the Icelandic Meteorological Service which develops the explanation further. On the face of it, the fact that the CRU has thrown out some of the raw data is troubling. It would be interesting if any scientists who comment here could tell us if this is normal or not.

The fabrications and evasions keep piling up. If anyone still believes in global warming after all this, then all I can say is: they’ve got rocks in their heads.

The means by which this data was obtained may well be criminal. We have no way of knowing what, if anything, was done to the data by the hacker(s). If the researchers at the CRU have behaved improperly then they should suffer the usual penalties for academic misconduct. But until that has been shown, they are entitled to the presumption of innocence at the very least and I would hope that academics like yourself and Dr Dembski would uphold that right. Seversky

Hi everyone. Here's an UPDATE on the global warming fiasco. Emphases are mine (VJT). Climate change: this is the worst scientific scandal of our generation by Christopher Booker, in The Telegraph, November 28, 2009.

What we are looking at here is the small group of scientists who have for years been more influential in driving the worldwide alarm over global warming than any others, not least through the role they play at the heart of the UN's Intergovernmental Panel on Climate Change (IPCC).... There are three threads in particular in the leaked documents which have sent a shock wave through informed observers across the world. Perhaps the most obvious, as lucidly put together by Willis Eschenbach (see McIntyre's blog Climate Audit and Anthony Watt's blog Watts Up With That), is the highly disturbing series of emails which show how Dr Jones and his colleagues have for years been discussing the devious tactics whereby they could avoid releasing their data to outsiders under freedom of information laws... But the question which inevitably arises from this systematic refusal to release their data is – what is it that these scientists seem so anxious to hide? The second and most shocking revelation of the leaked documents is how they show the scientists trying to manipulate data through their tortuous computer programmes, always to point in only the one desired direction – to lower past temperatures and to "adjust" recent temperatures upwards, in order to convey the impression of an accelerated warming. This comes up so often (not least in the documents relating to computer data in the Harry Read Me file) that it becomes the most disturbing single element of the entire story.... What is tragically evident from the Harry Read Me file is the picture it gives of the CRU scientists hopelessly at sea with the complex computer programmes they had devised to contort their data in the approved direction, more than once expressing their own desperation at how difficult it was to get the desired results... The third shocking revelation of these documents is the ruthless way in which these academics have been determined to silence any expert questioning of the findings they have arrived at by such dubious methods – not just by refusing to disclose their basic data but by discrediting and freezing out any scientific journal which dares to publish their critics' work. It seems they are prepared to stop at nothing to stifle scientific debate in this way, not least by ensuring that no dissenting research should find its way into the pages of IPCC reports.

Climate Change Data Dumped by Jonathan Leake, Environment Editor, in Times Online, November 29, 2009.

Scientists at the University of East Anglia (UEA) have admitted throwing away much of the raw temperature data on which their predictions of global warming are based. It means that other academics are not able to check basic calculations said to show a long-term rise in temperature over the past 150 years. The UEA's Climatic Research Unit (CRU) was forced to reveal the loss following requests for the data under Freedom of Information legislation. The data were gathered from weather stations around the world and then adjusted to take account of variables in the way they were collected. The revised figures were kept, but the originals — stored on paper and magnetic tape — were dumped to save space when the CRU moved to a new building.... In a statement on its website, the CRU said: "We do not hold the original raw data but only the value-added (quality controlled and homogenised) data." The CRU is the world's leading centre for reconstructing past climate and temperatures. Climate change sceptics have long been keen to examine exactly how its data were compiled. That is now impossible.

The data has been lost. How convenient. The fabrications and evasions keep piling up. If anyone still believes in global warming after all this, then all I can say is: they've got rocks in their heads. vjtorley

On a related note: Democracy Now’s Amy Goodman says, "dissent is what will save us." Amy Goodman felt ‘violated’ when Canadian border officials grilled her for no apparent good reason. ... Canadian Border Services Agency officials ultimately allowed her to enter Canada but returned her passport with a document demanding she leave the country within 48 hours. This is what you get when you allow governments to inch closer to a totalitarian state. The moment you try to stifle one voice, however wrong you might think it is, the moment you inject new blood to the emergence of a kind of thought police in all aspects of your life. JPCollado

Evidence of tribal barriers in climate science in Steve McIntyre's Invitation:

Dear Dr Wigley, Would you be interested in writing a guest post for Climate Audit . . . I realize that there’s been sort of a community boycott against climate scientists speaking directly to the audience at Climate Audit. Despite this past, I think that it is important that climate scientists now speak directly to this audience (which presently extends far beyond the core audience.) . . . You may write whatever editorial you wish without any restrictions on my part and have an author password. Regards, Steve McIntyre Wigley replied: No thanks. Tom.

DLH

waterbear. I retract my first 'oops' steatement. That is, I can not (yet) say for certain that output of this code was publicised with the "ARTIFICALLY CORRECTED" data. However, the second 'oops' comment still stands i.e. They did publish the bogus hockey stick chart. JGuy

waterbear @ 17 The basis for calling this fraud need not rely on the discovery of that line of code. There is more evidence, e.g. the intended suppression of the skeptics. But the fact that they even thought to make such code - commented out in the source here or not - is a compelling evidence of a serious problem. JGuy

JGuy @13 I thought that coming to a conclusion that this is unquestionably the biggest fraud in the history of science on the basis of some comments in some out of hundreds of source code files would be possibly premature. But now you've asserted that the data they published had been processed by the particular program with this comment in it I realise I was wrong. waterbear

Waterbear at 9. What evidence do you have that it was not used? Conversely, see: The Code at Bishop Hill's blog

Mark in the comments notes a program comment: "Apply a VERY ARTIFICAL correction for decline!! followed by the words `fudge factor' " See briffa_sep98_d.pro.

See the associated detailed discussion of the code.

; Apply a VERY ARTIFICAL correction for decline!! ; yrloc=[1400,findgen(19)*5.+1904] valadj=[0.,0.,0.,0.,0.,-0.1,-0.25,-0.3,0.,-0.1,0.3,0.8,1.2,1.7,2.5,2.6,2.6,$ 2.6,2.6,2.6]*0.75 ; fudge factor

The programmer himself states it was being used as a "correction for decline!!" or "hiding the decline" per Jones and Mann. With the clear statement of it being used, you / Jones/Mann bear the burden of proof to show that it was not used. DLH

waterbear Even though I think they could classify as terrorists of some kind. I never would expect any of them to go to gitmo. It's hyperbole that alluded you. However, I do think they should be prosecuted very severly. And again, any whistelblowers I'd expect to receive significant leniency. JGuy

@JGuy 1, 12.

I think you have a case for treason and fraud – even terrorism. ... Send them to Gitmo. ... Even if anyone were to go to Gitmo, I wouldn’t suggest sending any US citizen. Keep in mind, this is not an American fraud. It’s international.

Britain is unlikely to agree to surrender citizens to be sent to Gitmo, and will likely point out that US courts extradite and prosecute and punish foreign fraudsters by the normal route. As an approach to the UK authorities by normal channels will keep these people out of Gitmo, this would leave the option of snatching them covertly. Would it be best to do this now, or to wait for more evidence to come to light, risking that guilty parties might cover their tracks while we wait? waterbear

waterbear @ 9

Dr Dembski, According to this post looking at the briffa_sep98_d.pro file the “fudge factor” in the valadj matrix highlighted by the author of the American Thinker article is not actually used anywhere in the program so does not affect the end result of the data being processed by the program. But IDL is not a language of mine just as it is not a language the author of the American Thinker article is familiar with, so we could both be wrong. Does the “fudge factor” being unused agree with your analysis of the leaked source code?

Waterbear! You're are missing the elephant in the room. Sure, they have semicolons to diable parts of code in THIS rendition. But that is something programmers can do to test code with and without the extra code. Can you come up with a benevlonent reason as to why they would make "unused" code to hide a decline? Here are a few lines of that code: ; ; Apply a VERY ARTIFICAL correction for decline!! ; ; title='Northern Hemisphere temperatures, MXD and corrected MXD' title='Northern Hemisphere temperatures and MXD reconstruction' Explain why they would have a section of code for a VERY ARTIFICIAL correction? Think about it. Now, think about which versions of the plot was publicised? Ooops. Don't forget the hockey stick chart...ooops again. Is it clicking yet? JGuy

waterbear @ 7 Even if anyone were to go to Gitmo, I wouldn't suggest sneidng any US citizen. Keep in mind, this is not an American fraud. It's international. Any americnas involved can join Madoff and his multiple life term. JGuy

Highly recommend climate scientist Dr. Judith Curry's two recent opinions: An open letter to graduate students and young scientists in fields related to climate research (Mirrored)

In grappling with this issue, I would argue that there are three strategies for dealing with skeptics: 1. Retreat into the ivory tower 2. Circle the wagons/point guns outward: ad hominem/appeal to motive attacks; appeal to authority; isolate the enemy through lack of access to data; peer review process 3. Take the “high ground:” engage the skeptics on our own terms (conferences, blogosphere); make data/methods available/transparent; clarify the uncertainties; openly declare our values

And "Curry: On the credibility of climate research"

1. Transparency. Climate data needs to be publicly available and well documented.. . .In summary, given the growing policy relevance of climate data, increasingly higher standards must be applied to the transparency and availability of climate data and metadata. These standards should be clarified, applied and enforced by the relevant national funding agencies and professional societies that publish scientific journals. 2. Climate tribalism. . . . The motivation of scientists in the pro AGW tribes appears to be less about politics and more about professional ego and scientific integrity as their research was under assault for nonscientific reasons. . . .Hence it is difficult to understand the continued circling of the wagons by some climate researchers with guns pointed at skeptical researchers by apparently trying to withhold data and other information of relevance to published research, thwart the peer review process, and keep papers out of assessment reports. . . .Particularly on a topic of such great public relevance, scientists need to consider carefully skeptical arguments and either rebut them or learn from them. Trying to suppress them or discredit the skeptical researcher or blogger is not an ethical strategy and one that will backfire in the long run. . . . But the broader issue is the need to increase the public credibility of climate science. This requires publicly available data and metadata, a rigorous peer review process, and responding to arguments raised by skeptics. The integrity of individual scientists that are in positions of responsibility (e.g. administrators at major research institutions, editorial boards, major committees, and assessments) is particularly important for the public credibility of climate science. The need for public credibility and transparency has dramatically increased in recent years as the policy relevance of climate research has increased. The climate research enterprise has not yet adapted to this need, and our institutions need to strategize to respond to this need.

(Be patient. CA is overloaded at present. Discuss at new CAmirror) Article is at: at ClimateAudit.org Curry's discussion of "Tribalism" in science is insightful. That well describes some of the conflicts between Neo-Darwinian advocates vs ID. DLH

waterbear. I was not linking to the senators webwsite to get people sent to gitmo. My intent was for us to let the senators know we are heavily tracking on this fraud, and that any bills or legislation inspired by the presumption of stopping global warming be rejected with a big "NO" vote. Also, we can let them know we want them to investigate this fraud to it's source. And prosecute the batch, unless there is an invovled whistle blower, in which case he/she is granted mercy. JGuy

Dr Dembski, According to this post looking at the briffa_sep98_d.pro file the "fudge factor" in the valadj matrix highlighted by the author of the American Thinker article is not actually used anywhere in the program so does not affect the end result of the data being processed by the program. But IDL is not a language of mine just as it is not a language the author of the American Thinker article is familiar with, so we could both be wrong. Does the "fudge factor" being unused agree with your analysis of the leaked source code? waterbear

These people should get no less than 20 years in prison and there should be public protests and demonstrations until there is legal action. I have been reading the emails at http://www.eastangliaemails.com and noticed a lot of these guys are Russian Jewish scientists. Not that there is anything wrong with Russians or Jews but given the US's historical relationship with Russia and the nature of the proposed legislation for this AGW theory- I think it is definitely noteworthy of pointing out. The implications for global or international policy makes this scandal BIGGER than the Madoff scandal. Why isn't CNN and MSNBC and most of the big criminal failng newspapers raising awareness of this? Becauee Madoff was all part of their anti-american, anti-free market ideaology- and this Clmategete is totally AGAINST their one world governemnt, nanny state Ideaology. Frost122585

@DLH #6:

Simple: The corruption involves US climate scientists. Many of those involved in the ClimateGate emails are in the US taking US taxpayer funds. e.g. Michael Mann.

I see. Gitmo is an appropriate destination for US citizens suspected of fraud? waterbear

waterbear Simple: The corruption involves US climate scientists. Many of those involved in the ClimateGate emails are in the US taking US taxpayer funds. e.g. Michael Mann.

Mr. Inhofe, the senior Republican on the Senate Environment and Public Works Committee, sent letters to many of the scientists whose e-mail messages were made public and to a number of United States government agencies, asking them to preserve all correspondence as the first step in his inquiry.

See: Inhofe seeks probe of climate science Furthermore, evidence in the UK can be obtained to convict criminals in the US by letters rogatory. DLH

This climate change business never was about the science anyway. Its aboout power, the redistibution of wealth, the lining of the pockets of Al Gore, GE, George Soros and an assault on our sovereignty. Because of this I do not expect this to permanently derail the climate change cabal. There are way to many powerful interests that have to much at stake just to walk away. As for the true believers no amount of dedunking is going to change their thinking. They have embraced this powerful delusion. Crowd psychology has taken over. One need only to read "Popular Delusions and the Madness of Crowds" ( one of the books that started me questioning Darwinism) to see how history is replete with delusions caused by group think. Vivid vividbleau

Send them to Gitmo. ... Email your Senator – simple – because it’s all web-based

Thankyou for the link. How should I word a request to my Senator that he seek the detention, and transportation to Gitmo, of people at a university in England? I don't know whether the FBI have some jurisdiction in this matter or if it is a case best handled by Navy SEALs under the cover of darkness. waterbear

Sorry for the Obama slogan quip :P By the way, In the above link, under your senator, you will see the URL/link for the Web Form. From there, you can send a message to your senator. JGuy

This is a simple way to do something constructive now. Email your Senator - simple - because it's all web-based: http://www.senate.gov/general/contact_information/senators_cfm.cfm Yes you can! (I did) JGuy

Every one of these liars should be removed from their respective institutions and forced to find a new line of work, as was once the practice when academic fraud was uncovered[...]

Why such light punishment? Isn't this more than an intellectual crime. Disregarding a fame motive, if you could link it to an ideaology or money, then I think you have a case for treason and fraud - even terrorism. Sternberg used this term before under a different cicumstnace, but here it would be most applicable. These scientists would be actual intellectual terrorists. Afteall, the purpose of global warming hysteria was to instill fear to force an ideaology. Send them to Gitmo. JGuy