An unknown person posted a large amount of internal files from allegedly from CRU, which huge collection has become known as
The Hadley CRU file set To understand all this stuff, you need to know lots of climate science. I have only just started to go through this huge pile.
The original ftp server dropped the file (being stolen material and so on and so forth) and all those old links no longer work, but now the file is in bittorrent. The bittorrent link works with if you have installed a bittorrent client that support magnet links – magnet links being a highly decentralized way of publishing large files that does not expose any one server, router, or domain name to political pressure or possible reprisal, and prevents the illicit substitution of a changed file for the intended file. The file you get, will be the file I intend, which is not always the case with ftp or http links to politically sensitive data. The file is also available by http at such places as Megaupload, but pardon my paranoia, I don’t trust what they might do under pressure.
There is much preliminary analysis and discussion of this great pile of data
We can be pretty sure these files are genuine, since they explain the “science” behind some otherwise inexplicable published graphs that supposedly show the world warming up. These graphs are constructed pseudo scientifically. Rather than simply being pulled out of someone’s @%$#, they are constructed of numbers that reflect actual observations, but not observations of the quantity on the title bar of the graph.
Everyone is having lots of fun with this remark by Phil Jones:
I’ve just completed Mike’s Nature trick of adding in the real temps to each series for the last 20 years (ie from 1981 onwards) and from 1961 for Keith’s to hide the decline.
The decline to which Phil Jones refers is not the recent global temperature decline, which may well result from more accurate and more global methods of measuring temperature, which are therefore increasingly difficult to plausibly “correct”, but rather the failure of supposed temperature proxies to correspond to data supposedly derived from weather stations – the proxies are declining, so Phil Jones replaces the last few decades of the proxy, with the result that the last few decades of the graph for global temperature supposedly derived from the proxy agrees perfectly with the graph for global temperatures supposedly derived from weather stations, concealing the fact that there is no evidence that the proxy is in fact a proxy for temperature – indeed no evidence that either graph corresponds to global temperatures. Thus what is being fraudulently manufactured is not warming, but rather fraudulent agreement between various measures that supposedly measure warming.
The material seems psychologically genuine – they show conscious fraud that still retains much of the characteristics of self deception and unconscious cherry picking of data that it originated in.
There are just too many of these emails to be easily forged – you try writing many megabytes of text in the style of several well known people. Phil Jones has admitted them to be real, and is trying to spin some of his more embarrassing remarks, thereby drawing even more attention to them.
One thing apparent from the emails is the ease with which peer review can be captured by a clique and thus corrupted. This lesson is robust, because whether you agree with Mann that the climate skeptics have captured the peer review process at one journal, or you take the contrary view that the global warming clique has lost control of that one journal while retaining control of others, the result is the same: the peer review process can be captured and corrupted.
Peer review is not a necessary part of science. Science involves criticism, science filters truth from error by criticism and replication, but this does not have to be done by the mechanism of peer review. Peer review has the flaw that it is done relatively in secret and by a small, hand-picked number of reviewers: the wider community never sees the result until it has passed peer review. The same result – that is, a filtering of truth from error – could be achieved without this secretive method, and the secrecy of the method and the small number of reviewers introduces a vulnerability to capture which does not exist otherwise.
I don’t think peer review exists in order that science can be captured; I think its corruptibility is an accident. My tentative guess is that peer review exists to make a rapid decision about the quality of a work. A journal does not want to publish bad research but neither does it want to publish very old research (which has had enough time to be critiqued), nor does it want to publish already widely-known research (which research necessarily is if it is thoroughly critiqued), and so a journal will subject a paper to a preliminary review process – peer review. This is needed by journals for reasons specific to journals. It is not needed by science. And so the facile equation of peer review with scientific credibility that some make is false.
Additionally most of the papers published by journals are too trivial for anyone to want to read them unless they are assigned that as a task. In this role, journal publication a kind of vanity publication – publication for the sake of the author, not for the sake of the readers. Not to boost the author’s ego, but to pad his resume.
Constantinopoli:
Wikipedia is of course vulnerable to capture and has been captured, but a group of blogs that link to each other and to hostile blogs provide an alternative that is invulnerable to capture.
This alternative, however, requires significant work by the reader – you have to read and understand some critiques, and determine for yourself who is blowing smoke. Thereafter, you can trust some blogs and not others. This alternative fails to supply an authority that others can rely upon without significant work.
A fingerprint of the zip file could be widely disseminated in probably a legally safe way, so that those who use megaupload can verify that it was not tampered with.
But does anyone ever check file fingerprints? That is a job for a computer, not a person. A magnet link contains both a cryptographically strong checksum as uri, and a conventional url. If the checksum fails, you do not get a scary and hard to understand message about security. Instead the download just fails, and entities supplying data that fails to checksum get silently and automatically blacklisted. A magnet link provides no-click security.