The first independent report on Climategate was published today; Wednesday 31. March 2010. The UK Parliament's Science and Technology Committee announced that “Climate science must become more transparent”. Furthermore, the accompanying press release stated:
Phil Willis MP, Committee Chair, said:
"Climate science is a matter of global importance. On the basis of the science, governments across the world will be spending trillions of pounds on climate change mitigation. The quality of the science therefore has to be irreproachable. What this inquiry revealed was that climate scientists need to take steps to make available all the data that support their work and full methodological workings, including their computer codes. Had both been available, many of the problems at CRU could have been avoided."
The committee found that the focus on Director of CRU, Dr. Phil Jones had largely been misplaced and that his conduct related to data sharing had been in line with common practice.
Data policy is a matter for politicians, not scientists. I am therefore somewhat disappointed that the politicians in this S&T committee did not take full responsibility for it's own policy concerning data sharing. They rather put the blame (however little) on the University when the state:
“...The failure of the University to grasp fully the potential damage this [not sharing data following Freedom of Information] could do and did was regrettable. The University needs to re-assess how it can support academics whose expertise in FoI requests is limited.”
As far as I can tell there is obviously a conflict between several policies; Freedom of Information on one hand and restrictive rules of data sharing on the other. Both political decisions. This problem seem to be overlooked by the S&T committee – and everybody else for that matter. It is unfair to place the responsibility at the University.
An extract of the S&T committee's report can be found here on Scientificblogging.com. (by Patrick Lockerby)
A more detailed coverage of this problem can be found here Climategate – To Share Or Not To Share (6:33 min):
Article continues below the video.
In my view CRU has done it's best to share information about their data. It is openly available to everybody on their website.
Sharing Earth Observation Data
Open data policy is not a new topic (as some journalists seem to think). In 2003, a number of countries and international organizations created a forum, Group of Earth Observations (GEO) for high level discussions of overarching topics like open access to earth observation data. Essential climate variables ECV, such as temperature, of course included.
Global Data Sharing PolicyOne of the key goals for GEO is to establish guidelines for open data policy and it is described in the GEO Implementation Plan:
5.4 Data Sharing
The societal benefits of Earth observations cannot be achieved without data sharing.
The following are GEOSS data sharing principles:
• There will be full and open exchange of data, metadata, and products shared within GEOSS, recognizing relevant international instruments and national policies and legislation.
• All shared data, metadata, and products will be made available with minimum time delay and at minimum cost.
• All shared data, metadata, and products free of charge or no more than cost of reproduction will be encouraged for research and education.
Use of data or products does not necessarily imply agreement with or endorsement of the purpose behind the gathering of such data.National legislation and regulations
What is worth noting in these principles is first the wording “ recognizing relevant international instruments and national policies and legislation”. Even though the GEO members and participating organizations agreed on the fundamental idea of open data sharing, the wording of the principles could not conflict with national laws and regulations. The group of scientists in Climategate collected data from all over the world and had to respect a variety of rules and regulations, not only UK's.
Security and National Interests
It is also worth noticing the last phrase “Use of data or products does not necessarily imply agreement with or endorsement of the purpose behind the gathering of such data.”. This is included basically thanks to Brazil who'd have trouble with satellite data used to prove exploitation of their rain forest (which they deny). And such is modern earth observation, that we can gather a lot of information that used to be regarded as highly sensitive both for security and national interests. For the record; Brazil is not the only country with this 'problem'. It does point to a challenge for scientists who need data from a region that are under some political stress, or have an old-fashion view on data sharing. Like China's reluctance to share gravity data as they used to be sensitive data enabling rockets to find their way to the enemies territories. Today, high resolution bathymetric data is an example of data that are regarded highly sensitive by most nations. However, scientists can make a gentleman's agreements with good colleagues across boarders and get access to data that would never have been released by government or any official body. Obviously these data cannot be shared with people outside the group included in the gentleman's agreement, in some cases not even be mentioned as they would compromise helpful colleagues. Because it is for a common good, this practice is accepted.
Economy and Competition
I spend four years as Director of European Sea Level Service. Our main objective was to provide quality ensured sea level data from Europe to all users. Thus free open access to one of the essential climate variables. One of several epiphanies I had during that time was that when Meteorological organizations bragged about open access of their data, what they really meant was that within a group of met organizations they could all use each others data – after signing an agreement. Thus the data could not be made available to all users on the internet – as we wanted to do. Moreover, the closer you got to real time data, the harder it was to share with others. In this case it also a matter of business. Data are sold and give competitive advantages to companies. Some public organizations are obliged to make money on their data products and thus are not in a position to share the data freely.
Quality of data
Raw data should be shared, says the committee. Yes, indeed. But, the raw data is of no use if you do not know the quality of those data. Thus you need meta data; data about the data. Many things can go wrong when collecting data. Instruments can fail or simply have a built in error. The latter can be corrected if known, but if unknown will give more or less false data. The instruments can be moved by kids or workers or other unforeseen actions can take place. Meta data will also include information about these sort of things. We operate with the term “known quality”. The data can be of great use even if they are not perfect, but you need to know what quality they have to make best use of it. Sometimes, it is necessary to make so called derived product to make the raw data meaningful. That brings us to the question of prizing. Who should pay for the work of making those derived products? There might be intellectual rights involved etc.
The truth about transparency
What I miss in the Climategate statement and report from S&T Committee is an acknowledgement of the complexity of the issue of transparency and that they take on the responsibility to sort out the issues connected to open data policies. All they needed to do was to refer to UK's work in GEO. GEO was established as a forum for discussions of challenges of this character – that could not be solved without involvement of politicians.
UK S&T committee should have known about GEO and it's work particularly on data sharing policies. There are even Ministerial conferences – so there is no excuse really, not to know about this. There is a new chance to stay abreast at GEO Ministerial in China in November 2010 though.
I might be ignorant and unable to understand the 'saving of face' or underlying political strategy behind the UK Parliament's S&T Committee's statement. Because when it comes to sharing climate data, the truth about transparency is that politicians are responsible to create the framework for doing so first. Then we can talk about who break rules. I'll be bold enough to suggest politicians do their homework properly before the next reports are released.
You'll find Freedom of Information and UK Law a helpful introductory if you want to understand.
So far the name Climategate is undeserved. In my view even the BBC, here represented by blogger Richard Black, haven't got Climategate right. Black who in this relatively unbiased article says "...But in the main, it is in the established practices of scientists and their institutions that reform is urged -..."
Peer review is a well functioning scientific practice. It is the political framework that urgently needs to be changed.
Comments