GHCN v3 Timetable
August 10, 2010Posted by on
Way back (even before I started this blog) long, long ago (Dec 23rd 2009) in a reply to an email far away Dr. Peterson of NCDC told Willis Eschenbach that GHCN would be changing adjustment methods and that the new version should be ready in 2 to 3 months:
Partly in response to this concern, over the course of many years, a team here at NCDC developed a new approach to make homogeneity adjustments that had several advantages over the old approaches. Rather than building reference series it does a complex series of pairwise comparisons. Rather than using an adjustment technique (paper sent) that saw every change as a step function (which as the homogeneity review paper indicates was pretty standard back in the mid-1990s) the new approach can also look at slight trend differences (e.g., those that might be expected to be caused by the growth of a tree to the west of a station increasingly shading the station site in the late afternoon and thereby cooling maximum temperature data). That work was done by Matt Menne, Claude Williams and Russ Vose with papers published this year in the Journal of Climate (homogeneity adjustments) and the Bulletin of the AMS (USHCN version 2 which uses this technique).Everyone here at NCDC is very pleased with their work and the rigor they applied to developing and evaluating it.
They are currently in the process of applying their adjustment procedure to GHCN. Preliminary evaluation appears very, very promising (though of course some very remote stations like St Helena Island (which has a large discontinuity in the middle of its long record due to moving downhill) will not be able to be adjusted using this approach). GHCN is also undergoing a major update with the addition of newly available data. We currently expect to release the new version of GHCN in February or March along with all the processing software and intermediate files which will dramatically increase the transparency of our process and make the job of people like you who evaluate and try to duplicate surface temperature data processing much easier.
You can find the full email here: http://wattsupwiththat.com/2009/12/20/darwin-zero-before-and-after/#comment-272529
Well this of course caused a bit of interest far and wide, with a bunch of us waiting to see the differences between GHCN v2 adjusted and GHCN v3 adjusted, what new data was added to the files, which in turn would change the GISS output so there would be another comparison. However something must have happened on the way to debuting the files and programs because in an update to a paper submitted by Dr. Hansen of GISS for publication, he stated that GHCN v3 will not come out until late 2010:
When GHCN version 3 becomes available, expected in late 2010, we will make results of our analysis available on our website for both versions 2 and 3 for a period that is at least long enough to assess the effect of differences between the two versions.
That begs the question of what happened to delay the launch for over half a year? The adjustment process at the time of the email was in standard use for USHCN v2 so that shouldn’t have been too much trouble to come up with a program that took in the GHCN “raw” data instead of USHCN “raw” data. Possible areas that could have caused delay could be such as trying to come up with a Web Interface similar to what USHCN uses and not just rely on a ftp site to problems getting the program just right to spit out the intermediate steps which is not seen in USHCN v2. Another slightly less innocent reason for delay is based on the controversy of how the USHCN v2 adjustment method was devised.
Contrary to many, NOAA and NCDC were paying close attention to Anthony Watts and the Surface Stations Project ( http://www.surfacestations.org/). According to Mr. Watts after some initial contacts about possible collaboration between himself and NCDC about working on a paper/project to deal with what Mr. Watts had found, NCDC decided to go it alone. According to Mr. Watts; NCDC and Dr. Menne used an non quality controlled 43% complete dataset that Mr. Watts had made available online for general reference purposes only to perform an analysis and from that came up with the USHCN v2 adjusted method. According to Mr. Watts he has an almost 90% complete quality controlled dataset that he is working on with another climatologist to produce a journal paper. According to him their findings are vastly different to what Dr. Menne reached and he announced that on his blog Watts Up With That back in January of 2010. You can read the article here: http://wattsupwiththat.com/2010/01/27/rumours-of-my-death-have-been-greatly-exaggerated/
So with that announcement made just days before February rolled around NCDC might have taken a watch and see attitude, waiting to see when or if that paper makes it into the peer-reviewed lititchur and if it really does show their method is incorrect based on the underlying data.
Either way, don’t expect GHCN v3 until late October at the earliest would be my guess.