Tuesday 2 March 2010

GHCN work continues...

The slow progress of GHCN analysis continues, nothing to show just tidying up some code. Have generated records for all stations using the simple duplicate merge method, but haven't done anything with them yet. Instead I decided to put in place the ability to add new duplicate merging methods without losing the older methods and associated data. That should make it easier to compare different methods later. Nothing to show from doing this kind of stuff, but it should make work faster in the longrun. I am generally slower than most people, I notice quite a few people seem to have produce GHCN raw results in a couple of days. A number of people have already reproduced Tamino's result (http://tamino.wordpress.com/2010/03/01/replication-not-repetition/) and I am going to be too late that that party anyway :( all the drinks will have gone. Nevermind, it is good because now if my results don't match everyone elses I will know I've almost definitely done something wrong.

1 comment:

  1. Curious: Are you able to have everything in memory at once?

    I've avoided doing that, and either go grid box by grid box, or write intermediate files in order to save memory.

    ReplyDelete