tag:blogger.com,1999:blog-7975754875888992251.post8308444209757915026..comments2023-12-20T01:30:25.239-08:00Comments on Climate WTF: Merging Duplicate Station RecordsThe Blobhttp://www.blogger.com/profile/16066953906631389108noreply@blogger.comBlogger4125tag:blogger.com,1999:blog-7975754875888992251.post-62289634228417429142010-03-01T15:49:11.283-08:002010-03-01T15:49:11.283-08:00Your approach seems reasonable. Start simple, the...Your approach seems reasonable. Start simple, then revisit, experiment, and see what matters.<br /><br />Tamino only used the optimized method when combining different stations, not combining duplicates (which I think you know, just making sure). <br /><br />Some 'duplicates' look different enough that they could maybe be treated as entirely different stations. Happily this is not the rule.<br /><br />As for understanding how things work, when combining stations/duplicates that don't overlap or don't align well: I found it's best to just mess around with hypothetical data; things aren't always obvious until you see it in action.carrot eaternoreply@blogger.comtag:blogger.com,1999:blog-7975754875888992251.post-9622435923913731992010-03-01T12:11:38.564-08:002010-03-01T12:11:38.564-08:00"It's the ones with mismatched and non-ov..."It's the ones with mismatched and non-overlapping fragments that bother me"<br /><br />Me too and I want to revisit the method I use to merge duplicates to correct for as much as possible. What I am going to do in the meantime is just do things dead simple to get a global temperature plot and then go back and improve the method later. That can also give an idea of how much the method improvements change the final result.<br /><br />So at first for homogenization I am just going to average the trends of each merged station record in the gridbox.<br /><br />"oh, and are you always starting with record 0? If things are messy, it can matter where you start."<br /><br />yes I saw that I think on Chad's blog? I don't entirely understand why that's the case and my math/stat knowledge is too poor to try and fix it (I kind of understand Tamino's optimal least squares method, but have no idea how to implement such a thing..it's probably not best to follow someone else's method anyway). I will try starting with different records and hope that will show the order doesn't affect the final result much. For example I guess I could start with a random record for the merging and perform the analysis a few dozen times and see what kind of spread of results that produces.The Blobhttps://www.blogger.com/profile/16066953906631389108noreply@blogger.comtag:blogger.com,1999:blog-7975754875888992251.post-82859512363397646942010-02-28T17:35:29.177-08:002010-02-28T17:35:29.177-08:00oh, and are you always starting with record 0? I...oh, and are you always starting with record 0? If things are messy, it can matter where you start.carrot eaternoreply@blogger.comtag:blogger.com,1999:blog-7975754875888992251.post-12934390719894259572010-02-28T16:08:14.437-08:002010-02-28T16:08:14.437-08:00Your example is a happy yawn; nothing to worry abo...Your example is a happy yawn; nothing to worry about there. It's the ones with mismatched and non-overlapping fragments that bother me. A simple flag could just be looking for a calculated offset that's greater than some x.<br /><br />Of course, there are also some cases where you have mismatched fragments even within a single record - possibly due to a station move. Most people deal with that later in a homogenisation step, though I don't see any of the homebrew efforts tackling that yet. How ambitious are you feeling over there?carrot eaternoreply@blogger.com