The message from Terry Fields contains these words:
I will cut and post the figures here then, not having any active webspace nor any experience of ftp.
As the figures will be in a ms works spreadsheet I thought it would make things easier for you as excel should be able to read such files.
I might manage to post as early as tomorrow. OTOH I might not. :-)
Hmmm...that might count as binary file, which is verboten in text groups :-( and it would mean that others wouldn't see the data. I'm limited by experience and only having Excel to work with - perhaps someone else with an interest could do something more sophisticated that compare polynomials...
Comma delimited is OK.
The message from Roger contains these words:
Or even today. Figures below are for the unsmoothed annual anomaly from
1850 to 2007. I also have the figures for the smoothed curve if anyone is interested. I used them and the smoothing filter to check for extraction errors. I think most of the figures are accurate to 0.01 but I suspect there may be some minor distortion in the graph which might make matters worse.The smoothed curve and my smoothing exercise have a reasonable fit with only 5 years differing by 0.01 or more. (1850 - 0.023,1851 - 0.14, 1955 & 1956 - 0.01 opposite signs), 2007 - 0.014).
Over to you Terry.
-0.12
-0.03
-0.07
-0.09
-0.04
-0.01
-0.16
-0.21
-0.25
-0.14
-0.1
-0.29
-0.34
-0.11
-0.16
-0.01
0-0.05
0.13 0.02 0.03 0.02 0.02-0.01
-0.13
-0.05
-0.04
0.29 0.31 0.06 0.07 0.05 0.07 0.04-0.04
-0.06
0.02-0.05
0.03 0.13-0.1
-0.05
-0.1
-0.17
-0.11
-0.02
0.14 0.1-0.1
0.05 0.14 0.03-0.1
-0.19
-0.22
-0.03
0-0.17
-0.25
-0.28
-0.27
-0.28
-0.2
-0.18
-0.03
0.06-0.18
-0.25
-0.07
-0.1
-0.04
0-0.09
-0.05
-0.08
0.01 0.1 0.03 0.04-0.09
0.12 0.14 0.11-0.07
0.08 0.08 0.12 0.24 0.25 0.29 0.3 0.35 0.23 0.25 0.37 0.25 0.06 0.08 0.04 0.05-0.05
0.11 0.19 0.23 0.06-0.01
-0.04
0.2 0.26 0.17 0.14 0.24 0.25 0.24-0.03
0.06 0.13 0.14 0.15 0.27 0.21 0.11 0.23 0.34 0.06 0.14 0.05 0.34 0.24 0.34 0.38 0.41 0.29 0.47 0.27 0.26 0.31 0.45 0.45 0.38 0.53 0.48 0.34 0.38 0.45 0.56 0.41 0.64 0.8 0.54 0.52 0.68 0.74 0.74 0.72 0.76 0.71 0.69
A labour of love....
Thank goodness Excel allows one to fill in a data series; I'd have hated to have typed in all the years from 1850 - 2007 :-(
I've carried out the various analyses that Excel allows: moving average of chosen span; polynomials of various orders; linear regression; and logarithmic.
The last two produced silly trendlines and weren't proceded with.
Polynomials of second and third order produced interesting trendlines.
Moving averages of 3 and 10 points were carried out.
Results here:
10 point moving average:- only the 5-point moving average picks up a downturn circa 2004 or so
- the 3rd order polynomial has a slightly higher correlation coefficient than the 2nd order. The data at the 1850 end of the scale suggests recover from the Little Ice Age....
Discuss....;-)
[if anyone can do more sophisticated analyses than allowed by Excel, I'm sure the results would be of interest]The message from Terry Fields contains these words:
I think you have discovered why the Met Office have chosen a 21 point binomial filter to smooth the curve. The more 'sophisticated' filters ignore short term trends.
I would be interested to see how the polynomial filters would cope with a subsequent downturn so how about inverting the 1950 - 2004 about 2005 (2006 = 2004 .... 2060 = 1950) and then doing several finish dates to see when the downtown can first be clearly seen as well as what a mature downturn would look like.
Interesting, but see below.
Now done that; results here:
Second-order polynomial:
- the polynomials are, well, rubbish
- the 10- and 20-point moving averages shift everything to the right
- the 5-point moving average picks up the downturnclearly afetr 2 to
3 years, as might be expected.The thing that appears to come out of this exercise is that to pick up trends at the earliest opportunity, long moving averages are not the tool to use. Hadley seems to use a (weighted) 20-pointer; I'll bet they've done every kind of data run, in a more sophisticated manner then we have, and have a good idea of the current position.
One of the bloggers said that the 'downturn' data doesn't appear on the Hadley's 'Myth' page....I can't help wondering why. Perhaps, like Micawber, they're waiting for something to turn up (or down).
The message from Terry Fields contains these words:
Well they are no use for this sort of application anyway.
The Met Office filter is 21 points and seems to do the job very well except for the limits of the sequence and even there I am coming to the view that it is difficult to specify something else that lessens the weighting effect of the final year without introducing another factor potentially at least as bad.
It is generally accepted that El Nino and La Nina episodes have a significant warming or cooling effect respectively so the Met Office may well have a graph somewhere that attempts to compensate for those factors.
Those who see a definite trend already are cherry picking their data. Given a few more years the downturn will become evident if it is anything other than a transient event. The figures below are for the smoothed curve 1850 - 2060 using the inverted around 2005 which shows the downturn clearly and centred on 2005. (I hope no one picks these figures up and takes the forward projection as gospel.)
-0.09
-0.08
-0.08
-0.08
-0.09
-0.11
-0.13
-0.15
-0.17
-0.19
-0.19
-0.19
-0.18
-0.15
-0.12
-0.07
-0.04
-0.01
0.01 0.02 0.02 0.01-0.00
-0.01
0.00 0.03 0.06 0.09 0.11 0.11 0.09 0.07 0.05 0.02 0.01-0.00
-0.00
-0.00
-0.01
-0.02
-0.03
-0.05
-0.06
-0.06
-0.05
-0.02
0.00 0.02 0.03 0.02 0.00-0.03
-0.06
-0.09
-0.11
-0.12
-0.14
-0.17
-0.19
-0.21
-0.22
-0.21
-0.18
-0.15
-0.13
-0.12
-0.11
-0.11
-0.10
-0.09
-0.07
-0.06
-0.05
-0.04
-0.02
-0.00
0.01 0.02 0.03 0.04 0.05 0.06 0.07 0.07 0.09 0.12 0.15 0.19 0.23 0.26 0.27 0.28 0.27 0.26 0.23 0.19 0.15 0.11 0.09 0.07 0.08 0.09 0.10 0.10 0.10 0.10 0.11 0.13 0.15 0.17 0.18 0.18 0.17 0.15 0.14 0.13 0.13 0.15 0.16 0.18 0.19 0.19 0.19 0.19 0.18 0.19 0.21 0.24 0.27 0.31 0.33 0.35 0.35 0.35 0.35 0.35 0.37 0.39 0.41 0.42 0.43 0.44 0.44 0.45 0.47 0.50 0.54 0.57 0.60 0.62 0.64 0.66 0.69 0.71 0.72 0.72 0.72 0.71 0.69 0.66 0.64 0.62 0.60 0.57 0.54 0.50 0.47 0.45 0.44 0.44 0.43 0.42 0.41 0.39 0.37 0.35 0.35 0.35 0.35 0.35 0.33 0.31 0.27 0.24 0.21 0.19 0.18 0.19 0.19 0.19 0.19 0.18 0.16 0.15 0.13 0.13 0.14 0.15 0.17 0.18 0.18 0.17 0.15 0.13 0.11 0.10 0.10 0.10 0.08 0.06 0.03
What smoothing technique did you use?
This is how your data looks in graphical form here:
Of course. Phase delay..
I am a bit curuous as to where the data from the years 2010-2050 comes from...
The standard advice when using these to e.g. track stock prices is to buy/sell when the data crosses the average. This is margnally better than tossing dice or gazing into a crystakl ball.
Hadley seems to use a (weighted) 20-pointer; I'll bet
Roger suggested inverting the data from 1950 - 2005 and adding it on from 2006, to give a mirror image of the datatset based about 2005. The idea was to try to identify as soon as possible where any downturn could be confirmed. The second tranche of graphs was that exercise.
The data seem to be an exact mirror image of the preceding years? Simple enough to do but meaningless?
Oh, I see. Using pictures rather than maths to understand the filter characteristics.
Well you have completely reinvented the wheel, and discovered that predicting the future needs a model. That can only be proven to be correct (or not) once the future has happened.
The triumph of science is to pick models other than simple polynomials, sine waves and the like, that reflect what seems to be the underlying
*mechanisms*. And in this case thats a definite plural.Which is what climate change science is all about.
Just peering at the data and applying random simplistic filters wont actually get you very far at all.
As I said, if what you are looking for is sine waves, what you will get is sine waves.
I don't have to do all this: I did it years ago when working on software for a digital sampling and storage oscilloscope: when we got up towards the sampling frequency, we had various filters we could apply: the sine interpolation filter attempted to fit a sine wave to the data points. It always managed to do precisely that, irrespective of how sinusoidal the original signal might have been, whilst a moving average more or less wiped out te data at thse frequencies completely. It was better, in that it wasn't subject to such wild extrapolations, but it was worse, in that the bandwidth was totally lost.
In short, you cant do prediction with these sorts of brute force curve fits.
In terms of the actual climate change models so far proposed, there are two very very conflicting mechanisms at leats: Pollution from CO2 and methane, acting to raise temperatures, and pollutin from particulate emissions like carbon soot from diesel and coal, acting in reverse.
Leaving aside amplification, this would tend to make periods of rapid carbon based economic expansion, initially cooling in effect, as the greater amount of atmospheric soots is a net cooler: once into recession, these will wash out, leaving the longer term gaseous pollutants well able to increase temperatures rapidly. So watch out for a sharp upswing as the global recession bites deeper.
With current La Nina type conditions likely to come to an end, that should be a triple whammy.
The message from Terry Fields contains these words:
The Met Office's 21 point binominal filter except that in ms works case the total weighting only adds up to 0.99999893 and trying to encapsulate the whole function in one cell gave the wrong answers.
Thanks
The message from "Bob Mannix" contains these words:
Done to see how Excels polynomials coped with a sharp change in direction - they don't.
HomeOwnersHub website is not affiliated with any of the manufacturers or service providers discussed here. All logos and trade names are the property of their respective owners.