Calculating your carbon footprint - a load of bollocks

[The Natural Philosopher]

However if the climate system were *that* chaotic we'd be growing bananas one winter, and polar bears the next.

[Terry Fields]

Well, this whole exercise has been voyage of discovery - not the least because the basic data itself appears capable of different manipulations (such three or four groups analysing the satellite data, for example) even before moving on to whether the data supports one side of the debate or the other.

It seems like people may well be arguing over the difference between two wrong numbers!

The Jan 2009 data is going to be interesting....but whether it will settle any differences is probably a forlorn hope.

I'm sure we'll be back to this before 2012.

[Roger]

The message from Terry Fields contains these words:

Definitely (a forlorn hope). :-)

As long as there are people on both sides, or even just on one side, happy to cherry pick the data they use to prop up their position there will be disputes.

Most probably.

I am tempted to write to the Met Office about the bizarre way they terminate their smoothing filter. Doubt if that will even provoke a response let alone nudge them onto something a bit less extreme.

[The Natural Philosopher]

Smoothing filters are remarkably difficult to implement on data that comes to a sudden stop..

I've had occasion to implement them, sometimes.

It's amazing how, if what you are trying to do, is produce a sine wave, almost any random collection of points can be smoothed into one.

[Rod]

I have many times emailed site owners about lack of information, incorrectnesses, etc. The responses have varied enormously. At one end of the scale, thanks for pointing out the problem and near-immediate action to correct it. At the other, nothing, not even a reply, and no change.

However, I do believe that it helps everyone when such questions are asked, errors pointed out, etc. in the spirit of trying to get things right.

[Terry Fields]

I'm sure you'll get a reply, but the chances are it will be some sort of fob-off.

But why have a smoothing filter at all? Why not perform some sort of regression or other statistical analysis on the data set? There must be many such tools available. I need to monitor my blood pressure on a daily basis, and to determine lomg-term trends I use the regressions available in Excel to find the regression-line that results in the least value of the variance - at least that has some authority to it. A smoothing-filter exercise wouldn't even tell me the rate at which my BP is changing. The MetO seems to be a little disingenuous here.

[stuart noble]

Interesting. Care to post the formulae for that? I've been logging my readings over the last couple of months, and all I can deduce is that they're good in the mornings and not so good in the evenings. The rest is truly random it seems.

[Roger]

The message from Terry Fields contains these words:

The smoothing filter seems to be there to iron out the large year on year variations and give a better idea of the trend at a glance. It uses a binominal distribution over 11 years with the central year getting a weight of only 0.176197 and the adjacent years weighted at 0.160179. The ends have so little weight I am surprised they bother with years 1 - 3 and 19 - 21. On historic data it would appear that it does a pretty good job but IIUC at the end of the sequence when the 5 forward years are not available they simply project the final value forward for 5 more years. The previous year keeps its weight of 0.160179 but the end year then gets a weight of approximately 0.588* and the 3 previous points on the smoothed graph are significantly influenced in the same way which will give totally the wrong impression if the latest figure is on the margins for any reason.

*Doing that to the 1998 high would have had some of the watchers losing control of their bowels.

[Terry Fields]

Use statistics to get a grip on the figures; there lots of functions available in Excel., although they might take a bit of finding.

I use a number of X-Y charts:

Get your data into columns (I didn't know this function was available at the time, but put time and date into one column - 'format','cells'. time and date, 'special', and pick a style you like) with Systolic and Diastolic in separate columns alongside, then select all three columns.

Insert > Chart > X-Y scatter > Type: Scatter and follow the instructions. The chart takes a lot of setting up once it's generated, too long for me go go into here, but it's worth the exercise.

You can add a trendline of several sorts: linear regression, polynomial, etc. and you can change these at any time. Get the chart to show the regression equation and variance. Change between them to fins the least variance = best fit of data. There is a fair range of statistical functions in Excel

I've done this for a number of items: BP versus date I changed medication, BP as a function of time of day, pulse rate ditto, Systolic vs diastolic etc etc; its easy once you've tabulated the data and mastered how to set up a chart.

My BP has a smooth 'hump' as a function of time of day, peaking at about mid-day, about ten points above the early-morning and late-evening readings, but due to the scatter it would not otherwise be obvious. Mean readings are 138 sd 9 systolic, 87 sd 7 diastolic.

My BP is falling at about 1mmHg every 8 days, again not obvious but calculated form the regression equation slope of about - .12,

Have a go if you're interested, it's well worth going up the learning curve, even if it takes a little time. Currently I'm bamboozling my doctor with the graphs and statistics, as I don't want to take yet another pill....the last one didn't do me any good at all, and he wanted to put me on a combination that is currently not advised as it is implicated in the formation of diabetes. I said 'no'....

[Terry Fields]

To my mind this is the wrong approach - if you have data, then put the statistical tools to work on it. Smoothing things out so you can see what might seem to be a trend is IMVHO a very poor substitute for getting the figures via an analysis - they'll tell you which way things are going, and a whole lot more bedsides.

[The Natural Philosopher]

What is a smoothing tool if not an example of that?

[stuart noble]

Thanks. Yes, I really must get into Excel charts, but I have an aversion to pie charts and graphs. What puts me off all this monitoring is the extent to which I know I can influence the readings by slightly varying my everyday physical activities in the previous couple of hours. Simple things like taking a shower etc.

[tony sayer]

Which one was that then?..

[Terry Fields]

Amlodipine was the one that caused problems - and it wasn't the usual swollen ankles thing.

The suggested combination was beta-blockers/thiazide diuretics, which I'd been taken off a couple of years ago due to the diabetes risk.

HTH

[Terry Fields]

It's the difference bewteen 'smoothing' the data, which doesn't seem to do anything but tidy up the appearance, and statistics that tell you everything about the data - IOW, chalk and cheese.

[Terry Fields]

You might be interested in the following exercise, where I took my BP data and treated it to a) a 10-point smoothing exercise, and b) a linear regression, which had a lower variance than a second-order polynomial (not shown)(all data manipulation courtesy of Excel spreadsheet).

Basic data, no form of data reduction:

moving average:

regression:

doc looks at the first graph, and says "you've got a problem".

The MetO looks at the second graph, and says "your BP is going up"

I look at the third, and note that my BP is falling at the rate of 1 mm Hg per seven days (from the slope of the systolic regression line), and I say "if I keep this up, I won't need any extra medication, and my BP will reach 120 systolic after 178 days, or 80 diastolic after

134 days". I'm already 73 days into the trial.

To my mind, there's no comparison between smoothing (which gives me no figures and is misleading) and a regression exercise (from which I can calculate all sorts of data). I'll go for the latter every time.

[For Stuart Noble's interest, here's my BP as a function of time of day, complete with second-order polynomial regression - gives least value of variance:]

[Terry Fields]

You might like to see my post of a few minutes ago to Roger, where I illustrate several ways of manipulation of my BP data. For your interest I've included a BP vs Time of Day graph.

[tony sayer]

I

Interesting I note the reverse of that High in the morning low around midday and raising at night!..

[The Natural Philosopher]

Oh dear oh dear.

I see..

[Terry Fields]

Ah....but is that by using a second-order polynomial regression ;-)