Archive for October, 2015

My Weather Station: WOW!

October 19, 2015
My weather station sends its readings to the Met Offices Weather Observations Website every 10 minutes.

My weather station sends its readings to the Met Offices Weather Observations Website every 10 minutes. Click for larger picture.

This summer I bought an automatic weather station. This article is about which one I chose, how I set it up, and how I got my data off the weather station and onto the web!

If this kind of arcane detail doesn’t interest you, then walk away now.

In a future article I will look at the data and recount some the fascinating tales that it tells, but this article is just about the experience of setting up the station in the hope that it helps someone out there who wants to do the same.

Just so you can see what one gets in the end, here are three ways to view the live data from my weather station:

  • On my personal weather website!
  • On the Met Office’s Weather Observations Website (WOW). Type in ‘Teddington’ and my station is pretty much in the middle. Click on the symbol and you can view details about the site.
  • On the Weather Underground’s Wundermap: Type in ‘Teddington’ and my station is the one on Railway Road (ID is ITEDDING4)

So how does it all work?

Weather Station

Choosing a weather station is tricky because there are many to choose from – but none of them give any guarantee that they will allow uploads to web sites.

Having recently visited the Meteorological World Expo in Brussels, I am very aware that a professional grade weather station is likely to cost at least £1000.

However I wasn’t prepared to spend anywhere near that. So after hours of reading reviews I picked a £90 model from Amazon: (link here).

I picked it because:

And so it arrived


Although all the parts were present, and I enjoy this kind of assembly, putting all the parts together was tricky.

The main reason was that although the picture on the front shows all the components at rather jaunty angles, in fact there is not much choice about the assembly!

  • The tiny solar panel which sits on top of the thermometer screen has to face south – otherwise it won’t charge optimally.
  • The alignment of the wind vane also has to be aligned with a particular notch facing North.
  • After that there is little choice about position of the rain gauge.

So assembly took a couple of hours, and then I popped in the batteries. The weather station took two re-chargeable batteries – which were supplied – and indoor readout unit took two AA batteries. And immediately the weather station began to work, linking easily to the indoor monitoring unit.

I left it working this way for a week or so – just placed awkwardly on a table in the garden. This was when I noticed that if the solar panel was not aligned properly, the weather station would not charge and the station would stop working.


So now I had a working weather station. What did it consist of?

Photo of my weather station with the various parts labelled

Photo of my weather station with the various parts labelled. See text for details.

  • First was a wind vane. This outputs the direction of the wind, with 0 degrees indicating wind from the north and 180 degrees indicating wind from the south.
  • Second was an 3-cup anemometer to measure wind speed.
  • Third was a combined thermometer and hygrometer – or humidity meter. These were inside a plastic unit about the size of a telephone handset which was placed inside a ‘screen’ designed to shield the unit from direct solar heating.
  • Fourth is a  barometer – or pressure meter. This is located in the indoor unit not the outdoor unit and should (in principle) be corrected for the difference in height between the two units
  • Finally there was tipping-bucket rain gauge. This ingenious see-saw like device (invented by Sir Christopher Wren and Robert Hooke) records the amount and rate of rainfall. As a small bucket fills it imbalances and when it tips a single ‘tipping event’ is recorded. Cleverly, the tipping event engages a second bucket which catches the stream of rain without a gap.

Hidden in the thermometer-hygrometer unit are two rechargeable batteries linked to a small solar panel positioned on top of the thermometer.

Together the solar panel and the batteries supply power to read the sensors and operate a radio link to a battery-powered readout unit which is kept indoors.

I consider that good value – especially since it worked straight out of the box!

Understandably the quality was not everything it could be. Most significantly, the unit was manufactured in grey plastic not white! For the thermometer screen in particular, this is a poor choice of colour.

Siting and exposure: where to put it?

Traditionally weather stations are placed inside ‘Stevenson’ screens – white, louvred,  enclosures designed to protect the thermometer from direct sunlight and rain. The aim is that as far as possible, the thermometer should measure the temperature of the air.

Additionally weather stations are usually positioned in open areas away from trees and buildings. Obviously, that wasn’t going to be possible in my garden.

Instead I chose to mount the device above the back wall of our garden. I chose this because I thought it would be better than in the garden – where the walls would create a sheltered environment. In short, it is poor siting, but the best I could manage.

The view from the South looking North showing weather station mounted on the back wall of our garden.

The view from the South looking North showing weather station mounted on the back wall of our garden.

I bought a television antenna mounting kit, and firmly attached an antenna mounting pole to the wall. Then I attached  the weather station to this pole using the fittings supplied with the kit.

It is hard to overstate the importance of the siting and exposure of the weather station.

For example, the temperature of the air can easily change by several degrees on raising the station just a few metres above the ground. Similarly, the humidity and wind speed also vary strongly with height.

However I am not trying to obtain metrology grade data, but merely to record what the conditions are in this typical suburban location

Linking to the web

One of the main requirements for a valuable weather station is that it should operate around the clock. This is not easy to achieve, and here I describe what I have  done so far.

The outdoor components have so far (it is October 11 2015 as I write) managed to charge enough during the day to keep the unit operating all night. I will test this through the winter which has minimal sunlight and during which the cold may affect the operation of the batteries. If it fails I may need to run a power supply across the garden.

The indoor components are located in our converted loft – out of the way of our day-to-day life. 

The indoor unit is battery powered but I have just a bought a mains replacement unit and now I won’t need to worry about the batteries running out.

The indoor unit is linked via a (supplied) USB cable to a Windows XP laptop dating from 2005. The screen is failing and there are many lines of blank pixels, but the computer is adequate to run the Cumulus Software.

Schematic diagram of the way the weather station links to a laptop , and via that, to the web.

Schematic diagram of the way the weather station links to a laptop , and via that, to the web.

There have been two interruptions to data streamed to the web. The first arose from the old ‘Windows Update’ dialogue box which locked the computer until a response was made. I have now fully disabled this. The second occurred when – for no reason that I could determine – the computer lost its WiFi link to the internet. Moving the computer  few centimetres restored the link. In short, keeping the system working 24/7/365 is non-trivial but I am aiming for greater than 99% availability – i.e. less than three-and-a-half days lost each year. Helpfully, the Weather Underground sent me an e-mail after the system was unavailable for longer than two hours.

The Cumulus software includes options to publish data to a website, to link to WOW and to Weather Underground and to other sites that I have not chosen to link to. The software is ‘donation ware’, but it is so good that I gave £20 to its author.

Calibration? I’ve heard of it.

One question that will have occurred to many readers is whether the data is reliable? Good Question

I have deliberately avoided calibrating the unit because I wanted to see how this unit performed ‘out of the box’.

One interesting check of the data is to compare it with neighbouring weather stations at night under a cloudy sky when the temperature is rather uniform.

As the screenshot from the Met Office’s WOW site shows, the unit reports similar temperatures to its neighbours. When my station read 12.7 °C, its three nearest neighbours – each about 1 km away in three different directions – read 12.3 °C,12.1 °C,12.3 °C. I consider this level of agreement surprisingly good, indicative that the station is probably not reading too incorrectly.

WOW Screenshot

Screenshot from the Met Offices WOW website which allows me to compare my readings with those from neighbouring sites. Click for larger image.


So far this has been fun, but the real fun is about to begin.

As the station builds up a record of the data for the year, I get quantitative insights into the weather world around me, that helps me to make sense of what I experience. Gazing at the single full month of data I can already see fascinating stories developing. But I will write more about the data next week.

Counting Sunspots: How hard can it be?

October 12, 2015
NASA Image on Sunspots in 2001. How many sunspots can you see?

NASA Image of sunspots in 2001. How many sunspots can you see?

Warning: Never look directly at the Sun

The Sun is ever present – but rarely seen because we – very sensibly – never look at it directly.

And because of this, most people have never seen a sunspot – a blemish on the observable disc of the Sun.

However, since approximately 1610, people have made careful observations of the Sun’s observable disc, and counted the number of Sunspots. At least one observation has been made on almost every single day for the last 400 years.

People noticed that sunspots were formed near the poles of Sun and then, over the course of a few days or weeks, they travelled towards the Solar equator, where they faded away.

Over many years it became apparent that the number of Sunspots varied periodically. The periodicity ranged between 8 and 14 Earth years, but on average was approximately, 11 years in length.

The average number of sunspots visible on the Sun each year since 1700.

The average number of sunspots visible on the Sun each year since 1700. (Click for version: data location at end of this article)

In the modern scientific era, we have learned – from satellite measurements amongst other techniques – that the intensity of sunlight reaching the top of the Earth’s atmosphere (on average approximately 1366 watts per square metre) varies by up to 1.3 watts per square metre – or 0.1% of the total – with exactly the same cycle as the sunspot count.

This correlation allows us to infer the amount of energy emitted by the Sun back to 1610. And this is where things become interesting

While one can understand that measuring the radiant power of the Sun might be fraught with difficulty, not many people have considered the other end of the correlation: counting sunspots. After all: how hard can it be?

How hard can it be?

Reading an 80 page report by an international team, the answer to this question is straightforward: counting sunspots is very hard.

I can’t summarise the entire paper – it is in itself abridged in many ways! But here are four things that have struck me since reading the report: The main thing I learned; a surprising response; a lesson; and a treat.

1. The main thing I learned

Looking at a close-up image of sunspots, it is kind of obvious, but nonetheless surprising, that the idea that there is a particular number of discrete sunspots is nonsense.

Early observers had poor observing facilities and could perhaps see discrete spots. But as observational techniques improved it became clear that that there is in fact no unique answer to the question ‘How many sunspots are there?”

The first people to collate sunspot data from different observatories, found that in addition to technological factors, there were ‘personal factors’ varying by up to 50%.

Additionally, it is clear that groups of sunspots are more significant than individual sunspots, and yet one cannot resolve the sunspots within a group. And so the number of groups is also counted.

And what we actually plot is a number called the sunspot index that collates the sunspot number (SN) data from different observatories, count’s the number of groups (GN) as being  as equivalent to 10 sunspots, and applies a personal factor for each observatory based on ‘How they do it there!’

In short, the Sunspot Index, is not ‘data’, but what is known in modern parlance as a ‘data product’.

The purpose of the report was to look back at the procedures applied to the raw data, and to make sure that the comparable procedures were applied fairly to data over hundreds of years.

The final result is a revised Sunspot Index that removes historical biases and more fairly reflects ‘what really happened’. The differences are not very great, but they subtly alter the conclusions that can be drawn from the data.

And because of the link between sunspot number and solar output, these conclusions have implications for our understanding of climate history in the last few hundred years.

2. A surprising response

One conclusion that can be drawn from the revised dataset is that there has been essentially no upward trend in sunspot numbers – and by inference solar output – over the last few hundred years.

So I was surprised that the uber-denialist site Watt’s up with that (WUWT) accepted the revision with barely a comment.

The reason for my surprise is that WUWT routinely rejects the analogous revision of historical temperature data known as homogenisation. This is truly bizarre, because the temperature dataset is vastly more complex and subject to many more systematic biases than the sunspot index.

3. A lesson

The sunspot index provides us with a insight into the activity of the Sun in exceptional detail. However, many times the collation of the data has come close to collapse as observatories asked ‘What’s the point of that?’.

As I wrote last week, at first anyone involved in this kind of ‘pointless’ research is tolerated. But eventually they are seen to be ‘not pulling their weight’ in whatever the latest fashionable institutional activity is. And eventually, they are closed down.

But somehow, the sunspot record survived to an era – hundreds of years after its first records – in which people managed to correlate it with more objective measures of solar activity. And finally its true value was revealed.

So the lesson is this: although scientific activities do need to change over time, it may be hundreds of years before the impact of continuing or ceasing some activity is discovered.

4. A treat

And finally a breathtaking close-up movie of some sunspots as they they traverse the disk of Sun


Sunspot data can be downloaded from the Sunspot Index and Long-term Solar Observations SILSO web site

Target Culture and The Road to Hell

October 5, 2015
Granting a person a tenured post - i.e. one from which they cannot easily be dismissed - is a big commitment for a University. So they have an easily-understandable goal (G) to appoint only people who will work actively after they have become tenured. To achieve this goal they use a proxy goal (G*) that insists that candidates must publish research papers.

A VW Golf TDI ‘clean’ diesel: the people who set emission standards hoped that they would make the world a better place; why didn’t that work?

Be honest: the VW diesel-gate scandal was not a total surprise.

And the reason it didn’t surprise you, the reason you felt you already knew it, is because it is a specific example Goodhart’s Law.

In rough terms Goodhart’s Law states that:

If you announce a specified parameter is an organisational target, then people will ‘game’ the system until that parameter becomes meaningless.

Over at ‘Less Wrong they describe the process in detail.

  • ‘Managers’ want to achieve a difficult-to-specify goal G.
  • They formulate G* which is not G, but until now in usual practice, G and G* are correlated.
  • Subordinates are given the target G* and most people try to achieve G*. As time goes on, every means of achieving G* is sought.
  • G* was formulated because it is simple and more explicit than G. Hence, the persons, processes and organizations which aim at maximising G* achieve competitive advantage over those trying to juggle both G* and G.
  • Eventually the correlation between G and G* reduces and after a point, the correlation completely breaks down.

There are many examples that I could use to illustrate the point e.g. NHS target-culture, but let’s look specifically at education and emissions to see how Goodhart’s law describes the corruption of initially noble aims.


We consider that a worthy societal goal (G) is a well-educated population. But for reasons of cost and operational simplicity we substitute this with a correlated goal (G*), that students pass exams.

Initially teachers and schools try to bear in mind the overall goal (G), but after a while – and we are well into that ‘while’ – anyone attempting to achieve goal G in place of its proxy G* – passing exams – is seen as an organisational liability.

Eventually publishers get in on the act, buying up exam boards and lowering exam standards, maximising G*, but reducing progress towards the overall goal G, and minimising the correlation between G and G*.

Does that sound familiar?


Governments have a noble goal (G) of reducing air pollution  and so demand that automobile manufacturers make cars which emit less harmful chemicals.

To be fair to all manufacturers, they specify the precise conditions under which tests are conducted and demand the results from this test (G*) meet specified standards.

The tests are designed so their results (G*) are correlated with real world performance. However the tests are necessarily simpler than reality – and can’t be overly complex otherwise the cost of measurement would be too high.

Inevitably, manufacturers make sure there cars perform well in the precise conditions of the test and tune their engines accordingly. Why would they not?

US regulations are reasonably well-devised and what VW did was (probably) illegal.

But what you have not heard is the other manufacturers stating that ‘We don’t do this’.

Why are they silent?

In general

Managers and governments face complex, multi-dimensional problems. And I sympathise. But I do not sympathise with a response to this complexity which insists on a one-dimensional focus which ‘flattens’ reality.

The people on the ground – the teachers or engineers – still face the actual complexity, but their managers deny the existence of the other dimensions of the problem. Typically they use ‘metrics’ which are supposed to be an aid to judgement, as a substitute for judgement.

This leads to workplace stress, a ‘blame’ culture and – ultimately – failure to achieve the original organisational goals.

Goodhart’s law is widely applicable, but on reflection I think it may be merely a specific implementation of the more widely-known aphorism:

The road to Hell, is paved with good intentions.

Thanks to Andrea Sella for the original tip off.


%d bloggers like this: