Be honest: the VW diesel-gate scandal was not a total surprise.
And the reason it didn’t surprise you, the reason you felt you already knew it, is because it is a specific example Goodhart’s Law.
In rough terms Goodhart’s Law states that:
If you announce a specified parameter is an organisational target, then people will ‘game’ the system until that parameter becomes meaningless.
Over at ‘Less Wrong‘ they describe the process in detail.
- ‘Managers’ want to achieve a difficult-to-specify goal G.
- They formulate G* which is not G, but until now in usual practice, G and G* are correlated.
- Subordinates are given the target G* and most people try to achieve G*. As time goes on, every means of achieving G* is sought.
- G* was formulated because it is simple and more explicit than G. Hence, the persons, processes and organizations which aim at maximising G* achieve competitive advantage over those trying to juggle both G* and G.
- Eventually the correlation between G and G* reduces and after a point, the correlation completely breaks down.
There are many examples that I could use to illustrate the point e.g. NHS target-culture, but let’s look specifically at education and emissions to see how Goodhart’s law describes the corruption of initially noble aims.
We consider that a worthy societal goal (G) is a well-educated population. But for reasons of cost and operational simplicity we substitute this with a correlated goal (G*), that students pass exams.
Initially teachers and schools try to bear in mind the overall goal (G), but after a while – and we are well into that ‘while’ – anyone attempting to achieve goal G in place of its proxy G* – passing exams – is seen as an organisational liability.
Eventually publishers get in on the act, buying up exam boards and lowering exam standards, maximising G*, but reducing progress towards the overall goal G, and minimising the correlation between G and G*.
Does that sound familiar?
Governments have a noble goal (G) of reducing air pollution and so demand that automobile manufacturers make cars which emit less harmful chemicals.
To be fair to all manufacturers, they specify the precise conditions under which tests are conducted and demand the results from this test (G*) meet specified standards.
The tests are designed so their results (G*) are correlated with real world performance. However the tests are necessarily simpler than reality – and can’t be overly complex otherwise the cost of measurement would be too high.
Inevitably, manufacturers make sure there cars perform well in the precise conditions of the test and tune their engines accordingly. Why would they not?
US regulations are reasonably well-devised and what VW did was (probably) illegal.
But what you have not heard is the other manufacturers stating that ‘We don’t do this’.
Why are they silent?
Managers and governments face complex, multi-dimensional problems. And I sympathise. But I do not sympathise with a response to this complexity which insists on a one-dimensional focus which ‘flattens’ reality.
The people on the ground – the teachers or engineers – still face the actual complexity, but their managers deny the existence of the other dimensions of the problem. Typically they use ‘metrics’ which are supposed to be an aid to judgement, as a substitute for judgement.
This leads to workplace stress, a ‘blame’ culture and – ultimately – failure to achieve the original organisational goals.
Goodhart’s law is widely applicable, but on reflection I think it may be merely a specific implementation of the more widely-known aphorism:
The road to Hell, is paved with good intentions.
Thanks to Andrea Sella for the original tip off.