A bad month at the office…

My badge for the Fundamental Constants Meeting

My badge for the Fundamental Constants Meeting

An expert is a someone who has made all the mistakes which can be made, in a narrow field.

Niels Bohr

Some time ago  – together with colleagues at NPL and SUERC – I made a very accurate estimate of the Boltzmann constant.

The Boltzmann constant is the number that specifies how much energy particles have at a particular temperature. It provides a numerical link between thermal and mechanical energy.

The work took 6 years of my life, and possibly took six years off my life!

But at the start of February, at an international conference in Germany on the value of Fundamental Constants I had to admit that our estimate was wrong. And wrong by more than the margin of error that we had anticipated.

I have been feeling terrible about this all month.

I am aware that there are lots of reasons why I shouldn’t feel bad: For example:

  • We had in fact considered the possibility that this type of major error could occur. And we mentioned in our paper how to correct our estimate if it did occur.
  • And also the difference isn’t much in the grand scheme of things: our answer was wrong by 2.7 parts per million, which is  equivalent to estimating a distance of 1 kilometre incorrectly by 2.7 millimetres.
  • And also nobody will die as a result of the mistake.
  • And as it happens, it was revealed at this meeting that all the ‘best’ recent estimates of the Boltzmann constant suffered from a similar error – so it was not just me.
  • And our revised estimate is still the most accurate ever made in human history!

But nonetheless, I have felt absolutely terrible all month.

What went wrong?

[The next bit gets technical:sorry]

In the experiment we had to estimate the average kinetic energy of a molecule in argon gas at a known temperature.

To estimate the average kinetic energy of a molecule we needed to estimate the  average speed of the molecules of the gas, and their average mass.

We estimated the average speed of the molecules from measurements of the speed of sound in the gas. This part of the experiment worked very well.

Our mistake was with our estimate of the average mass.

Natural argon in the atmosphere consists of 3 different types of argon, called  isotopes. Most of the argon molecules weigh 40 times as much a hydrogen atom and so are referred to as argon-40.

But roughly 1 molecule in 300 is only 36 times as heavy as a hydrogen atom and so is referred to as argon-36.

And roughly 1 molecule in 1500 is 38 times as heavy as a hydrogen atom and so is referred to as argon-38.

A representation of the distribution of istopes in natural argon. For every 1500 molecules of argon-40 (green) there is on average 1 molecule of argon argon 38 (purple) and 5 molecules of argon 36 (black). We seem to have mis-estimate the exact ratio of argon 40 to argon 36 molecules in our sample.

A representation of the distribution of istopes in natural argon. Very roughly, for every 1500 molecules of argon-40 (green) there is on average 1 molecule of argon argon 38 (purple) and 5 molecules of argon 36 (black). We seem to have mis-estimate the exact ratio of argon 40 to argon 36 molecules in our sample.

Argon is captured from atmospheric air, purified and sold in pressurised cylinders. We  had previously shown that the amount of argon-36 and argon-38 varied from one cylinder to the next. So we needed to analyse gas from the actual cylinder we used.

Colleagues at SUERC compared the relative amounts of the different isotopes with the relative of amounts of those isotopes in atmospheric air.

And then we used a previous measurement of the relative amounts of the different isotopes in the air by a laboratory in Korea, KRISS, to work out how much of each isotope was in our samples.

And somewhere along that chain of measurements, there was an error.  This was finally revealed when we sent a sample of our gas directly to KRISS (something that wasn’t possible when we published otherwise we would have done it already!) .

We had estimated that the ratio of argon-40 molecules to argon 36 molecules was close to 298.9. In fact it now seems likely to have been closer to 296.9. So there was slightly more argon-36 than we thought in our experimental gas – and hence the gas was a little less dense than we thought.

Heigh Ho.

What will we do?

The first thing I will do  is to apologise to everyone I meet for having been so unjustifiably confident.

Then I will catch my breath, and remind myself of the words of Niels Bohr at head of this article: truly I am becoming an expert.

And then I hope to be able to persuade my colleagues to allow me to finish this measurement properly.

What we will do is to obtain some samples of gas each consisting of just a single type of argon isotope. These gases are very expensive which is partly the reason we didn’t try this in the first place.

We will then weigh these very carefully and mix them together in precisely known amounts to produce a sample of gas in which we know the relative amounts of the different isotopes

We will then ask our colleagues at SUERC to compare our experimental gas – we still have some gas from that bottle – against our isotopically-prepared sample of gas.

And then finally we will have an estimate for the average mass of a molecule of argon in our samples of gas.

And hopefully that answer will make sense!

8 Responses to “A bad month at the office…”

  1. H Stiles (@HStiles1) Says:

    I feel for you… imagine all the scientists through the last few hundred years though, who were just off or just pipped to the post by colleagues or ‘rivals’… the list is long! You did good work, so well done!

  2. Ed Hui Says:

    Any embarrassment is caused entirely by other people who don’t understand what science is supposed to be like. Science is supposed to be precisely like this- a really good scientist who, regardless of how the words come out, is actually delighted that the science has just been improved, and that the error is now corrected / correctable. The inconvenience of publishing erroneous results and the need to correct them afterwards is just administrivia. And the thermometer is still a great piece of hardware!

  3. ravisubbie Says:

    I agree with the previous comment. This blog post is a perfect example of good science. You did an experiment; you were clear about your method and the possible errors; you published what you found; and in the light of new information you have publicly corrected your error.

    Isn’t this exactly how science is meant to proceed. Rather than being embarrassed I think you should be proud of the honesty and dignified way you’ve dealt with this.

  4. ravisubbie Says:

    Reblogged this on More Known Than Proven and commented:
    Despite this science blogger’s protestations that he’s had a bad day at the office this is a brilliant example of how science should be done.

    This is worth a read as you don’t really need to understand any physics to understand the process of what happened here

    He did an experiment; was clear about his method and possible errors; published his work; and when he became aware of errors in his work he corrected them and republished his results.

    The world of politics could learn a lot from this approach!

  5. Nestor Patrikios Says:

    Well said Michael. This was hardly a schoolboy howler and the admission of a mistake just helps to set science apart from those who are not even wrong.

  6. Emma Woolliams Says:

    Oh, Michael … I wish I’d known. When I had EXACTLY the same experience (albeit with errors of 0.6% against an uncertainty of 0.1% , but as you and I know that is a detail), you gave me a very comforting hug. And it helped.

    I know what it feels like to stand up at an international meeting and say that I got it wrong. And I feel for you in empathy. By the way, does this result bring you closer to the result by the other lab?

  7. iamamro Says:

    It’s just as good knowing something isn’t correct and it is knowing that it’s correct. Don’t be downhearted. Many people will think back to this work with gratitude.

  8. Why I love weighing | Protons for Breakfast Blog Says:

    […] Making sense of science « A bad month at the office… […]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: