A recent article by Lebatard and many co-authors in the Proceedings of the National Academy of Sciences shows why correct error propagation is important. In this article, incorrect error propagation leads to a wrong conclusion. Correcting it largely invalidates the point of the paper. The article is open access and is here.

What these authors are trying to do is date the occurrence of the early human ancestor Sahelanthropus tchadensis in a lake sediment section in Chad. They use the Be-10/Be-9 ratio of the lake sediments to accomplish this, on the basis that the 10/9 ratio of leachable Be in the lake and surface sediments is constant through time. Thus, the  present 10/9 ratio of lake sediments preserved in a stratigraphic section is related to their age by:

$t = \frac{-1}{\lambda}ln{\left( \frac{R}{R_{0}} \right)}$

Where $R_{0}$ is the 10/9 ratio at the time of sediment deposition (assumed to be constant through time), $R$ is the measured 10/9 ratio in the sediments of unknown age, $\lambda$ is the decay constant for Be-10 (4.99e-7 /yr) and $t$ is the age of the sediment.

This is straightforward except that there is no way to be sure from first principles that the key assumption — constant depositional 10/9 ratio over time — is true. Be-10 is supplied from fallout of cosmic-ray produced Be-10 in the atmosphere, which ought to be more or less steady. Be-9, on the other hand, comes from dissolution of Be-bearing minerals somewhere in the lake basin, which might not be steady. Thus, the only way to know whether this key assumption is true is to look at the change in the 10/9 ratio over time: if we see i) a smooth, steady decrease in 10/9 ratio with stratigraphic age, and ii) the same 10/9 ratio in stratigraphically closely spaced samples that should share the same age, then we might reasonably conclude that the depositional 10/9 ratio is more or less constant. The authors of this paper follow this sort of reasoning, as follows. First, they observe that ages from sets of samples from the same stratigraphic unit — which should be the same within measurement error — show values of the MSWD statistic that are near 1. MSWD near 1 indicates that the scatter in data is commensurate with the uncertainties in the data, i.e. no excess scatter is present. Second, they observe that average Be-10/Be-9 ages from certain stratigraphic intervals are broadly in agreement with biostratigraphic age constraints.  These two observations lead them to conclude that the 10/9 ratio in lake sediments is constant through time.

The problem with this line of reasoning is that they have incorrectly calculated MSWD values, because of incorrect error propagation. This is clear from two observations. First, the stated relative uncertainties in the ages are  greater than the relative uncertainties in the ratio measurements. For example, a sample at 6.1 meters in their section TM254 (readers who care may want to look at their Table 2 at this point) has an 8% measurement uncertainty on the ratio and an apparent age of 6.5 Ma. The reported uncertainty on the age is 0.75 Ma, a 10% uncertainty. This can’t be correct. Think about it — 7.2 Ma is five half-lives of Be-10. A 50% uncertainty on the ratio measurement would mean an uncertainty of one half-life, or 1.4 Ma. If the age is five half-lives, this is only 20% of the total age. So a 50% uncertainty in the ratio measurement becomes only a 20% uncertainty in the age estimate. The relationship is not quite linear, but this means that an 8% error in the ratio at this age should become something like a 4% error in the age. This is a general property of age uncertainties in radioactive decay systems: as age increases, the relative uncertainty on the age becomes much smaller than the relative uncertainty on the amount of parent remaining. Thus, the fact that these authors report relative age uncertainties that are larger than relative ratio measurement uncertainties indicates that something is wrong.

The other observation that clearly indicates that something is wrong comes from calculating the MSWD on the measured ratios, instead of the ages, for the sets of samples from particular stratigraphic levels that the authors have averaged. For example, six samples from 7.3-8.5 m in the TM254 section (Table 2 again) have 10/9 ratios that vary by a factor of 5 and have measurement uncertainties of 9-16 %. These measurements clearly do not belong to a single population and have a MSWD of 49.8. However, when the authors transform these ratios to ages, somehow the ages from the same samples have a MSWD of 1.1. If the ratios don’t belong to a single population, then clearly the ages derived from those ratios can’t belong to a single population either. Something is seriously wrong here.

An additional notable observation is that some of the MSWD values reported by the authors (0.10-0.28) are wildly improbable. This suggests overestimation of uncertainties.

So what happens if we do the error propagation correctly? Here is how to do the error propagation. Uncertainties in the ages come from three sources: i) uncertainty in the estimate for the depositional 10/9 ratio ($R_{0}$), ii) uncertainty in the Be-10 decay constant, and iii) measurement uncertainties in the observed 10/9 ratio ($R$). In computing uncertainties on ages for a MSWD calculation, we should only consider iii), the measurement error in the sample 10/9 ratio. This is because we are comparing different samples to each other, so must only consider errors that are independent between samples. The uncertainties in the decay constant and the initial ratio are common to all samples, so do not enter into a MSWD calculation. Using normal linear error propagation, the uncertainty in the age $\sigma t$ that should be used in calculating the MSWD is:

$\sigma t=\sqrt{\left( \frac{\partial t}{\partial R}\sigma R \right)^{2}}$

where

$\frac{\partial t}{\partial R} = \frac{-1}{\lambda R}$

and $\sigma R$ is the uncertainty in the measured 10/9 ratio. The following table shows stratigraphic heights, measured ratios, ages, uncertainties reported by the authors, and actual uncertainties for the six samples in section TM254 discussed above.

 Stratigraphic ht (m) 10/9 ratio (x 10^-10) Apparent age (Ma) Reported age uncertainty Correct age uncertainty 8.5 16.36 +/- 2.60 5.39 0.92 0.31 8.4 7.68 +/- 0.76 6.87 0.80 0.19 8.4 3.07 +/- 0.34 8.67 1.11 0.22 8.1 9.04 +/- 1.13 6.55 0.91 0.25 7.9 6.89 +/- 0.65 7.08 0.80 0.18 7.3 6.20 +/- 0.83 7.39 1.08 0.26

For these six samples, again, the authors computed a MSWD for the ages of 1.14 using the incorrect age uncertainties. Based on this value, they concluded that the ages belonged to a single population and they could properly average them to obtain a summary age and standard error for this part of the stratigraphic section. This is incorrect. The actual MSWD of these ages is 18, clearly showing that the apparent ages do not belong to a single population, as we expect from the fact that the 10/9 ratios clearly do not belong to a single population.

A plot of ratios and ages from this section shows this situation clearly:

The plot on the left shows that 10/9 ratios, as expected from the general concept of the method, do generally decrease with stratigraphic depth. However, they are widely scattered around this general trend by an amount well in excess of measurement uncertainty. More about this later. The plot in the center shows apparent ages with the (incorrect) uncertainties reported by the authors. It is clear from this plot why getting the error propagation wrong leads to a misleading conclusion: the large errors here give the impression that the data are scattered around a smooth increase in age with depth, by an amount that is commensurate with the measurement error. The third plot shows the same apparent ages with the correct uncertainties. It is clear that although ages do generally increase with depth, ages from the same stratigraphic level, like ratios from the same stratigraphic level, disagree by amounts well in excess of measurement uncertainty. Note again that the fact that we have not included uncertainties in the initial ratio and the decay constant does not change these conclusions: errors in these parameters would shift the entire array of ages without changing the relationship between them.

To summarize, one of the key observations that the authors cite in support of their claim that the depositional 10/9 ratio is constant through time is that ages from closely spaced stratigraphic levels agree within uncertainty. Doing the error propagation correctly shows that, in fact, this is not the case. In fact, the spread of 10/9 ratios from closely spaced levels is well in excess of measurement uncertainty. This shows fairly clearly that, in fact, the 10/9 ratio was not constant over time. It most likely stayed within a certain range — as shown by the overall trend of decreasing ratio with stratigraphic depth — but varied by as much as a factor of 5 over short time intervals.

Why this variation? Remember Be-9 is delivered to the lake by dissolution of Be-bearing minerals in the watershed. It seems certain that the rate of Be-9 supply is affected by hydrologic changes, and the fact that the sediments in question show a fluctuating lake level indicates that there were hydrologic changes. Thus, it seems very likely that orbital-scale hydrologic changes affected Be-9 delivery to the lake, and thus the 10/9 ratio of leachable Be in the lake system, on relatively short time scales. In any case, the data in this paper pretty clearly show that the assumption of constant 10/9 ratio that is necessary to apply this dating method, is false at short time scales.

Is the entire dating exercise wrong then? Probably not. Clearly the assumption of strictly constant depositional 10/9 ratio is wrong. However, ratios do clearly decrease with age, showing that changes in the depositional ratio most likely took place on a shorter time scale than is represented by the entire section, and thus that the ratio probably stayed within some bounds. So the 10/9 ratios do give us some age information. The important conclusion is that the true uncertainty in the actual age of the samples that the authors dated is much bigger than the measurement uncertainty. If the initial ratio is only known to a factor of 4, then the age can only be known with a precision of two half-lives of Be-10, that is, 2.8 Ma. So the ages reported in this paper are most likely within two million years of the true age of the true age of the sediments, and fossils, in question. The important conclusion is that the precision of the Be-10/Be-9 method is much poorer than proposed by the authors. The authors call attention to the fact that the ages agree with biostratigraphic age constraints (which also have a precision of 1-2 Ma) and suggest that the 10/9 ages are more accurate than the biostratigraphic ages. In fact this is not the case; the precision of the two methods is similar.

Could this be fixed? Yes. The key is to know the amount and the time scale of changes in the depositional 10/9 ratio. These could easily be obtained by high-resolution sampling at the presumably orbital time scales that have the largest effect on the lake basin hydrology. Once known, this information could be used to find out what amount of time averaging would be needed to ensure that the constant-initial-ratio assumption was true.

Is Sahelanthropus tchadensis actually 6.8-7.2 Ma? Perhaps. Nothing in this paper disproves that hypothesis. However, it does not prove the hypothesis either. Likewise, the hypothesis that Australopithecus bahrelghazali at this site is contemporaneous with the extremely well dated (by Ar-Ar) Lucy skeleton in Ethiopia is neither supported nor refuted by the Be-10/Be-9 results.