Design and analysis of experiments: a tentative timeline

One can distinguish three broad periods of innovation directly associated with application

David Cox, 1984 [1]

Causal confession

One sometimes reads the claim that statistics never had anything to say about causation. I must confess that early in my career as a statistician I said as much myself. I can remember exactly when I changed my mind. I was attending the 1980 RSS conference on multivariate graphical methods at Sheffield. A senior French statistician had been lecturing about causal methods (a lot of matrix algebra was involved) and in the bus to an outing to do some hiking on Lose Hill, Mam Tor and Rushup Edge I said to the statistician sitting next to me, 'I thought statistics didn't do causality'. He replied, 'what about design of experiments then?'. I immediately realised how stupid my remark had been.

In my defence, I had studied Economics and Statistics (1971-1974, Exeter) as an undergraduate - a little but not much on experiments there. In my first job (1975-1978) I had been a statistician in the health services supporting management: the analysis of routine data supplemented by some surveys is what we did. In my subsequent job as a lecturer, which I had begun less than two years before the Sheffield conference, I taught statistics on a variety of courses but had not had to teach experimental design.

So my thinking changed at the start of the 1980s. By the end of the decade I was well into my job as a statistician in the pharmaceutical industry. Causation was all we thought about and experiments were mainly the way that we examined it. I became extremely interested in the design of experiments and its relationship to analysis and in retrospect I am amazed and ashamed that it took me so long. However, if I could make a mistake of overlooking experimental design 5 years into working as a statistician, why should I be surprised that non-statisticians would overlook it?

Hence, the justification for this post. I thought I would set the ball rolling for a general discussion regarding milestones in the statistical theory of the design of experiments. I am bound to have missed many important things and I have deliberately kept the list to the first 2/3 of the 20th century. I have also largely kept to events, books or developments that were fairly directly linked to experimental design. I have thus excluded, for example, Neyman and Pearson's work even though the concept of power they introduced has been much used (and abused) in the design of experiments. I have also stretched the field of experimental design to include quality control, since assigning causes is central in deciding when and how to intervene. Student's 1908 paper is justified since the motive was experimental and the first illustration of it was a clinical trial.

Some milestones

Figure1: A tentative and very personal timeline of design of experiments

Figure 1 above gives some milestones of experimental design as they appeared to me. This, is of course, a very personal choice and at least one of the threads or themes, that of the Rothamsted school (see The Rule of Three), is one to which I have at least a treble personal connection, since I lived in Harpenden for 8 years, as did Fisher, and have been secretary of the Fisher memorial trust for over a dozen years and became good friends with John Nelder during my time in Harpenden. I shall briefly discuss some of these themes but I shall not discuss all the figures in the timeline.

The division into agricultural, industrial and clinical research that I shall use follows the division suggested by David Cox in his review[1] of experimental design for the Royal Statistical Society's 150th anniversary.

Agriculture, The Rothamsted School and Others

RA Fisher joined Rothamsted in 1919 and immediately made a mark, publishing several contribution to statistics and genetics, during his time there. Analysis of variance, randomisation, blocking and replication were developed by him (and, of course, many other statistical ideas not linked directly to experimental design). In particular, the idea that experimental design, as regards both efficient estimation of effects and valid estimation of uncertainty of estimates, was a topic in its own right is due to him.

Although Frank Yates and Fisher only overlapped by two years, by the time that Fisher had left Rothamsted for University College London, Yates had already made his mark on experimental design in particular as regards factorial experiments and incomplete block designs and he succeeded Fisher as head of statistics. Other important figures who spent time at Rothamsted included David Finney, William Cochran and Oscar Kempthorne.

When Yates retired he was succeeded as head of statistics by John Nelder, who had already made important contributions developing a formal algebra of analysis and who was to push forward the development of his ideas as part of the computer package Genstat(R).

Industrial Experiments, Variation and Quality

Dealing with sources of variation was key in what the Rothamsted school achieved but they were mainly concerned with optimising expected yield. Although this can be a concern of manufacturing, typically variation can be a problem in its own right, since parts produced by one process need to work with parts produced by another. For this and other reasons, reducing variation becomes a key goal of improving quality.

Although much of the work in industrial design involved applying and extending some of the design ideas developed at Rothamsted, for example fractional factorial designs and alternatives to them, mastering variation became a key concern in quality control, with influential figures such as Deming and Taguchi working on it. However, this particular strand can be regarded as having its initial development independently of Rothamsted, since Shewhart, a pioneer in using control charts, started at Western Electric in 1918, the year before Fisher started at Rothamsted.

Clinical Trials, The Vagaries of Human Subjects

Experimentation in medicine has a long history but the dawn of the modern era is generally agreed to be the Medical Research Council trial of streptomycin in tuberculosis (published 1948), in which Bradford Hill introduced randomisation as a means of allocating patients to treatment, even though, as he later explained[2], he avoided using the word in order not to confuse or deter the physicians.

Patients are independent agents and trialists commonly find that they do not do as they (the trialists) would like. Independence may lead to noncompliance and missing data are common for this and other reasons. Also, of course, for trials of treatments for life-threatening diseases, unless such trials are run with very long follow-up, some patients will still be alive at the end of the trial. Thus, censored data are also an issue. Both missing data and censored data are primarily a challenge for analysis but, as Fisher himself taught us, issues for analysis are ultimately issues for design and vice versa.

Conclusions

I shall conclude with a table that gives my summary of how David Cox[1] characterised research in these three fields. I have rather presumptuously added a few further themes in a final column. It goes without saying, that his original article is well worth reading!

References

1. Cox DR. Present Position and Potential Developments: Some Personal Views: Design of Experiments and RegressionPresent Position and Potential Developments: Some Personal Views: Design of Experiments and Regression. Journal of the Royal Statistical Society, Series A. 1984;147(2):306-315. 

2. Hill AB. Suspended judgment. Memories of the British Streptomycin Trial in Tuberculosis. The first randomized clinical trial. Controlled clinical trials. Apr 1990;11(2):77-9. doi:10.1016/0197-2456(90)90001-i

Acknowledgements

I am grateful to Val Fedorov, Andy Grieve and Ron Kenett for helpful discussions and to Paul Gentry and Peter Diggle for helpful information on the Sheffield Conference. I also absolve all of them of all responsibility for any nonsense in this blog!