PRODUCING DATA

PRINCIPLES OF STUDY DESIGN

The Scientific Method

Principles of Scientific Experimentation

developed for agriculture, 1920's and 1930's by R.A. Fisher

adapted to other scientific fields e.g. medicine, 1950's by A. Bradford Hill

used to study industrial processes - recent e.g. Taguchi

Experimental design is one of the major contributions from Statistics to solving scientific problems.

Example - Salk polio vaccine trial

Salk polio vaccine trial (Freedman, Pisani and Purves, Chapter 1). Refer to Appendix Two for a reprint of the article by Meier, detailing the experiment.

vaccine developed by Salk, early 1950s

safe and effective in laboratory tests

how effective would it be in practice?

U.S. Public Health Service conducted an experiment.

Subjects were nearly 2 million schoolchildren in grades 1, 2 and 3 living in high risk areas.

Outline of experiment

• Why not vaccinate everybody?
  You couldn't tell if the vaccine was effective because you would have no basis for comparison (polio rates varied from year to year so comparison with the past is of no use)

  Why not use "refusals" as controls?

  High income parents were more likely to refuse, but risk of polio was thought to be higher in the higher income groups so the study would be biased in favour of the vaccine

  To avoid bias you need the treatment and control groups to be as similar as possible (except for the treatment)

  If the groups differ by some factor which can affect the outcome (or response) then the treatment effects may be confounded by this factor

  To ensure that treatment and control groups were comparable, children were allocated to these groups by randomisation (equivalent to tossing a coin), so that each child had an equal chance of being allocated to either group.

  To increase comparability the control group was also given a placebo injection. A placebo is a neutral substance which resembles the actual treatment. In this case the placebo was an injection of saline solution.

  Parents and children were not told to which group they had been assigned so they didn't know if the child was injected with vaccine or placebo (i.e. they were blinded). As a result there were refusals in the control group as well.

  Doctors diagnosing polio were not told whether the child had been given the vaccine or the placebo, so they too were blinded. Therefore the study was double-blinded.

  Groups were different sizes but the results could be made comparable by calculating the rate of polio cases per 100,000 children.

e.g. If 95 cases occurred in 250,000 children

= 38 per 100,000

Randomised, Controlled, Double-Blinded Trial

Conclusion: The vaccine was effective.

Summary of Study Design

Descriptive studies and observational studies report on how something is.

Experiments help to answer "Why?"

A design that incorporates randomisation yields data and analysis-results that are subject to the laws of probability.

Data produced from a good study design are used to draw conclusions about some wider population, a process called statistical inference. The reasoning of inference relies on the laws of probability.

For this reason, variability (or what would happen if the experiment were repeated many times) and probability theory (the mathematical study of randomness) are discussed in the next section of this book.