PRODUCING DATA

INTRODUCTION

Example - Public opinion polls

Polling opinion on the United Nations.

On September 19, 1983, during a dispute at the United Nations, a Soviet delegate questioned whether the role of host nation really suited the United States. The next day a national television news program invited its viewers to participate in a "phone-in" on the issue. A random telephone poll was conducted at the same time. The results of the two polls are shown below.

"Should the United Nations continue to be based in the United States?"

Bias

Which of the two samples more accurately reflects the opinion of the 150 million adults in the US.? The phone-in sample was biased in the following ways:

• selection bias - everyone invited to call in was watching that news program, which excluded all people watching other programs and all people not watching television at all
• response bias - the people who did respond were self-selected, they felt strongly enough about the issue to get on the telephone (and may have even called more than once).

Hence the phone-in sample may not represent the population as a whole in an accurate way, even though the sample size is large.

Randomisation

However, the 1200 people in the random telephone sample were not selected according to their television viewing habits or their eagerness to express themselves on the issue - they were chosen by random-digit dialling.

Randomising makes it possible to assess the accuracy of the poll in an objective and quantitative way. In fact, by using the principles of statistical inference that will be covered later in the course, we can be reasonably confident that the true percentage in the population who would have answered "yes" would lie between 69% and 75%.

Given that a random telephone survey is preferable to a phone-in, what potential problems exist in using random telephone surveys to obtain data?

Some points to consider would include -

• unlisted phone numbers (if using the telephone directory as a source of numbers)
• unanswered calls - are the people who are frequently not at home different in any important way?
• inaccurate and dishonest answers - a problem which lies outside the direct control of the statistician.

In summary, statistical analyses cannot be better than the quality of the data. It is important to have valid and reliable measurement procedures and survey designs based on randomisation.

Example - Teenage driving safety

Sometimes correct numbers are given incorrect interpretations, particularly when an apparent cause-and-effect relationship is caused by a "hidden or confounding variable".

A California study of teenage drivers showed that the accident rates for males and females were 0.162 and 0.075 respectively. That is, for the period of study,

Male teenagers have twice the accident rate that females do. Are male teenagers worse drivers?

A third variable, number of miles driven, was taken into account. The results are shown below.

An insurance company is interested only in line 1 of the table. Since males have more accidents and hence cost the company more in claims, they may charge males higher premiums. However, an employer hiring a teenager driver for a delivery route may review line 3 of the table and decide that a male or female driver would have the same likelihood of having an accident in the company car.

The relationship between the variables can be illustrated below:

In another similar example, a local council was concerned about an outbreak of gastroenteritis which seemed to be more likely to occur in people who had been swimming at the local beach. An investigation was undertaken to determine the cleanliness of the water. After some detailed investigation, however, the cause of the outbreak was traced to a mobile ice-cream van which visited the beach.

Hence, sometimes an apparent relationship between two variables may disappear when you control for a third variable and that, even if two variables are associated, it does not mean that one has caused the other.

Example - Salary versus age

An analyst in a large corporation studied the relation between current annual salary and age, for the 46 computer programmers presently employed in the company. She concluded that the relationship was an inverted U-curve, increasing to a maximum salary at age 47 years and then decreasing.

Does this imply that the salary for a programmer increases until age 47 and then decreases?

Cross-Sectional versus Longitudinal Studies

The interpretation here is incorrect because the data describes the salary for programmers at one single point of time - it is called a cross-sectional study. In order to make conclusions about the changes in a programmers salary over time, one would have to conduct a longitudinal study where the investigator follows a group of people through time.

Hence, the manner in which the data are collected must be taken into account when interpreting the results.