Lesson 2: Bias and procedures that ensure randomness

Evaluate the design of a council survey

SkillsSpotting bias in samplesProcedures that ensure randomness

Worked example

  • Complex unfamiliar
  • Technology free

A regional Queensland council wants to estimate the proportion of its 24 000 households that would use a kerbside green-waste bin. It mails a survey to every household in the three suburbs closest to the council offices, with the question

"Would you support a green-waste bin to help protect our environment?"

Of the 3000 surveys mailed, 900 are returned, and 81 % of those say yes.

(a) Identify three sources of bias in this survey and explain the likely effect of each.

(b) The council has a list of all 24 000 households and knows that 9600 are in town, 10 800 in suburbs and 3600 on rural blocks. Design a better procedure to select a sample of 400 households.

1 of 6 steps
  1. Step 1: Look at who could be chosen

    Only three suburbs, all near the council offices, are sampled. That is undercoverage: rural households, which may already compost or burn green waste, have no chance of selection. The estimate could be pushed either way, but it can't be trusted to represent the whole council area.

Try a similar one

A fresh question of the same kind, checked as you answer, with hints and a full solution.

Method

  1. Ask three questions: who could be chosen? who actually responded? how was the question asked?

  2. For each source of bias, say which way it pushes the estimate.

  3. Fix the frame with a full list, the selection with random numbers (stratified if groups differ), and the response with neutral wording and follow-up.

Watch out for

  • Saying "the sample is too small". 900 is plenty; the problem is how it was chosen.

  • Naming a bias without explaining its effect on the estimate.

Save your progress with a free account