Lesson 8 of 10 · 20 min

Analysing a treatment trial

Analyse a three-group placebo-controlled trial, calculating means, standard deviations and the placebo and treatment effects, reading error bars, and drawing conclusions from t-tests.

What you will be able to do

  • Identify the variables, hypotheses and design of a placebo-controlled trial.
  • Calculate the mean, standard deviation and standard error of each group, and the placebo and treatment effects.
  • Interpret a column graph with error bars and the results of unpaired t-tests.
  • Evaluate the reliability and validity of a trial and suggest improvements.

This lesson works through one trial from start to finish, the way a data test would. The trial is fictional, but its design is the one used to test real treatments.

Note

The trial

Researchers recruited 24 adult volunteers who had each been diagnosed with GAD by a clinician. They were randomly allocated to three groups of 8. For eight weeks, one group took a new medication, one took an identical-looking placebo, and one was placed on a waiting list (no treatment). The trial was double-blind for the two pill groups. Each participant's anxiety was rated on a 0–40 scale before and after, and the improvement (fall in score, in points) was recorded.

Improvement in anxiety score after eight weeks (points). Fictional data.
GroupImprovement of each participantMean
Medication12, 9, 14, 10, 11, 13, 8, 1111.00
Placebo7, 5, 8, 6, 4, 9, 6, 76.50
No treatment2, 4, 1, 3, 2, 5, 3, 02.50

Variables and design. The independent variable is the treatment condition (medication, placebo or no treatment). The dependent variable is the improvement in anxiety score after eight weeks, operationalised as the score before minus the score after on the 0–40 rating scale. Different people are in each condition, so this is an independent groups design. Improvement is a number of points on a rating scale with equal steps, so it is treated as interval data (a score of 0 on the scale does not mean no anxiety at all), and means and standard deviations suit it.

For the key comparison, the hypotheses are:

  • H0H_0: there is no difference in mean improvement between people who take the medication and people who take the placebo.
  • H1H_1: there is a difference in mean improvement between people who take the medication and people who take the placebo.

(These are two-tailed hypotheses, matching the two-tailed t-tests below.)

What is the dependent variable in this trial?

Choose one answer

Worked example

Calculate the standard deviation of a trial group

  • Simple familiar
  • Technology active

In the fictional GAD trial, the 8 participants in the medication group improved by 12, 9, 14, 10, 11, 13, 8 and 11 points.

Calculate the mean and the sample standard deviation of the improvement, and the standard error of the mean, each to 2 decimal places.

4 steps

Try it yourself first, then check the working one step at a time.

Method

  1. Add the scores and divide by n for the mean.

  2. Square each deviation from the mean and add them.

  3. Divide by n − 1, then take the square root.

  4. SE = s ÷ √n.

Watch out for

  • Dividing by n=8n = 8 instead of n−1=7n - 1 = 7, which gives 1.871.87.

  • Forgetting to square the deviations; they then add to zero.

Descriptive statistics for each group (calculated with the sample standard deviation)
GroupnMeanSDSE95% CI
Medication811.002.000.719.61 to 12.39
Placebo86.501.600.575.39 to 7.61
No treatment82.501.600.571.39 to 3.61

Mean improvement in anxiety score after eight weeks. Error bars show ±1 standard deviation. Fictional data.

The effects. Using the means:

  • Placebo effect =6.50−2.50=4.00= 6.50 - 2.50 = 4.00 points. Believing they were treated improved participants' scores by 4 points more than waiting.
  • Treatment effect =11.00−6.50=4.50= 11.00 - 6.50 = 4.50 points. The active ingredient added 4.5 points beyond the placebo.
  • The no-treatment group still improved by 2.5 points on average: natural recovery, or simply being in a study.

So of the medication group's 11-point improvement, 2.5 points would have happened anyway, 4 came from the placebo effect and 4.5 from the drug.

A different trial found mean improvements of 9.2 points (medication), 7.8 points (placebo) and 3.1 points (no treatment). Calculate the placebo effect, in points.

in points

Inferential statistics. The researchers ran unpaired t-tests (independent groups):

  • Medication against placebo: t(14)=4.97t(14) = 4.97, p=0.0002p = 0.0002.
  • Placebo against no treatment: t(14)=4.99t(14) = 4.99, p=0.0002p = 0.0002.

Both p-values are well below 0.05, so the researchers reject the null hypothesis in each case. The medication produced a significantly greater improvement than the placebo, and the placebo produced a significantly greater improvement than no treatment: the placebo effect was real and significant in this trial. (The degrees of freedom are n1+n2−2=8+8−2=14n_1 + n_2 - 2 = 8 + 8 - 2 = 14.)

Worked example

Draw a conclusion from a t-test on placebo data

  • Complex familiar

In the fictional GAD trial, the placebo group (mean improvement 6.50 points, SD 1.60, n = 8) was compared with the no-treatment group (mean 2.50 points, SD 1.60, n = 8) using an unpaired t-test: t(14)=4.99t(14) = 4.99, p=0.0002p = 0.0002.

State suitable hypotheses for this comparison and draw a conclusion. Explain what the result shows about the placebo effect.

4 steps

Try it yourself first, then check the working one step at a time.

Method

  1. Write H0H_0 (no difference) and H1H_1 in terms of the variables.

  2. Compare p with 0.05 and decide about H0H_0.

  3. Interpret the difference in the context of the trial.

Watch out for

  • Concluding the placebo "worked" as a treatment for GAD: the result shows the effect of expectation, not an active ingredient.

Interactive

Test the medication against the placebo

The trial's medication and placebo data. Switch the error bars between SD, SE and the 95% CI, then edit the scores.

In another trial, the medication group improved more than the placebo group, but an unpaired t-test gave p=0.21p = 0.21. Which conclusion is correct?

Choose one answer

Reliability and validity. Strengths of this trial: random allocation, a placebo group, double-blinding and a standardised rating scale make it internally valid: the difference between the medication and placebo groups can be put down to the medication. Its limitations are about reliability and external validity: only 8 people per group, a single eight-week trial, and volunteers who may differ from the wider population of people with GAD. A larger sample, a replication, and a follow-up some months later would strengthen the evidence. The no-treatment group cannot be blind (they know they are waiting), so their result may be affected by disappointment.

Exam tip

Data test technique

Show each calculation with numbers substituted and units ("11.00−6.50=4.5011.00 - 6.50 = 4.50 points"), give statistics to 2 decimal places, and state a conclusion with the p-value, the decision about H0H_0 and what it means in the context of the trial.

Common mistake

Calling the whole improvement the drug's effect

The medication group's 11-point improvement includes natural recovery and the placebo effect. The drug's own effect is medication minus placebo: 4.5 points.

Summary

  • A placebo-controlled trial is an independent groups design with a treatment group, a placebo group and a no-treatment group.

  • Placebo effect = placebo mean − no-treatment mean; treatment effect = treatment mean − placebo mean.

  • Describe each group with its mean and sample standard deviation; error bars show SD, SE or a 95% CI.

  • An unpaired t-test with p < 0.05 means reject H0H_0; p ≥ 0.05 means retain it (which does not prove no effect).

  • Random allocation, placebo control and double-blinding give internal validity; larger samples and replication improve reliability.

Finished the lesson?

Try the 3 quick checks above and the lesson completes itself, or mark it complete now.

Lesson 8 of 100% read

Save your progress with a free account