Lesson 8 of 10 · 20 min
Analysing a treatment trial
Analyse a three-group placebo-controlled trial, calculating means, standard deviations and the placebo and treatment effects, reading error bars, and drawing conclusions from t-tests.
What you will be able to do
- Identify the variables, hypotheses and design of a placebo-controlled trial.
- Calculate the mean, standard deviation and standard error of each group, and the placebo and treatment effects.
- Interpret a column graph with error bars and the results of unpaired t-tests.
- Evaluate the reliability and validity of a trial and suggest improvements.
This lesson works through one trial from start to finish, the way a data test would. The trial is fictional, but its design is the one used to test real treatments.
Note
The trial
Researchers recruited 24 adult volunteers who had each been diagnosed with GAD by a clinician. They were randomly allocated to three groups of 8. For eight weeks, one group took a new medication, one took an identical-looking placebo, and one was placed on a waiting list (no treatment). The trial was double-blind for the two pill groups. Each participant's anxiety was rated on a 0–40 scale before and after, and the improvement (fall in score, in points) was recorded.
| Group | Improvement of each participant | Mean |
|---|---|---|
| Medication | 12, 9, 14, 10, 11, 13, 8, 11 | 11.00 |
| Placebo | 7, 5, 8, 6, 4, 9, 6, 7 | 6.50 |
| No treatment | 2, 4, 1, 3, 2, 5, 3, 0 | 2.50 |
Variables and design. The independent variable is the treatment condition (medication, placebo or no treatment). The dependent variable is the improvement in anxiety score after eight weeks, operationalised as the score before minus the score after on the 0–40 rating scale. Different people are in each condition, so this is an independent groups design. Improvement is a number of points on a rating scale with equal steps, so it is treated as interval data (a score of 0 on the scale does not mean no anxiety at all), and means and standard deviations suit it.
For the key comparison, the hypotheses are:
- : there is no difference in mean improvement between people who take the medication and people who take the placebo.
- : there is a difference in mean improvement between people who take the medication and people who take the placebo.
(These are two-tailed hypotheses, matching the two-tailed t-tests below.)
What is the dependent variable in this trial?
Worked example
Calculate the standard deviation of a trial group
In the fictional GAD trial, the 8 participants in the medication group improved by 12, 9, 14, 10, 11, 13, 8 and 11 points.
Calculate the mean and the sample standard deviation of the improvement, and the standard error of the mean, each to 2 decimal places.
Step 1: Find the mean
points.
Step 2: Find each deviation and square it
Deviations from 11: . Squares: . Sum .
Step 3: Divide by n − 1 and take the square root
points.
The course uses the sample standard deviation, which divides by because the sample mean sits closer to the data than the population mean would.
Step 4: Standard error
points.
Try it yourself first, then check the working one step at a time.
Answer
Mean points, standard deviation points, standard error points.
Method
Add the scores and divide by n for the mean.
Square each deviation from the mean and add them.
Divide by n − 1, then take the square root.
SE = s ÷ √n.
Watch out for
Dividing by instead of , which gives .
Forgetting to square the deviations; they then add to zero.
| Group | n | Mean | SD | SE | 95% CI |
|---|---|---|---|---|---|
| Medication | 8 | 11.00 | 2.00 | 0.71 | 9.61 to 12.39 |
| Placebo | 8 | 6.50 | 1.60 | 0.57 | 5.39 to 7.61 |
| No treatment | 8 | 2.50 | 1.60 | 0.57 | 1.39 to 3.61 |
Mean improvement in anxiety score after eight weeks. Error bars show ±1 standard deviation. Fictional data.
The effects. Using the means:
- Placebo effect points. Believing they were treated improved participants' scores by 4 points more than waiting.
- Treatment effect points. The active ingredient added 4.5 points beyond the placebo.
- The no-treatment group still improved by 2.5 points on average: natural recovery, or simply being in a study.
So of the medication group's 11-point improvement, 2.5 points would have happened anyway, 4 came from the placebo effect and 4.5 from the drug.
A different trial found mean improvements of 9.2 points (medication), 7.8 points (placebo) and 3.1 points (no treatment). Calculate the placebo effect, in points.
Inferential statistics. The researchers ran unpaired t-tests (independent groups):
- Medication against placebo: , .
- Placebo against no treatment: , .
Both p-values are well below 0.05, so the researchers reject the null hypothesis in each case. The medication produced a significantly greater improvement than the placebo, and the placebo produced a significantly greater improvement than no treatment: the placebo effect was real and significant in this trial. (The degrees of freedom are .)
Worked example
Draw a conclusion from a t-test on placebo data
In the fictional GAD trial, the placebo group (mean improvement 6.50 points, SD 1.60, n = 8) was compared with the no-treatment group (mean 2.50 points, SD 1.60, n = 8) using an unpaired t-test: , .
State suitable hypotheses for this comparison and draw a conclusion. Explain what the result shows about the placebo effect.
Step 1: Write the hypotheses
: there is no difference in mean improvement between the placebo and no-treatment groups. : there is a difference in mean improvement between the two groups.
Step 2: Compare p with 0.05
, so the difference is statistically significant.
Step 3: Decide
Reject .
Step 4: Interpret
Participants who took an inactive pill improved by 4.00 points more on average () than those who received nothing. Because the pill had no active ingredient and the groups were randomly allocated, the difference is evidence of a placebo effect.
Random allocation is what lets you put the difference down to the condition rather than to differences between participants.
Try it yourself first, then check the working one step at a time.
Answer
, so reject : the placebo group improved significantly more (by 4.00 points) than the no-treatment group, showing a significant placebo effect in this trial.
Method
Write (no difference) and in terms of the variables.
Compare p with 0.05 and decide about .
Interpret the difference in the context of the trial.
Watch out for
Concluding the placebo "worked" as a treatment for GAD: the result shows the effect of expectation, not an active ingredient.
The trial's medication and placebo data. Switch the error bars between SD, SE and the 95% CI, then edit the scores.
In another trial, the medication group improved more than the placebo group, but an unpaired t-test gave . Which conclusion is correct?
Reliability and validity. Strengths of this trial: random allocation, a placebo group, double-blinding and a standardised rating scale make it internally valid: the difference between the medication and placebo groups can be put down to the medication. Its limitations are about reliability and external validity: only 8 people per group, a single eight-week trial, and volunteers who may differ from the wider population of people with GAD. A larger sample, a replication, and a follow-up some months later would strengthen the evidence. The no-treatment group cannot be blind (they know they are waiting), so their result may be affected by disappointment.
Exam tip
Data test technique
Show each calculation with numbers substituted and units (" points"), give statistics to 2 decimal places, and state a conclusion with the p-value, the decision about and what it means in the context of the trial.
Common mistake
Calling the whole improvement the drug's effect
The medication group's 11-point improvement includes natural recovery and the placebo effect. The drug's own effect is medication minus placebo: 4.5 points.
Summary
A placebo-controlled trial is an independent groups design with a treatment group, a placebo group and a no-treatment group.
Placebo effect = placebo mean − no-treatment mean; treatment effect = treatment mean − placebo mean.
Describe each group with its mean and sample standard deviation; error bars show SD, SE or a 95% CI.
An unpaired t-test with p < 0.05 means reject ; p ≥ 0.05 means retain it (which does not prove no effect).
Random allocation, placebo control and double-blinding give internal validity; larger samples and replication improve reliability.
Finished the lesson?
Try the 3 quick checks above and the lesson completes itself, or mark it complete now.