Lesson 11 of 11 · 20 min

Investigating the Stroop effect: repeated measures and the paired t-test

Measure the Stroop effect with a repeated measures experiment, control order effects with counterbalancing, describe the data with means, medians and error bars, and test it with a paired t-test, allowing for type I and II errors.

What you will be able to do

  • Describe Stroop's (1935) finding and explain the Stroop effect with automatic and controlled processing.
  • Design a repeated measures Stroop experiment: variables, hypotheses, counterbalancing and other controls, and ethics.
  • Describe the data with the mean, median, interquartile range and standard deviation, and choose the median when there are outliers.
  • Choose and interpret a paired t-test, and explain type I and type II errors.
Interactive

Name the ink, not the word

Say or tap the colour of the ink as fast as you can. Your times stay in your browser and are not a diagnosis.

Naming the colour of the ink should be easy. But when the word itself names a different colour (RED printed in blue), most people slow down and some say the word instead. This interference is the Stroop effect.

History

Stroop (1935)

John Ridley Stroop asked university students to work through cards of 100 items as fast as they could.

  • Reading colour words aloud took about as long whether the words were printed in black or in clashing ink colours (a difference of about 2 seconds).
  • Naming the ink colours of clashing colour words took far longer than naming the colours of plain squares: on average 110.3 s against 63.3 s, 47 seconds or 74% longer.

Conclusion: the meaning of a word interferes with naming its colour, but the colour hardly interferes with reading the word.

Why? For a practised reader, reading is an automatic process: it happens fast and without effort, and you cannot stop it. Naming an ink colour is a controlled process that needs attention. When the word and the ink disagree, the automatic reading response competes with the colour you are trying to name, and selective attention has to suppress it, which takes time. Young children who cannot yet read fluently show little Stroop effect.

A class experiment. Each of eight Year 11 students names aloud the ink colours of two cards of 20 colour words, timed with a stopwatch:

  • Congruent: each word printed in its own colour (RED in red).
  • Incongruent: each word printed in a different colour (RED in blue).

Every student does both conditions, so this is a repeated measures design.

The design of the Stroop experiment
FeatureIn this experiment
IVCongruence of the words and ink colours: congruent or incongruent
DVTime in seconds to name aloud the ink colours of all 20 words, measured with a stopwatch
Controlled variablesThe same four colours and 20 words on each card, the same font size and card, the same lighting, the same experimenter and stopwatch, standardised instructions ("name the colour of the ink as fast as you can")
DesignRepeated measures, with counterbalancing
H₀There will be no difference in the mean time Year 11 students take to name the ink colours of 20 congruent and 20 incongruent colour words.
H₁Year 11 students will take a different mean time to name the ink colours of 20 incongruent colour words from 20 congruent colour words.

Key idea

Order effects and counterbalancing

In repeated measures, the order of conditions can change the results: by the second card a student has practised (faster) or is bored or tired (slower). These are order effects. Counterbalancing cancels them: half the participants do congruent then incongruent, the other half incongruent then congruent, so any order effect is spread equally across the two conditions.

The advantage of repeated measures is that each student is compared with themselves, so differences between people (reading speed, eyesight) cannot explain the result, and fewer participants are needed.

Interactive

Counterbalance the Stroop experiment

Practice makes the second card faster. Watch the measured effect with and without counterbalancing.

Why should a Stroop experiment with a repeated measures design be counterbalanced?

Choose one answer
Time (s) to name the ink colours of 20 words
StudentCongruentIncongruentDifference (I − C)
110.217.57.3
212.119.06.9
39.815.15.3
411.520.28.7
513.018.85.8
610.716.45.7
712.421.38.9
811.118.17.0
Summary statistics (sample SD; quartiles by the median of each half)
ConditionMeanMedianIQRSD
Congruent11.3511.301.801.11
Incongruent18.3018.452.652.00
Difference (I − C)6.956.95—1.34

Mean time to name the ink colours of 20 words. Error bars show ±1 standard deviation.

Definition

Median and interquartile range

The median is the middle score when the scores are in order (the mean of the two middle scores when nn is even). The interquartile range (IQR) is the spread of the middle half: Q3−Q1Q_3 - Q_1, where Q1Q_1 and Q3Q_3 are the medians of the lower and upper halves of the data.

Use the median and IQR when the data have outliers or are skewed: they barely move, while the mean and SD can be pulled a long way by one extreme score. Response times often have outliers (a student loses their place and takes 40 seconds).

Student 8 lost their place and their incongruent time was recorded as 41.1 s instead of 18.1 s. Which statistic of the incongruent condition would change least?

Choose one answer

Which t-test? Each student gave two scores, so the scores come in pairs. Use the paired t-test, which works on each student's difference (incongruent − congruent). It asks whether the mean difference, 6.95 s, is far enough from 0, given how much the differences vary.

t=dˉsd/n=6.951.3416/8=14.65,df=n−1=7t = \frac{\bar{d}}{s_d / \sqrt{n}} = \frac{6.95}{1.3416 / \sqrt{8}} = 14.65, \quad df = n - 1 = 7

(Keep sds_d unrounded, 1.34161.3416: with 1.341.34 you get 14.6714.67.)

In a spreadsheet: =T.TEST(congruent, incongruent, 2, 1) (two-tailed, paired). p<0.001p < 0.001 (about 0.000002).

Key idea

Paired or unpaired?

  • Repeated measures or matched participants: each score in one condition belongs with one in the other → paired t-test, df=n−1df = n - 1 (pairs).
  • Independent groups: different people in each condition → unpaired t-test, df=n1+n2−2df = n_1 + n_2 - 2.

Using the unpaired test on paired data throws away the pairing. When people differ a lot from each other, that can turn a significant result into a non-significant one.

Worked example

Analyse a Stroop experiment with a neutral condition

  • Complex familiar
  • Technology active

Six students named the ink colours of 20 rows of XXXX (neutral) and of 20 incongruent colour words, in counterbalanced order. Times in seconds:

Student123456
Neutral12.410.914.211.813.512.0
Incongruent17.915.221.016.219.416.6

(a) Calculate each student's difference (incongruent − neutral) and the mean difference. (b) Identify the appropriate t-test and its degrees of freedom. (c) The test gives p=0.00005p = 0.00005. State the conclusion.

4 steps

Try it yourself first, then check the working one step at a time.

Method

  1. Subtract each participant's two scores in the same order.

  2. Paired data: paired t-test, df=n−1df = n - 1.

  3. Decision (compare pp with 0.05), then a conclusion in words with the means.

Watch out for

  • Using df=n1+n2−2=10df = n_1 + n_2 - 2 = 10, the unpaired value.

  • Subtracting in different orders for different students.

Interactive

Paired or unpaired?

Sample data from a different repeated measures experiment. Each line joins one person's two scores. What happens to pp when the pairing is ignored?

Type I and type II errors. A decision made from a sample can be wrong in two ways.

  • A type I error (a false positive) is rejecting the null hypothesis when it is actually true: concluding the IV had an effect when the difference was just chance. With a 0.05 significance level, this happens in about 5% of experiments where there is no real effect.
  • A type II error (a false negative) is retaining the null hypothesis when it is false: missing a real effect. It is more likely with small samples, a lot of variability, or a small effect.

Lowering the significance level (to 0.01) makes type I errors rarer but type II errors more common. A larger sample reduces type II errors.

Interactive

Run 100 experiments

Each run is a simulated experiment with two groups and an unpaired t-test. Count the false alarms and the misses.

A class of six students finds no significant Stroop effect (p=0.09p = 0.09), although the effect is well established. If there really is an effect, which error has the class made, and what is its most likely cause?

Choose one answer

Note

Ethics and safety

Check for colour vision deficiency (about 1 in 12 males) before the experiment and give those students another role: their data would not be valid, and the task could be uncomfortable. Gain informed consent, allow withdrawal at any time, keep times confidential, and debrief. Avoid comments on individual students' times.

Interactive

Check a proposal

A first draft of a class Stroop practical. Mark each step as fine or name the ethical principle it breaches.

Summary

  • Stroop (1935): naming the ink colours of clashing colour words took 74% longer than naming colour squares; reading is automatic and interferes with the controlled task of naming colours.

  • Repeated measures designs compare each participant with themselves; counterbalancing spreads order effects (practice, fatigue) equally across conditions.

  • The median and IQR resist outliers; the mean and SD are pulled by them.

  • Paired data (repeated measures, matched participants) use the paired t-test on the differences, df = n − 1; independent groups use the unpaired t-test.

  • A type I error rejects a true H₀ (false positive, about 5% at α = 0.05); a type II error retains a false H₀ (a miss, more likely with small samples).

Finished the lesson?

Try the 3 quick checks above and the lesson completes itself, or mark it complete now.

Lesson 11 of 110% read

Save your progress with a free account