Lesson 5: Reliability and validity of IQ and EQ tests
Judge the test–retest reliability of an EQ questionnaire
SkillsReliability and validity of testsResearch methods in intelligence
Worked example
Eight students completed a self-report emotional intelligence questionnaire (scored out of 50), then completed it again four weeks later (illustrative data).
| Student | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 |
|---|---|---|---|---|---|---|---|---|
| First score | 34 | 41 | 29 | 38 | 45 | 31 | 36 | 40 |
| Retest score | 36 | 39 | 33 | 35 | 44 | 30 | 38 | 37 |
(a) Use a calculator to find Pearson's r, to 2 decimal places. (b) What type of reliability does r measure here? (c) The p-value for this r is 0.004. Evaluate the questionnaire's reliability.
Step 1: Enter the pairs and find r
Enter first scores as and retest scores as in the calculator's two-variable statistics. .
Step 2: Name the reliability
The same people took the same test on two occasions, so r is a test–retest reliability coefficient.
Step 3: Describe and test r
is a strong positive correlation, and , so it is statistically significant: students who scored high the first time tended to score high again.
Step 4: Compare with the standard for decisions about individuals
Psychologists look for 0.9 or above before using a test to make decisions about individuals. 0.88 is close but just below.
Step 5: Note the limits of the evidence
Only 8 students from one school took part, so the estimate is imprecise. And reliability is not validity: a consistent self-report score could still measure how students see themselves rather than their actual emotional skills.
Answer
(a) . (b) Test–retest reliability. (c) A strong, significant positive correlation shows good consistency over four weeks, just under the 0.9 usually wanted for decisions about individuals; with only 8 students the estimate is uncertain, and reliability does not show the questionnaire is valid.
Try a similar one
A fresh question of the same kind, checked as you answer, with hints and a full solution.
Method
Same test, same people, two occasions: test–retest.
Find r, describe it, check p against 0.05.
Compare with 0.9 for individual decisions; comment on sample size and validity.
Watch out for
Calling a high r "valid". It shows consistency, not accuracy.
Saying the scores are reliable because the means are similar (36.75 and 36.5). Reliability is about whether each person keeps their position, which r measures.