Lesson 5 of 8 · 18 min

Reliability and validity of IQ and EQ tests

Discuss how reliable and valid IQ and EQ tests are, using test–retest, internal consistency, content, construct and criterion evidence, and judge when a score can mislead.

What you will be able to do

  • Define reliability and validity and name their main types.
  • Interpret a reliability coefficient, including a test–retest correlation.
  • Discuss the degree to which IQ tests are reliable and valid.
  • Consider whether IQ and EQ tests can be misleading or inaccurate.

Imagine a bathroom scale. If it shows a different weight each time you step on it, it is unreliable. If it always shows exactly 3 kg too much, it is reliable but not valid. A psychological test is judged the same way, and the judgement matters most when one score decides a school placement, a diagnosis or a job.

Definition

Reliability and validity

Reliability is the consistency of a test: whether it gives the same results when it is repeated, scored by another person or split into parts.

Validity is whether a test measures what it claims to measure, so that conclusions drawn from its scores are justified.

Interactive

Reliable, valid, both or neither

One fictional person sits the same test many times. Random error scatters the scores; systematic error shifts them all one way.

Reliability

Reliability is reported as a reliability coefficient, a correlation between two sets of scores from 0 (no consistency) to 1 (perfect). For tests used to make decisions about individuals, psychologists want 0.9 or above.

Types of reliability
TypeQuestion it answersHow it is measured
Test–retestAre scores consistent over time?The same people sit the same test twice, weeks apart; correlate the two sets of scores
Internal consistency (split-half)Do all the items measure the same thing?Correlate scores on two halves of the test, such as odd and even items
Inter-raterDo different scorers agree?Two trained psychologists score the same responses; compare their scores
Parallel formsDo two versions of a test give the same result?The same people sit both versions; correlate the scores
Interactive

A test–retest check

Illustrative data: ten adults sat the same IQ test twice. The points lie close to a straight line, so test–retest reliability is high.

Worked example

Judge the test–retest reliability of an EQ questionnaire

  • Complex familiar
  • Technology active

Eight students completed a self-report emotional intelligence questionnaire (scored out of 50), then completed it again four weeks later (illustrative data).

Student12345678
First score3441293845313640
Retest score3639333544303837

(a) Use a calculator to find Pearson's r, to 2 decimal places. (b) What type of reliability does r measure here? (c) The p-value for this r is 0.004. Evaluate the questionnaire's reliability.

5 steps

Try it yourself first, then check the working one step at a time.

Method

  1. Same test, same people, two occasions: test–retest.

  2. Find r, describe it, check p against 0.05.

  3. Compare with 0.9 for individual decisions; comment on sample size and validity.

Watch out for

  • Calling a high r "valid". It shows consistency, not accuracy.

  • Saying the scores are reliable because the means are similar (36.75 and 36.5). Reliability is about whether each person keeps their position, which r measures.

How reliable are IQ tests? The full-scale IQs of the WAIS-IV, WISC-V and SB5 have reliability coefficients above 0.9, by internal consistency and test–retest, which is very high for any psychological measure. Single subtests are less reliable, so a psychologist interprets the full-scale and index scores, not one subtest. Two things lower reliability in practice:

  • Practice effects: sitting the same test again soon raises scores by a few points, because the tasks are familiar.
  • The testing situation: tiredness, illness, anxiety, low motivation or an inexperienced tester add random error on the day.

That error is why reports give a confidence interval: a full-scale IQ of 104 might be reported as "99 to 109 with 95% confidence".

Two psychologists independently score the same child's answers to the vocabulary subtest and give the same scaled score. What does this show?

Choose one answer

Validity

A test can be perfectly consistent and still measure the wrong thing, so validity is the harder question.

Types of validity
TypeQuestion it answersEvidence for IQ tests
Content validityDo the items cover the whole of what is being measured?Wechsler scales sample verbal, spatial, reasoning, memory and speed tasks, but leave out creativity, practical and emotional abilities
Construct validityDoes the test measure the construct (intelligence), not something else?Subtests correlate with one another as g predicts; but scores also reflect education, language and familiarity with tests
Criterion validity: concurrentDo scores agree with another measure taken at the same time?Scores on the WAIS-IV and SB5 correlate highly with each other
Criterion validity: predictiveDo scores predict a later outcome?IQ predicts school results (correlation about 0.5), years of education and job training success

The evidence makes IQ tests valid measures of the abilities they sample, especially those needed for school, and moderately good predictors of educational outcomes. A correlation of about 0.5 with school results is strong for psychology, but it leaves most of the differences in school results to motivation, teaching, health, family and other abilities. The tests are less valid as measures of intelligence in the wider sense that Gardner, Sternberg and Goleman describe.

Researchers find that students' WISC-V scores in Year 5 correlate r=0.52r = 0.52 with their Year 9 NAPLAN numeracy results. What type of validity does this provide evidence for?

Choose one answer

Can IQ and EQ tests mislead?

A test can be reliable and valid on average and still mislead about one person, or be used for something it was not designed for.

Ways a score can mislead

  1. Measurement error near a cut-off

    A score of 72 with a confidence interval of 67 to 78 could fall either side of 70. Decisions near a cut-off need other evidence.

  2. Old norms

    Average scores rose about 3 points per decade in the twentieth century (the Flynn effect), so a test scored against norms collected 20 years earlier can overestimate a person's IQ by about 6 points.

  3. A single number for an uneven profile

    A full-scale IQ of 100 can come from very high verbal and very low processing speed scores. The total hides what matters for support.

  4. A norm group that does not represent the person

    Norms from one country, language or culture may not fit a person from another. The next lesson is about this.

  5. The testing situation

    Anxiety, fatigue, illness or low motivation lower a score on the day.

EQ tests are more open to these problems. Self-report questionnaires have good internal consistency, but they measure how emotionally skilled people believe they are, and people can present themselves well when a job is at stake. Their scores overlap heavily with personality traits such as extraversion and emotional stability, so their construct validity is questioned. Ability tests of EI are harder to fake, but deciding the "correct" emotional response needs an expert or majority judgement, which may not suit every culture, and some of their parts have lower reliability. EQ scores add only a little to the prediction of work performance once IQ and personality are known.

Select all the reasons a self-report EQ questionnaire used for job selection could give a misleading result.

Select all that apply.

Common mistake

Reliable is not the same as valid

A test can be reliable without being valid: a vocabulary test in English gives consistent scores for a recent arrival who speaks little English, and consistently underestimates their reasoning. A test cannot be valid without being reliable, because a score that changes at random cannot be accurate.

Exam tip

Discussing "the degree to which"

Answer with a degree, not yes or no: "IQ tests are highly reliable (full-scale coefficients above 0.9) and valid for predicting school results (r ≈ 0.5), but their validity as measures of intelligence in the broader sense is limited because…". Name each type of reliability or validity you use.

Summary

  • Reliability is consistency (test–retest, internal consistency, inter-rater, parallel forms); validity is measuring what the test claims (content, construct, criterion: concurrent and predictive).

  • Full-scale IQs on current scales have reliability coefficients above 0.9; subtests are less reliable.

  • IQ tests predict school results moderately well (r ≈ 0.5) but sample only some abilities.

  • Scores can mislead through measurement error near cut-offs, old norms (the Flynn effect), uneven profiles, unrepresentative norms and the testing situation.

  • Self-report EQ can be faked and overlaps with personality; ability-based EQ is harder to fake but its "correct" answers rest on judgement.

Finished the lesson?

Try the 3 quick checks above and the lesson completes itself, or mark it complete now.

Lesson 5 of 80% read

Save your progress with a free account