Lesson 6 of 8 · 17 min

Cultural bias in intelligence testing

Evaluate how far intelligence tests are culturally biased, how test makers reduce bias, and how well each theory of intelligence applies across social and cultural contexts.

What you will be able to do

  • Explain the main sources of cultural bias in intelligence tests.
  • Explain how test makers and psychologists try to reduce bias, and the limits of culture-fair tests.
  • Evaluate the degree to which intelligence tests are culturally biased.
  • Discuss how well theories of intelligence apply across social and cultural contexts.

Here is an invented test item: "What should you fit to a car's tyres before driving on a snowy mountain road?" A student from Hobart who skis might know at once. A student from Cairns, just as quick a thinker, might never have heard of snow chains. The item rewards experience of a particular way of life, not reasoning, and that is the core of cultural bias.

Definition

Cultural bias

Cultural bias is a systematic difference in how a test works for people from different cultures, so that their scores reflect familiarity with the test's language, knowledge and ways of doing things as well as the ability it claims to measure.

Cultural bias is a validity problem. A biased test is not measuring the same construct for everyone who sits it, so a score means something different for one group than for another. It can still be perfectly reliable: it can underestimate the same person by the same amount every time.

Sources of cultural bias
SourceWhat happensExample
LanguageThe test is given in a language or dialect the person does not speak fluently, or its wording favours one dialectA child who speaks Aboriginal English or Torres Strait Creole at home sits a vocabulary subtest in Standard Australian English
Content and knowledgeItems assume knowledge more common in one cultureQuestions about snow chains, a particular sport or a particular holiday
Format and familiaritySome people have far more practice with timed, written, multiple-choice puzzlesStudents who have sat many school tests are "test-wise"
NormsThe norm group does not include people like the person testedComparing a recent arrival with norms from people raised and schooled in Australia
Testing situationAnswering a stranger's questions, alone and against the clock, is normal in some cultures and uncomfortable or impolite in othersA child taught not to show off, or not to answer an adult's question directly
ValuesThe test rewards what one culture counts as intelligent, such as speedA culture that values careful, unhurried answers loses marks on timed subtests

Differences in average scores. When two groups' average scores differ, psychologists treat it first as a question about the test, not about the people: does each item, and the test as a whole, work the same way in both groups? One way to check is to compare people from the two groups who have the same overall score. If they still differ on one item, that item is behaving differently in the two groups and is probably biased. Test makers use this check to remove biased items before a test is published.

Interactive

Find the biased item

The item results of two groups matched on overall ability. A fair item is passed equally often by both.

Worked example

Find and explain a biased test item

  • Complex unfamiliar

A test developer trialled five items of a new reasoning test with two groups of Year 7 students from different cultural backgrounds. The groups were matched on their total scores on the rest of the test. The table shows the percentage of each group who answered each item correctly (illustrative data).

Item12345
Group A (%)7265588044
Group B (%)7063317846

Item 3 asks which of four winter sports uses a toboggan.

(a) Identify the item most likely to be culturally biased, using the data. (b) Explain why it is biased, and evaluate what the developer should do.

5 steps

Try it yourself first, then check the working one step at a time.

Method

  1. With groups matched on ability, compare pass rates item by item.

  2. A large gap on one item suggests bias in that item.

  3. Explain what besides the ability the item measures.

  4. Evaluate the remedy and its limits.

Watch out for

  • Choosing the hardest item (item 5) instead of the one with the biggest gap between groups.

  • Treating a gap between matched groups as a difference in ability.

Which of these is the clearest evidence that a particular test item is culturally biased?

Choose one answer

Reducing bias

  • Culture-reduced tests use non-verbal items. Raven's Progressive Matrices (1938) asks the person to choose the piece that completes a pattern of shapes; Cattell's Culture Fair Intelligence Test (1949) uses similar puzzles.
  • Non-verbal scores: the SB5 gives a nonverbal IQ and the WISC-V a nonverbal index, built from tasks that need little spoken language.
  • Local norms, such as the Australian and New Zealand edition of the WISC-V.
  • Testing in the person's first language, with a trained interpreter where needed, and translated tests that have been re-normed.
  • A culturally competent psychologist who builds rapport, explains the purpose, consults with the person's family and community, and interprets the score alongside other evidence.

Note

Culture-fair is not culture-free

Even pattern puzzles are a cultural product. Children who have done years of school worksheets have practised exactly this kind of abstract, timed problem. Scores on Raven's matrices rose more over the twentieth century than scores on almost any other test (the Flynn effect), which shows how much they depend on experience. No test can measure intelligence free of culture.

Note

Assessing First Nations peoples

Most standardised tests were developed and normed with populations that included few Aboriginal peoples and Torres Strait Islander peoples, and many First Nations children grow up speaking more than one language or dialect. Guidance for Australian psychologists stresses cultural consultation, involving family and community, considering language, and treating a score from a standardised test with great caution. A low score in these conditions may say more about the test's fit than about the person.

Do theories of intelligence apply across cultures?

Theories carry their culture's idea of intelligence with them. When researchers have asked people in different communities what makes someone intelligent, many answers in African and Asian communities included being socially responsible, cooperative, respectful and wise, not only quick at solving puzzles. A theory built on Western school tasks may not describe what intelligence means in those communities.

How each approach travels across social and cultural contexts
ApproachApplies well because…Applies less well because…
PsychometricPositive correlations among mental tests (g) are found in many countriesTests must be translated, adapted and re-normed; they reward school-type, timed tasks
Information processingSpeed and working memory tasks need little language or knowledgeNot every culture values speed; people less familiar with laboratory tasks may perform below their ability
SternbergPractical intelligence is defined as success in your own context, so it expects intelligence to look different in different placesIts measures of tacit knowledge must be rebuilt for each context
GardnerValues abilities, such as interpersonal and naturalist, that many cultures prizeWhich abilities count as intelligences is itself a cultural choice
GolemanRecognising and managing emotions matters everywhereRules about which emotions to show, and how, differ between cultures, so the "correct" emotional response differs too

Social contexts matter as well as cultural ones. Access to schooling, books, nutrition and health care differs with family income and with remoteness, and each affects test performance. A test normed on mostly urban, schooled children may disadvantage children with less access to schooling, whatever their culture.

Which theory most directly expects that what counts as intelligent behaviour will differ from one cultural context to another?

Choose one answer

Exam tip

Evaluating the degree of bias

"Evaluate the degree" wants a judgement with evidence on both sides. Biased: language and content favour the culture of the test makers; norms may not represent the person; the testing situation and the value placed on speed differ between cultures. Less biased than they were: items are screened for bias, there are non-verbal scores and local norms, and skilled psychologists interpret scores in context. Judgement: current tests are much less biased than early ones, but no test is culture-free, so a score from someone outside the norm group must be interpreted with care.

Summary

  • Cultural bias is a validity problem: the test measures cultural familiarity as well as the ability it claims to measure.

  • Bias comes from language, content, format, norms, the testing situation and the values a test rewards.

  • An item is probably biased if groups matched on overall ability pass it at different rates; a difference in averages is a reason to check the test, not evidence about any group's ability.

  • Culture-reduced tests (Raven's matrices), non-verbal scores, local norms, first-language testing and culturally competent psychologists reduce bias, but no test is culture-free.

  • Each theory reflects its culture's idea of intelligence; Sternberg's practical intelligence is the most explicitly contextual.

Finished the lesson?

Try the 2 quick checks above and the lesson completes itself, or mark it complete now.

Lesson 6 of 80% read

Save your progress with a free account