Lesson 7: Reliability and validity of diagnosis

Calculate per cent agreement between two clinicians

SkillReliability and validity of diagnosis

Worked example

  • Simple familiar

Two clinicians independently assess the same 40 fictional people for an anxiety disorder. Their judgements are shown in the table.

Clinician B: disorderClinician B: no disorder
Clinician A: disorder145
Clinician A: no disorder318

(a) Calculate the per cent agreement. (b) State what this measures. (c) Suggest one way to improve it.

1 of 4 steps
  1. Step 1: Find the agreed cases

    The clinicians agree when both say "disorder" (14) or both say "no disorder" (18): 14+18=3214 + 18 = 32. The other 5+3=85 + 3 = 8 are disagreements; check: 32+8=4032 + 8 = 40.

Try a similar one

A fresh question of the same kind, checked as you answer, with hints and a full solution.

Method

  1. Agreed cases = both "disorder" + both "no disorder".

  2. Agreement = agreed ÷ total × 100%.

  3. Name it as reliability, not validity.

Watch out for

  • Using only the 14 "both disorder" cases: 1440=35%\frac{14}{40} = 35\%.

  • Calling 80% agreement evidence that the diagnoses are valid.

Save your progress with a free account