Skip to main content

Advertisement

Table 1 Inter-rater reproducability results for the initial group of three trainee clinicians for 14 pathoanatomic categories across the four coding rounds, expressed as a Kappa co-efficient and percentage agreement with 95% confidence intervals (95% CI)

From: Inexperienced clinicians can extract pathoanatomic information from MRI narrative reports with high reproducibility for use in research/quality assurance

Pathoanatomic category Reliability index (95%CI) Coding round 1 Coding round 2 Coding round 3 Coding round 4
Intervertebral disc degeneration Prevalence* Kappa Percentage agreement 71.7% (50.2-93.1) 0.66 (0.54-0.77) 80.8% (73.8-87.9) 94.0 (92.0-96.0) 0.94 (0.84-1.00) 95.8 (92.3-99.4) 80.0 (80.0-80.0) n/a 98.2 (95.8-100.0) 83.3 (80.1-86.6) 0.91 (0.80-1.00) 97.5 (94.7-100.0)
Intervertebral disc bulge Prevalence Kappa Percentage agreement 73.3% (70.1-76.6) 0.91 (0.78-1.00) 95.0% (94.7-100.0) 87.0 (83.1-90.9) 0.86 (0.76-0.90) 92.5 (87.7-97.2) 70.0 (70.0-70.0) 1.00 (0.87-1.00) 100.0 48.3 (45.1-51.6) 0.89 (0.76-1.00) 99.2 (97.5-100.0)
Intervertebral disc herniation Prevalence Kappa Percentage agreement 40.0% (34.3-45.7) 0.91 (0.73-1.00) 97.5% (94.7-100.0) 61.0 (59.0-63.0) 1.00 (0.85-1.00) 100.0 40.0 (40.0-40.0) 0.98 (0.87-1.00) 96.5 (93.1-99.9) 45.0 (45.0-45.0) 1.00 (0.82-1.00) 100.0
Nerve root compromise Prevalence Kappa Percentage agreement 13.3% (4.7-22.0) n/a 97.5% (94.7-100.0) 42.0 (31.9-52.1) 0.74 (0.49-1.00) 94.2 (90.0-98.4) 31.7 (23.0-40.3) 0.63 (0.45-0.81) 93.0 (88.3-97.7) 18.3 (15.1-21.6) 1.00 (0.75-1.00) 99.2 (97.5-100.0)
Modic change type 1 Prevalence Kappa Percentage agreement 26.7% (23.4-29.9) 0.86 (0.68-1.00) 95.0% (91.1-98.1) 18.0 (15.6-20.4) 0.86 (0.60-1.00) 98.3 (96.0-100.0) 20.0 (20.0-20.0) 1.00 (0.82-1.00) 100.0 18.3 (15.1-21.6) 0.86 (0.60-1.00) 100.0
Modic change type 2 Prevalence Kappa Percentage agreement 25.0% (19.3-30.7) 0.76 (0.51-1.00) 97.5% (94.7-100.0) 10.0 (5.6-14.4) n/a 98.3 (96.0-100.0) 20.0 (20.0-20.0) 1.00 (0.82-1.00) 100.0 5.0 (5.0-5.0) n/a 100.0
Spondylolisthesis Prevalence Kappa Percentage agreement 20.0% (20.0-20.0) n/a 97.5% (94.7-100.0) 19.0 (17.0-21.0) 1.00 (0.75-1.00) 100.0 10.0 (10.0-10.0) 1.00 (0.82-1.00) 100.0 0.0 (0.0-0.0) n/a 100.0
Stenosis Prevalence Kappa Percentage agreement 6.7% (3.4-9.9) n/a 97.5% (94.7-100.0) 23.0 (20.6-25.4) 0.84 (0.59-1.00) 99.2 (97.5-100.0) 25.0 (25.0-25.0) 0.95 (0.74-1.00) 99.1 (97.4-100.0) 0.0 (0.0-0.0) n/a 100.0
Scoliosis Prevalence Kappa Percentage agreement 25.0% (25.0-25.0) 1.00 (0.82-1.00) 100.0% 6.0 (4.0-8.0) n/a 95.0 (91.1-98.9) 5.0 (5.0-5.0) n/a 99.1(97.4-100.0) 5.0 (5.0-5.0) n/a 100.0
Osteophytes Prevalence Kappa Percentage agreement 5.0% (5.0-5.0) n/a 100.0% 1.0 (0.0-3.0) n/a 100.0 10.0 (10.0-10.0) n/a 99.1 (97.4-100.0) 10.0 (10.0-10.0) 1.00 (0.75-1.00) 99.2 (97.5-100.0)
Facet joint arthrosis Prevalence Kappa Percentage agreement 0.0% (0.0) n/a 100.0% 14.0 (12.0-16.0) 0.68 (0.42-0.93) 95.0 (91.1-98.9) 20.0 (20.0-20.0) 1.00 (0.82-1.00) 100.0 5.0 (5.0-5.0) n/a 100.0
Endplate irregularities Prevalence Kappa Percentage agreement # 1.0 (0.0-3.0) n/a 99.2 (97.5-100.0) 25.0 (19.3-30.7) 1.00 (0.82-1.00) 98.2 (95.8-100.0) 18.3 (11.8-24.9) 0.74 (0.56-0.92) 97.5 (94.7-100.0)
Red flags Prevalence (%) Kappa Percentage agreement # 0.0 (0.0-0.0) n/a 100.0 8.3 (1.8-14.9) n/a 98.2 (95.8-100.0) 0.0 (0.0-0.0) n/a 100.0
High intensity zone Prevalence (%) Kappa Percentage agreement # 15.0 (11.9-18.1) 0.84 (0.59-1.00) 99.2 (97.5-100.0) 26.7 (20.1-33.2) 0.85 (0.67-1.00) 98.2 (95.8-100.0) 25.0 (25.0-25.0) 1.00 (0.82-1.00) 100.0
  1. * The prevalence estimates are calculated on a 'whole person' level. That is, the presence of a particular pathology in a person at one or more vertebral levels was counted as a single instance.
  2. 95%CI = 95% confidence interval in brackets.
  3. # not included in coding round 1.
  4. n/a = kappa co-efficient not calculated. Calculation of kappa in STATA requires the prevalence of the pathology to be 10% to 90% at each and every vertebral level being compared. This restriction applies to the findings of all raters being compared.