Skip to main content

Advertisement

Table 3 Inter-rater reproducability reliability results for the second group of three trainee clinicians for 14 pathoanatomic categories across a single coding round, expressed as a Kappa co-efficient and percentage agreement with 95% confidence intervals (95% CI)

From: Inexperienced clinicians can extract pathoanatomic information from MRI narrative reports with high reproducibility for use in research/quality assurance

Pathoanatomic category Reliability index (95%CI) Inter-rater Coding round 1 Inter-rater Coding round 2 Test-retest (intra-rater)
Intervertebral disc degeneration Prevalence* Kappa Percentage agreement 90.0% (90.0-90.0) 0.99 (0.87-1.00) 99.3% (99.0-100.0) 90.0% (90.0-90.0) 0.93 (0.82-1.00) 96.7% (95.0-99.0) - 0.88 to 1.00 97.3% (94.0-100.0)
Intervertebral disc bulge Prevalence Kappa Percentage agreement 90.0% (90.0-90.0) 0.96 (0.84-1.00) 98.0% (97.0-99.0) 85.0% (79.3-90.7) 0.93 (0.80-1.00) 96.7% (95.0-99.0) - 0.79 to 0.96 94.3% (91.0-97.0)
Intervertebral disc herniation Prevalence Kappa Percentage agreement 58.3% (55.1-61.6) 0.94 (0.80-1.00) 98.7% (98.0-99.0) 58.3% (55.1-61.6) 0.94 (0.80-1.00) 98.7% (98.0-100.0) - 0.86 to 0.91 97.3% (97.0-98.0)
Nerve root compromise Prevalence Kappa Percentage agreement 53.3% (50.1-56.6) 0.76 (0.58-0.94) 96.0% (94.0-98.0) 55.0% (49.3-60.7) 0.81 (0.63-0.99) 97.3% (96.0-98.0) - 0.61 to 0.86 95.3% (94.0-98.0)
Modic change type 1 Prevalence Kappa Percentage agreement 20.0% (20.0-20.0) 1.00 (0.82-1.00) 100.0% (100.0-100.0) 21.7% (18.4-24.9) 0.93 (0.75-1.00) 99.3% (99.0-100.0) - 0.89 to 1.00 99.7% (99.0-100.0)
Modic change type 2 Prevalence Kappa Percentage agreement 10.0% (10.0-10.0) 1.00 (0.75-1.00) 100.0% (100.0-100.0) 8.3% (5.1-11.6) n/a 99.3% (99.0-100.0) - 1.00 to 1.00 99.7% (99.0-100.0)
Spondylolisthesis Prevalence Kappa Percentage agreement 20.0% (20.0-20.0) 1.00 (0.75-1.00) 100.0% (100.0-100.0) 20.0% (20.0-20.0) 1.00 (0.75-1.00) 100.0% (100.0-100.0) - 1.00 to 1.00 100.0% (100.0-100.0)
Stenosis Prevalence Kappa Percentage agreement 16.7% (13.2-19.9) n/a 99.3% (99.0-100.0) 15.0% (15.0-15.0) n/a 100.0% (100.0-100.0) - n/a 99.7% (99.0-100.0)
Scoliosis Prevalence Kappa Percentage agreement 5.0% (5.0-5.0) n/a 100.0% (100.0-100.0) 5.0% (5.0-5.0) n/a 100.0% (100.0-100.0) - n/a 100.0% (100.0-100.0)
Osteophytes Prevalence Kappa Percentage agreement 0.0% (0.0-0.0) n/a 100.0% (100.0-100.0) 0.0% (0.0-0.0) n/a 100.0% (100.0-100.0) - n/a 100.0% (100.0-100.0)
Facet joint arthrosis Prevalence Kappa Percentage agreement 15.0% (15.0-15.0) 1.00 (0.82-1.00) 100.0% (100.0-100.0) 15.0% (15.0-15.0) 0.92 (0.74-1.00) 98.7% (98.0-100.0) - 0.88 to 1.00 99.3% (98.0-100.0)
Endplate irregularities Prevalence Kappa Percentage agreement 5.0% (5.0-5.0) n/a 100.0% (100.0-100.0) 1.7% (0.0-4.9) n/a 99.3% (99.0-100.0) - n/a 99.3% (99.0-100.0)
Red flags Prevalence Kappa Percentage agreement 0.0% (0.0-0.0) n/a 100.0% (100.0-100.0) 0.0% (0.0-0.0) n/a 100.0% (100.0-100.0) - n/a 100.0% (100.0-100.0)
High intensity zone Prevalence Kappa Percentage agreement (%) 15.0% (15.0-15.0) 1.00 (0.75-1.00) 100.0% (100.0-100.0) 15.0% (15.0-15.0) 1.00 (0.75-1.00) 100.0% (100.0-100.0) - n/a 100.0% (100.0-100.0)
  1. * The prevalence estimates are calculated on a 'whole person' level. That is, the presence of a particular pathology in a person at one or more vertebral levels was counted as a single instance.
  2. 95%CI = 95% confidence interval in brackets.
  3. n/a = kappa co-efficient not calculated. Calculation of kappa in STATA requires the prevalence of the pathology to be 10% to 90% at each and every vertebral level being compared. This restriction applies to the findings of all raters being compared.