Inexperienced clinicians can extract pathoanatomic information from MRI narrative reports with high reproducibility for use in research/quality assurance

Peter Kent; Andrew M Briggs; Hanne B Albert; Andreas Byrhagen; Christian Hansen; Karina Kjaergaard; Tue S Jensen

doi:10.1186/2045-709X-19-16

Table 3 Inter-rater reproducability reliability results for the second group of three trainee clinicians for 14 pathoanatomic categories across a single coding round, expressed as a Kappa co-efficient and percentage agreement with 95% confidence intervals (95% CI)

From: Inexperienced clinicians can extract pathoanatomic information from MRI narrative reports with high reproducibility for use in research/quality assurance

Pathoanatomic category	Reliability index (95%CI)	Inter-rater Coding round 1	Inter-rater Coding round 2	Test-retest (intra-rater)
Intervertebral disc degeneration	Prevalence* Kappa Percentage agreement	90.0% (90.0-90.0) 0.99 (0.87-1.00) 99.3% (99.0-100.0)	90.0% (90.0-90.0) 0.93 (0.82-1.00) 96.7% (95.0-99.0)	- 0.88 to 1.00 97.3% (94.0-100.0)
Intervertebral disc bulge	Prevalence Kappa Percentage agreement	90.0% (90.0-90.0) 0.96 (0.84-1.00) 98.0% (97.0-99.0)	85.0% (79.3-90.7) 0.93 (0.80-1.00) 96.7% (95.0-99.0)	- 0.79 to 0.96 94.3% (91.0-97.0)
Intervertebral disc herniation	Prevalence Kappa Percentage agreement	58.3% (55.1-61.6) 0.94 (0.80-1.00) 98.7% (98.0-99.0)	58.3% (55.1-61.6) 0.94 (0.80-1.00) 98.7% (98.0-100.0)	- 0.86 to 0.91 97.3% (97.0-98.0)
Nerve root compromise	Prevalence Kappa Percentage agreement	53.3% (50.1-56.6) 0.76 (0.58-0.94) 96.0% (94.0-98.0)	55.0% (49.3-60.7) 0.81 (0.63-0.99) 97.3% (96.0-98.0)	- 0.61 to 0.86 95.3% (94.0-98.0)
Modic change type 1	Prevalence Kappa Percentage agreement	20.0% (20.0-20.0) 1.00 (0.82-1.00) 100.0% (100.0-100.0)	21.7% (18.4-24.9) 0.93 (0.75-1.00) 99.3% (99.0-100.0)	- 0.89 to 1.00 99.7% (99.0-100.0)
Modic change type 2	Prevalence Kappa Percentage agreement	10.0% (10.0-10.0) 1.00 (0.75-1.00) 100.0% (100.0-100.0)	8.3% (5.1-11.6) n/a 99.3% (99.0-100.0)	- 1.00 to 1.00 99.7% (99.0-100.0)
Spondylolisthesis	Prevalence Kappa Percentage agreement	20.0% (20.0-20.0) 1.00 (0.75-1.00) 100.0% (100.0-100.0)	20.0% (20.0-20.0) 1.00 (0.75-1.00) 100.0% (100.0-100.0)	- 1.00 to 1.00 100.0% (100.0-100.0)
Stenosis	Prevalence Kappa Percentage agreement	16.7% (13.2-19.9) n/a 99.3% (99.0-100.0)	15.0% (15.0-15.0) n/a 100.0% (100.0-100.0)	- n/a 99.7% (99.0-100.0)
Scoliosis	Prevalence Kappa Percentage agreement	5.0% (5.0-5.0) n/a 100.0% (100.0-100.0)	5.0% (5.0-5.0) n/a 100.0% (100.0-100.0)	- n/a 100.0% (100.0-100.0)
Osteophytes	Prevalence Kappa Percentage agreement	0.0% (0.0-0.0) n/a 100.0% (100.0-100.0)	0.0% (0.0-0.0) n/a 100.0% (100.0-100.0)	- n/a 100.0% (100.0-100.0)
Facet joint arthrosis	Prevalence Kappa Percentage agreement	15.0% (15.0-15.0) 1.00 (0.82-1.00) 100.0% (100.0-100.0)	15.0% (15.0-15.0) 0.92 (0.74-1.00) 98.7% (98.0-100.0)	- 0.88 to 1.00 99.3% (98.0-100.0)
Endplate irregularities	Prevalence Kappa Percentage agreement	5.0% (5.0-5.0) n/a 100.0% (100.0-100.0)	1.7% (0.0-4.9) n/a 99.3% (99.0-100.0)	- n/a 99.3% (99.0-100.0)
Red flags	Prevalence Kappa Percentage agreement	0.0% (0.0-0.0) n/a 100.0% (100.0-100.0)	0.0% (0.0-0.0) n/a 100.0% (100.0-100.0)	- n/a 100.0% (100.0-100.0)
High intensity zone	Prevalence Kappa Percentage agreement (%)	15.0% (15.0-15.0) 1.00 (0.75-1.00) 100.0% (100.0-100.0)	15.0% (15.0-15.0) 1.00 (0.75-1.00) 100.0% (100.0-100.0)	- n/a 100.0% (100.0-100.0)

* The prevalence estimates are calculated on a 'whole person' level. That is, the presence of a particular pathology in a person at one or more vertebral levels was counted as a single instance.
95%CI = 95% confidence interval in brackets.
n/a = kappa co-efficient not calculated. Calculation of kappa in STATA requires the prevalence of the pathology to be 10% to 90% at each and every vertebral level being compared. This restriction applies to the findings of all raters being compared.

Back to article page