Skip to main content

Table 3 Inter-rater reproducability reliability results for the second group of three trainee clinicians for 14 pathoanatomic categories across a single coding round, expressed as a Kappa co-efficient and percentage agreement with 95% confidence intervals (95% CI)

From: Inexperienced clinicians can extract pathoanatomic information from MRI narrative reports with high reproducibility for use in research/quality assurance

Pathoanatomic category Reliability index (95%CI) Inter-rater
Coding round 1 		Inter-rater
Coding round 2 		Test-retest
(intra-rater)
Intervertebral disc degeneration Prevalence*
Kappa
Percentage agreement 		90.0% (90.0-90.0)
0.99 (0.87-1.00)
99.3% (99.0-100.0) 		90.0% (90.0-90.0)
0.93 (0.82-1.00)
96.7% (95.0-99.0) 		-
0.88 to 1.00
97.3% (94.0-100.0)
Intervertebral disc bulge Prevalence
Kappa
Percentage agreement 		90.0% (90.0-90.0)
0.96 (0.84-1.00)
98.0% (97.0-99.0) 		85.0% (79.3-90.7)
0.93 (0.80-1.00)
96.7% (95.0-99.0) 		-
0.79 to 0.96
94.3% (91.0-97.0)
Intervertebral disc herniation Prevalence
Kappa
Percentage agreement 		58.3% (55.1-61.6)
0.94 (0.80-1.00)
98.7% (98.0-99.0) 		58.3% (55.1-61.6)
0.94 (0.80-1.00)
98.7% (98.0-100.0) 		-
0.86 to 0.91
97.3% (97.0-98.0)
Nerve root compromise Prevalence
Kappa
Percentage agreement 		53.3% (50.1-56.6)
0.76 (0.58-0.94)
96.0% (94.0-98.0) 		55.0% (49.3-60.7)
0.81 (0.63-0.99)
97.3% (96.0-98.0) 		-
0.61 to 0.86
95.3% (94.0-98.0)
Modic change type 1 Prevalence
Kappa
Percentage agreement 		20.0% (20.0-20.0)
1.00 (0.82-1.00)
100.0% (100.0-100.0) 		21.7% (18.4-24.9)
0.93 (0.75-1.00)
99.3% (99.0-100.0) 		-
0.89 to 1.00
99.7% (99.0-100.0)
Modic change type 2 Prevalence
Kappa
Percentage agreement 		10.0% (10.0-10.0)
1.00 (0.75-1.00)
100.0% (100.0-100.0) 		8.3% (5.1-11.6)
n/a
99.3% (99.0-100.0) 		-
1.00 to 1.00
99.7% (99.0-100.0)
Spondylolisthesis Prevalence
Kappa
Percentage agreement 		20.0% (20.0-20.0)
1.00 (0.75-1.00)
100.0% (100.0-100.0) 		20.0% (20.0-20.0)
1.00 (0.75-1.00)
100.0% (100.0-100.0) 		-
1.00 to 1.00
100.0% (100.0-100.0)
Stenosis Prevalence
Kappa
Percentage agreement 		16.7% (13.2-19.9)
n/a
99.3% (99.0-100.0) 		15.0% (15.0-15.0)
n/a
100.0% (100.0-100.0) 		-
n/a
99.7% (99.0-100.0)
Scoliosis Prevalence
Kappa
Percentage agreement 		5.0% (5.0-5.0)
n/a
100.0% (100.0-100.0) 		5.0% (5.0-5.0)
n/a
100.0% (100.0-100.0) 		-
n/a
100.0% (100.0-100.0)
Osteophytes Prevalence
Kappa
Percentage agreement 		0.0% (0.0-0.0)
n/a
100.0% (100.0-100.0) 		0.0% (0.0-0.0)
n/a
100.0% (100.0-100.0) 		-
n/a
100.0% (100.0-100.0)
Facet joint arthrosis Prevalence
Kappa
Percentage agreement 		15.0% (15.0-15.0)
1.00 (0.82-1.00)
100.0% (100.0-100.0) 		15.0% (15.0-15.0)
0.92 (0.74-1.00)
98.7% (98.0-100.0) 		-
0.88 to 1.00
99.3% (98.0-100.0)
Endplate irregularities Prevalence
Kappa
Percentage agreement 		5.0% (5.0-5.0)
n/a
100.0% (100.0-100.0) 		1.7% (0.0-4.9)
n/a
99.3% (99.0-100.0) 		-
n/a
99.3% (99.0-100.0)
Red flags Prevalence
Kappa
Percentage agreement 		0.0% (0.0-0.0)
n/a
100.0% (100.0-100.0) 		0.0% (0.0-0.0)
n/a
100.0% (100.0-100.0) 		-
n/a
100.0% (100.0-100.0)
High intensity zone Prevalence
Kappa
Percentage agreement (%) 		15.0% (15.0-15.0)
1.00 (0.75-1.00)
100.0% (100.0-100.0) 		15.0% (15.0-15.0)
1.00 (0.75-1.00)
100.0% (100.0-100.0) 		-
n/a
100.0% (100.0-100.0)
  1. * The prevalence estimates are calculated on a 'whole person' level. That is, the presence of a particular pathology in a person at one or more vertebral levels was counted as a single instance.
  2. 95%CI = 95% confidence interval in brackets.
  3. n/a = kappa co-efficient not calculated. Calculation of kappa in STATA requires the prevalence of the pathology to be 10% to 90% at each and every vertebral level being compared. This restriction applies to the findings of all raters being compared.
Back to article page

Chiropractic & Manual Therapies

ISSN: 2045-709X