Skip to main content

Advertisement

Springer Nature is making Coronavirus research free. View research | View latest news | Sign up for updates

Table 1 Inter-rater reproducability results for the initial group of three trainee clinicians for 14 pathoanatomic categories across the four coding rounds, expressed as a Kappa co-efficient and percentage agreement with 95% confidence intervals (95% CI)

From: Inexperienced clinicians can extract pathoanatomic information from MRI narrative reports with high reproducibility for use in research/quality assurance

Pathoanatomic category Reliability index (95%CI) Coding round 1 Coding round 2 Coding round 3 Coding round 4
Intervertebral disc degeneration Prevalence*
Kappa
Percentage agreement
71.7% (50.2-93.1)
0.66 (0.54-0.77)
80.8% (73.8-87.9)
94.0 (92.0-96.0)
0.94 (0.84-1.00)
95.8 (92.3-99.4)
80.0 (80.0-80.0)
n/a
98.2 (95.8-100.0)
83.3 (80.1-86.6)
0.91 (0.80-1.00)
97.5 (94.7-100.0)
Intervertebral disc bulge Prevalence
Kappa
Percentage agreement
73.3% (70.1-76.6)
0.91 (0.78-1.00)
95.0% (94.7-100.0)
87.0 (83.1-90.9)
0.86 (0.76-0.90)
92.5 (87.7-97.2)
70.0 (70.0-70.0)
1.00 (0.87-1.00)
100.0
48.3 (45.1-51.6)
0.89 (0.76-1.00)
99.2 (97.5-100.0)
Intervertebral disc herniation Prevalence
Kappa
Percentage agreement
40.0% (34.3-45.7)
0.91 (0.73-1.00)
97.5% (94.7-100.0)
61.0 (59.0-63.0)
1.00 (0.85-1.00)
100.0
40.0 (40.0-40.0)
0.98 (0.87-1.00)
96.5 (93.1-99.9)
45.0 (45.0-45.0)
1.00 (0.82-1.00)
100.0
Nerve root compromise Prevalence
Kappa
Percentage agreement
13.3% (4.7-22.0)
n/a
97.5% (94.7-100.0)
42.0 (31.9-52.1)
0.74 (0.49-1.00)
94.2 (90.0-98.4)
31.7 (23.0-40.3)
0.63 (0.45-0.81)
93.0 (88.3-97.7)
18.3 (15.1-21.6)
1.00 (0.75-1.00)
99.2 (97.5-100.0)
Modic change type 1 Prevalence
Kappa
Percentage agreement
26.7% (23.4-29.9)
0.86 (0.68-1.00)
95.0% (91.1-98.1)
18.0 (15.6-20.4)
0.86 (0.60-1.00)
98.3 (96.0-100.0)
20.0 (20.0-20.0)
1.00 (0.82-1.00)
100.0
18.3 (15.1-21.6)
0.86 (0.60-1.00)
100.0
Modic change type 2 Prevalence
Kappa
Percentage agreement
25.0% (19.3-30.7)
0.76 (0.51-1.00)
97.5% (94.7-100.0)
10.0 (5.6-14.4)
n/a
98.3 (96.0-100.0)
20.0 (20.0-20.0)
1.00 (0.82-1.00)
100.0
5.0 (5.0-5.0)
n/a
100.0
Spondylolisthesis Prevalence
Kappa
Percentage agreement
20.0% (20.0-20.0)
n/a
97.5% (94.7-100.0)
19.0 (17.0-21.0)
1.00 (0.75-1.00)
100.0
10.0 (10.0-10.0)
1.00 (0.82-1.00)
100.0
0.0 (0.0-0.0)
n/a
100.0
Stenosis Prevalence
Kappa
Percentage agreement
6.7% (3.4-9.9)
n/a
97.5% (94.7-100.0)
23.0 (20.6-25.4)
0.84 (0.59-1.00)
99.2 (97.5-100.0)
25.0 (25.0-25.0)
0.95 (0.74-1.00)
99.1 (97.4-100.0)
0.0 (0.0-0.0)
n/a
100.0
Scoliosis Prevalence
Kappa
Percentage agreement
25.0% (25.0-25.0)
1.00 (0.82-1.00)
100.0%
6.0 (4.0-8.0)
n/a
95.0 (91.1-98.9)
5.0 (5.0-5.0)
n/a
99.1(97.4-100.0)
5.0 (5.0-5.0)
n/a
100.0
Osteophytes Prevalence
Kappa
Percentage agreement
5.0% (5.0-5.0)
n/a
100.0%
1.0 (0.0-3.0)
n/a
100.0
10.0 (10.0-10.0)
n/a
99.1 (97.4-100.0)
10.0 (10.0-10.0)
1.00 (0.75-1.00)
99.2 (97.5-100.0)
Facet joint arthrosis Prevalence
Kappa
Percentage agreement
0.0% (0.0)
n/a
100.0%
14.0 (12.0-16.0)
0.68 (0.42-0.93)
95.0 (91.1-98.9)
20.0 (20.0-20.0)
1.00 (0.82-1.00)
100.0
5.0 (5.0-5.0)
n/a
100.0
Endplate irregularities Prevalence
Kappa
Percentage agreement
# 1.0 (0.0-3.0)
n/a
99.2 (97.5-100.0)
25.0 (19.3-30.7)
1.00 (0.82-1.00)
98.2 (95.8-100.0)
18.3 (11.8-24.9)
0.74 (0.56-0.92)
97.5 (94.7-100.0)
Red flags Prevalence (%)
Kappa
Percentage agreement
# 0.0 (0.0-0.0)
n/a
100.0
8.3 (1.8-14.9)
n/a
98.2 (95.8-100.0)
0.0 (0.0-0.0)
n/a
100.0
High intensity zone Prevalence (%)
Kappa
Percentage agreement
# 15.0 (11.9-18.1)
0.84 (0.59-1.00)
99.2 (97.5-100.0)
26.7 (20.1-33.2)
0.85 (0.67-1.00)
98.2 (95.8-100.0)
25.0 (25.0-25.0)
1.00 (0.82-1.00)
100.0
  1. * The prevalence estimates are calculated on a 'whole person' level. That is, the presence of a particular pathology in a person at one or more vertebral levels was counted as a single instance.
  2. 95%CI = 95% confidence interval in brackets.
  3. # not included in coding round 1.
  4. n/a = kappa co-efficient not calculated. Calculation of kappa in STATA requires the prevalence of the pathology to be 10% to 90% at each and every vertebral level being compared. This restriction applies to the findings of all raters being compared.