Skip to main content

Table 7 Quality items and scores of four studies using neuroimaging outcome measures included in a systematic review on the effect of spinal manipulation on ‘brain function’

From: Unravelling functional neurology: does spinal manipulation have an effect on the brain? - a systematic literature review

1st Author
Yr of publication
-Were study subjects in sham controlled studies reported to be blind? (Yes / No / Unclear)
-If yes / unclear, was the blinding tested for success? (Yes / No)
-If yes, was it successful? (Yes / No)
-Were study subjects in studies with control group reported to be naive? (Yes / No / Unclear)
-Was the origin of the subjects reported (Yes / No)
-If yes, does it allow to exclude any interest? (Yes / No / Unclear)
Were study subjects reported to have been randomly allocated to study groups? (Yes / No / Unclear) Were study groups comparable in relation to symptoms when studying symptomatic subjects (duration and pain intensity) (NA when crossover study design)? (Yes / No) Were the intervention and control(s) well described (at least where and how)? (Yes / No) Was the assessor reported to be blind to group allocation? (Yes / No) Were losses and exclusions of study subjects reported or obvious in result section (including in tables or graphs)? (Yes / No / Unclear) Was the person who statistically analyzed the data reported to be blind to group allocation? (Yes / No) Comments by the technical experts (i) on the statistical analysis, and (ii) in relation to the methodology and/or technical aspects
Quality score (risk of bias, also including an external validity criteria) and classification
-Unclear (recruited on the campus of Tohoku University)
       1: The extent the threshold for the voxel cluster size was defined as “10 to 50 voxels minimum”. The purpose of this varying threshold is unclear.
3: Lenient statistical threshold: Z = 3, extent threshold; 10 voxels.
2/6 (33%) low NA = No 0 pt Unclear (“counterbalanced”) 0.5 pt NA (cross-over) -No 0 pt
-Yes 0.5 pt
No 0 pt Yes 1 pt No 0 pt
       1: The phrasing “(e.g., 10 voxels minimum)” suggests again (see the comment in relation to Ogura 2011) that this threshold was not fixed.
2/6 (33%) low NA = No 0 pt Yes 1 pt NA (cross-over) -Yes 0.5 pt
-Yes 0.5 pt
No 0 pt No 0 pt No 0 pt
-Unclear (recruited from the campus of the University of Florida and UF Health Hospital and the local community)
-Authors “corrected for the number of separate RM-ANOVAs conducted across the 120 ROI-to-ROI pairs by using a p value less than .01 as significant.” (p.618). This threshold (p = 0.01) correction for multiple comparisons is not conservative enough.
-There was neither between-groups statistical test at “pre”, nor at “post”.
3: Lenient statistical threshold: p = 0.01 with 120 comparisons.
5/7 (71%)
NA = No 0 pt Yes 1 pt Yes 1 pt Yes 0.5 pt
-Yes 0.5 pt
Yes 1 pt Yes 1 pt No 0 pt
        1: The authors used an alpha = 0.01 threshold for the fMRI analysis. It is not conservative enough in my opinion (as discussed by Eklund et al. 2015, and Lieberman & Cunningham 2009).
-Unclear whether statistical threshold applied across the whole brain or just for the region of interest.
-It is unclear how the region of interest was defined
5.5/7 (79%) acceptable = Unclear 0.5 pt NA Yes 1 pt Yes 1 pt -Yes 0.5 pt
-Yes 0.5 pt
Yes 1 pt Yes 1 pt No 0 pt
  1. fMRI functional magnetic resonance imaging, NA Not applicable