Response to Lawrence DJ: the global summit on the efficacy and effectiveness of spinal manipulative therapy for the prevention and treatment of non-musculoskeletal disorders: a systematic review of the literature
Chiropractic & Manual Therapies volume 29, Article number: 26 (2021)
Thank you for the opportunity to respond to the Letter to the Editor by Dana J. Lawrence. In his letter, Lawrence states that the results of our systematic review may be due to bias. However, he does not adequately substantiate his claims.
First, Lawrence perceived that we “brushed away” the possibility of reviewer bias. This comment is surprising because, as described in the paper, we carefully planned for possible reviewer bias and took several methodological steps to minimize its potential impact. Specifically, we: 1) developed an instruction manual for critical appraisal that we distributed to all reviewers prior to the Global Summit; 2) used the Scottish Intercollegiate Guidelines Network (SIGN) criteria, a standardized critical appraisal tool, to assess the risk of bias of randomized controlled trials; 3) adapted the language of the SIGN tool to minimize the risk of misinterpretation of the critical appraisal items; 4) provided detailed notes that accompanied the SIGN checklist, and edited to match the purpose of this review; 5) conducted three rounds of independent reviews to ensure that the risk of bias assessment was consistent across reviewers; 6) undertook a quality control assessment by two independent reviewers, who reviewed all critical appraisal checklists and risk of bias ratings; and 7) invited all Global Summit participants to independently review and vote on the final risk of bias assessment results; the risk of bias table was approved by 98.0% (49/50) of participants. Therefore, the position advanced by Lawrence that we “brushed away” the possibility of reviewer bias is not justified.
Second, Lawrence states that publication bias may be responsible for our findings. He wrote: “… publication bias tilted toward positive, not negative, findings have been demonstrated even in Cochrane reviews themselves.” and he uses the study by Kicinski et al. to support his assertion . This indicates that Lawrence misinterpreted the concept of publication bias in this instance. In their study of systematic reviews included in the Cochrane database, Kicinski et al. reported that: “In the meta-analyses of efficacy, outcomes favoring treatment had on average a 27% (95% Credible Interval (CI): 18% to 36%) higher probability to be included than other outcomes.”  We refer Lawrence to our discussion where we clearly stated: “Finally, publication bias may be present in this field of research. However, it is unlikely that publication bias compromised the validity of our results because studies most unlikely to be published are those that failed to obtain a ‘positive’ result.” In other words, we are in agreement with Kicinski et al. and disagree with Lawrence’s conclusion. In fact, our position is consistent with a large body of methodological literature which clearly indicates that a trial is more likely to be published if the results favor a specific intervention (positive results) compared to a trial which does not support an effect of an intervention (negative results) [1,2,3,4,5,6,7,8,9,10,11,12,13]. We would like to remind Lawrence that we identified and critically appraised studies regardless if they had “negative” or “positive” results. However, the methodological quality of “positive” trials was low and their results could not be used to inform our synthesis.
Third, Lawrence indicated that clinicians did not have a voice in the project. We respectfully disagree with this assertion. Several participants to the Global Summit maintain an active practice and most participants have had long and successful clinical careers.
In summary, the statements made by Lawrence about our methodology are incorrect and ill informed. While we thank Lawrence for his interest in our work, his statements about our methodology are in our opinion misconstrued and hence not appropriate.
Availability of data and materials
Kicinski M, Springate DA, Kontopantelis E. Publication bias in meta-analyses from the Cochrane database of systematic reviews. Stat Med. 2015;34(20):2781–93. https://doi.org/10.1002/sim.6525.
Turner EH, Matthews AM, Linardatos E, Tell RA, Rosenthal R. Selective publication of antidepressant trials and its influence on apparent efficacy. N Engl J Med. 2008;358(3):252–60. https://doi.org/10.1056/NEJMsa065779.
Lee K, Bacchetti P, Sim I. Publication of clinical trials supporting successful new drug applications: a literature analysis. PLoS Med. 2008;5(9):e191. https://doi.org/10.1371/journal.pmed.0050191.
Decullier E, Lheritier V, Chapuis F. Fate of biomedical research protocols and publication bias in France: retrospectivecohort study. Br Med J. 2005;331(7507):19–22. https://doi.org/10.1136/bmj.38488.385995.8F.
Dickersin K, Min YI, Meinert CL. Factors influencing publication of research results. Follow-up of applications submitted to two institutional review boards. J Am Med Assoc. 1992;267(3):374–8. https://doi.org/10.1001/jama.1992.03480030052036.
Stern JM, Simes RJ. Publication bias: evidence of delayed publication in a cohort study of clinical research projects. Br Med J. 1997;315(7109):640–5. https://doi.org/10.1136/bmj.315.7109.640.
Chan A. Empirical evidence for selective reporting of outcomes in randomized trials: comparison of protocols to published articles. J Am Med Assoc. 2004;291(20):2457–65. https://doi.org/10.1001/jama.291.20.2457.
Chan AW, Altman DG. Identifying outcome reporting bias in randomised trials on PubMed: review of publications and survey of authors. Br Med J. 2005;330(7494):753. https://doi.org/10.1136/bmj.38356.424606.8F.
Rising K, Bacchetti P, Bero L. Reporting bias in drug trials submitted to the Food and Drug Administration: review of publication and presentation. PLoS Med. 2008;5(11):e217, discussion e217. https://doi.org/10.1371/journal.pmed.0050217.
Riveros C, Dechartres A, Perrodeau E, Haneef R, Boutron I, Ravaud P. Timing and completeness of trial results posted at ClinicalTrials.gov and published in journals. PLoS Med. 2013;10(12):e1001566.
Dwan K, Altman DG, Clarke M, Gamble C, Higgins JP, Sterne JA, et al. Evidence for the selective reporting of analyses and discrepancies in clinical trials: a systematic review of cohort studies of clinical trials. PLoS Med. 2014;11(6):e1001666.
Dwan K, Gamble C, Williamson PR, Kirkham JJ, Reporting Bias Group. Systematic review of the empirical evidence of study publication bias and outcome reporting bias - an updated review. PLoS One. 2013;8(7):e66844. https://doi.org/10.1371/journal.pone.0066844 PMID: 23861749; PMCID: PMC3702538.
Song F, Parekh S, Hooper L, Loke YK, Ryder J, Sutton AJ, et al. Dissemination and publication of research findings: an updated review of related biases. Health Technol Assess. 2010;14(8):1–193. https://doi.org/10.3310/hta14080 iii, ix-xi. PMID: 20181324.
No funding was obtained for this letter to the editor.
Ethics approval and consent to participate
Consent for publication
The following individuals had nothing to declare: CA, IA, JB, DDC, JDC, AE, RE, GG, CAH, MH, SI, AK, AAM, ALM, DN, ADR, HS, MS, AV, JJW. MA Blanchette reports research grants unrelated to this work from University of Quebec-Trois-Rivieres; fees medical-legal expertise from the Canadian Chiropractic Protective Association and Intact Assurance. A Bussières is an Editorial Board member of Chiropractic & Manual Therapies but had no part in the editorial or peer-review process. He reports receiving grants unrelated to this work from the Canadian Chiropractic Research Foundation. C Cancelliere is funded by the Canadian Chiropractic Research Foundation. HW Christensen has received grants unrelated to this work from The Danish Chiropractic Research Fund and my institution has paid for accommodation and travel expenses. P Côté reports research grants unrelated to this work from Aviva Canada, Canadian Institutes of Health Research-Canada Research Chair Program, Canadian Chiropractic Association, Canadian Chiropractic Research Foundation, College of Chiropractors of British Columbia, Et liv i bevegelse” (ELIB), French Chiropractic Association, Financial Services Commission of Ontario, Ontario Ministry of Finance, Ontario Trillium Foundation; travel expenditures unrelated to this work from Griffith University - Whiplash Symposium 2017, Eurospine, Southern Denmark University, Institut Franco-Europeen de Chiropraxie, Karolinska Institutet, North American Spine Society, University of Quebec-Trois-Rivieres, University of Zurich, World Federation of Chiropractic; fees for medical-legal expertise from the Canadian Chiropractic Protective Association. K Deluca reports she has multiple research grants from Australian funding agencies, unrelated to this work. She has received coverage of travel expenditures associated with the Global Summit. SD French is a Deputy Editor-in-Chief of Chiropractic & Manual Therapies but had no part in the editorial or peer-review process of this manuscript. He has received research grants unrelated to this work from Australian and international funding agencies and from chiropractic professional associations. J Hartvigsen reports that he holds multiple research grants from Danish and international funding agencies and charities. He has received coverage of travel expenditures from multiple sources internationally in connection with speaking engagements. Within the past year he has received speaking fees from Parker Seminars and Novartis. J Hébert receives salary support from the Canadian Chiropractic Research Foundation and the New Brunswick Health Research Foundation. He is also an Associate Editor of Chiropractic and Manual Therapies but played no part in the review of this submission and was blinded to the process. G Kawchuk reports active research grants unrelated to this work from The Natural Sciences and Engineering Research, The National Institutes of Health, The Alberta Spine Foundation, The American Orthotic and Prosthetic Association, The New Frontiers in Research Fund and the Canadian Chiropractic Research Foundation. Travel expenditures unrelated to this work in the past year include Kiropraktik i Sverige Live, Et liv i bevegelse” (ELIB), the Nordic Institute of Chiropractic and Clinical Biomechanics, The American Chiropractic Association, The National Institutes of Health, The British Columbia Chiropractic Association, and The World Federation of Chiropractic. Fees for medicallegal expertise unrelated to this work from the Canadian Chiropractic Protective Association. HH Lauridsen holds multiple research grants from funding agencies and charities of which the main contributor is the Danish Foundation for Chiropractic Research and Postgraduate Education. HHL is presently employed as associate professor and head of chiropractic studies at University of Southern Denmark. No funding has been received for the present project. C Leboeuf-Yde reports no competing interests as she obtains no funding from the chiropractic profession. S Mior reports receiving grants unrelated to this work from the Canadian Chiropractic Association, Ontario Chiropractic Association, and Canadian Chiropractic Research Foundation. S O’Neill reports funding by local, regional health authorities and by the Danish chiropractic foundation for research and postgraduate education, but not in relation specifically to this project I Pagé has received grants unrelated to this work. S Perle is an Associate Editor of Chiropractic & Manual Therapies but had no part in the editorial or peer-review process. He has received grants unrelated to this work from the West Hartford Group, Inc. and NCMIC Foundation, Inc. E Poulsen reports research grants from the Danish Foundation for Chiropractic Research and Postgraduate Education, which is funded by the Danish chiropractors, and the Danish Regions. No funding has been received for the present project. J Quon has funding unrelated to this work from the Canadian Chiropractic Guideline Initiative and The Vancouver Foundation. He is a member of the editorial board of Chiropractic & Manual Therapies but had no part in the editorial or peer-review process for this manuscript. M Rezai is a member of the World Federation of Chiropractic Research Disability and Rehabilitation Committee. JS is am active licensed member of Ontario and Canadian Chiropractic Associations. He is currently receiving research support from the Ontario Chiropractic Association. Previously received research support from Canadian Chiropractic Research Foundation and Ontario Chiropractic Association. M Swain holds research grants unrelated to this work. He received the Chiropractic Australia Research Foundation Travel Grant 2019 to cover travel expenses associated with the Global Summit. Within the past year he has received consultation fees unrelated to this work. He is a non-executive director of Chiropractic Australia. He is a governor of the Sydney College of Chiropractic. K Weber was supported by grants from the National Institute of Neurological Disorders and Stroke (K23NS104211 and L30NS108301) of the National Institutes of Health (Bethesda, Maryland, USA). The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health. K J Young received a Personal PhD Grant (2018) from the Royal College of Chiropractors (UK - £1600) that partially funded two papers unrelated to this work.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Côté, P., Hartvigsen, J., Axén, I. et al. Response to Lawrence DJ: the global summit on the efficacy and effectiveness of spinal manipulative therapy for the prevention and treatment of non-musculoskeletal disorders: a systematic review of the literature. Chiropr Man Therap 29, 26 (2021). https://doi.org/10.1186/s12998-021-00380-7