Skip to main content

Response to Lawrence DJ: the global summit on the efficacy and effectiveness of spinal manipulative therapy for the prevention and treatment of non-musculoskeletal disorders: a systematic review of the literature

Thank you for the opportunity to respond to the Letter to the Editor by Dana J. Lawrence. In his letter, Lawrence states that the results of our systematic review may be due to bias. However, he does not adequately substantiate his claims.

First, Lawrence perceived that we “brushed away” the possibility of reviewer bias. This comment is surprising because, as described in the paper, we carefully planned for possible reviewer bias and took several methodological steps to minimize its potential impact. Specifically, we: 1) developed an instruction manual for critical appraisal that we distributed to all reviewers prior to the Global Summit; 2) used the Scottish Intercollegiate Guidelines Network (SIGN) criteria, a standardized critical appraisal tool, to assess the risk of bias of randomized controlled trials; 3) adapted the language of the SIGN tool to minimize the risk of misinterpretation of the critical appraisal items; 4) provided detailed notes that accompanied the SIGN checklist, and edited to match the purpose of this review; 5) conducted three rounds of independent reviews to ensure that the risk of bias assessment was consistent across reviewers; 6) undertook a quality control assessment by two independent reviewers, who reviewed all critical appraisal checklists and risk of bias ratings; and 7) invited all Global Summit participants to independently review and vote on the final risk of bias assessment results; the risk of bias table was approved by 98.0% (49/50) of participants. Therefore, the position advanced by Lawrence that we “brushed away” the possibility of reviewer bias is not justified.

Second, Lawrence states that publication bias may be responsible for our findings. He wrote: “… publication bias tilted toward positive, not negative, findings have been demonstrated even in Cochrane reviews themselves.” and he uses the study by Kicinski et al. to support his assertion [1]. This indicates that Lawrence misinterpreted the concept of publication bias in this instance. In their study of systematic reviews included in the Cochrane database, Kicinski et al. reported that: “In the meta-analyses of efficacy, outcomes favoring treatment had on average a 27% (95% Credible Interval (CI): 18% to 36%) higher probability to be included than other outcomes.” [1] We refer Lawrence to our discussion where we clearly stated: “Finally, publication bias may be present in this field of research. However, it is unlikely that publication bias compromised the validity of our results because studies most unlikely to be published are those that failed to obtain a ‘positive’ result.” In other words, we are in agreement with Kicinski et al. and disagree with Lawrence’s conclusion. In fact, our position is consistent with a large body of methodological literature which clearly indicates that a trial is more likely to be published if the results favor a specific intervention (positive results) compared to a trial which does not support an effect of an intervention (negative results) [1,2,3,4,5,6,7,8,9,10,11,12,13]. We would like to remind Lawrence that we identified and critically appraised studies regardless if they had “negative” or “positive” results. However, the methodological quality of “positive” trials was low and their results could not be used to inform our synthesis.

Third, Lawrence indicated that clinicians did not have a voice in the project. We respectfully disagree with this assertion. Several participants to the Global Summit maintain an active practice and most participants have had long and successful clinical careers.

In summary, the statements made by Lawrence about our methodology are incorrect and ill informed. While we thank Lawrence for his interest in our work, his statements about our methodology are in our opinion misconstrued and hence not appropriate.

Availability of data and materials

Not applicable.

References

  1. 1.

    Kicinski M, Springate DA, Kontopantelis E. Publication bias in meta-analyses from the Cochrane database of systematic reviews. Stat Med. 2015;34(20):2781–93. https://doi.org/10.1002/sim.6525.

    Article  PubMed  Google Scholar 

  2. 2.

    Turner EH, Matthews AM, Linardatos E, Tell RA, Rosenthal R. Selective publication of antidepressant trials and its influence on apparent efficacy. N Engl J Med. 2008;358(3):252–60. https://doi.org/10.1056/NEJMsa065779.

    CAS  Article  PubMed  Google Scholar 

  3. 3.

    Lee K, Bacchetti P, Sim I. Publication of clinical trials supporting successful new drug applications: a literature analysis. PLoS Med. 2008;5(9):e191. https://doi.org/10.1371/journal.pmed.0050191.

    Article  PubMed  PubMed Central  Google Scholar 

  4. 4.

    Decullier E, Lheritier V, Chapuis F. Fate of biomedical research protocols and publication bias in France: retrospectivecohort study. Br Med J. 2005;331(7507):19–22. https://doi.org/10.1136/bmj.38488.385995.8F.

    Article  Google Scholar 

  5. 5.

    Dickersin K, Min YI, Meinert CL. Factors influencing publication of research results. Follow-up of applications submitted to two institutional review boards. J Am Med Assoc. 1992;267(3):374–8. https://doi.org/10.1001/jama.1992.03480030052036.

    CAS  Article  Google Scholar 

  6. 6.

    Stern JM, Simes RJ. Publication bias: evidence of delayed publication in a cohort study of clinical research projects. Br Med J. 1997;315(7109):640–5. https://doi.org/10.1136/bmj.315.7109.640.

    CAS  Article  Google Scholar 

  7. 7.

    Chan A. Empirical evidence for selective reporting of outcomes in randomized trials: comparison of protocols to published articles. J Am Med Assoc. 2004;291(20):2457–65. https://doi.org/10.1001/jama.291.20.2457.

    CAS  Article  Google Scholar 

  8. 8.

    Chan AW, Altman DG. Identifying outcome reporting bias in randomised trials on PubMed: review of publications and survey of authors. Br Med J. 2005;330(7494):753. https://doi.org/10.1136/bmj.38356.424606.8F.

    Article  Google Scholar 

  9. 9.

    Rising K, Bacchetti P, Bero L. Reporting bias in drug trials submitted to the Food and Drug Administration: review of publication and presentation. PLoS Med. 2008;5(11):e217, discussion e217. https://doi.org/10.1371/journal.pmed.0050217.

    Article  PubMed  PubMed Central  Google Scholar 

  10. 10.

    Riveros C, Dechartres A, Perrodeau E, Haneef R, Boutron I, Ravaud P. Timing and completeness of trial results posted at ClinicalTrials.gov and published in journals. PLoS Med. 2013;10(12):e1001566.

    Article  Google Scholar 

  11. 11.

    Dwan K, Altman DG, Clarke M, Gamble C, Higgins JP, Sterne JA, et al. Evidence for the selective reporting of analyses and discrepancies in clinical trials: a systematic review of cohort studies of clinical trials. PLoS Med. 2014;11(6):e1001666.

    Article  Google Scholar 

  12. 12.

    Dwan K, Gamble C, Williamson PR, Kirkham JJ, Reporting Bias Group. Systematic review of the empirical evidence of study publication bias and outcome reporting bias - an updated review. PLoS One. 2013;8(7):e66844. https://doi.org/10.1371/journal.pone.0066844 PMID: 23861749; PMCID: PMC3702538.

    CAS  Article  PubMed  PubMed Central  Google Scholar 

  13. 13.

    Song F, Parekh S, Hooper L, Loke YK, Ryder J, Sutton AJ, et al. Dissemination and publication of research findings: an updated review of related biases. Health Technol Assess. 2010;14(8):1–193. https://doi.org/10.3310/hta14080 iii, ix-xi. PMID: 20181324.

    Article  Google Scholar 

Download references

Acknowledgements

not applicable.

Funding

No funding was obtained for this letter to the editor.

Author information

Affiliations

Authors

Contributions

Pierre Côté, Jan Hartvigsen, Iben Axen and Charlotte Leboeuf-Yde wrote the Letter to the editor. Melissa Corso, Heather Shearer,Jessica Wong, Andrée-Anne Marchand, J. David Cassidy, Simon French, Gregory N. Kawchuk, Silvano Mior, Erik Poulsen, John Srbely, Carlo Ammendolia, Marc-André Blanchette, Jason W. Busse, André Bussières, Carolina Cancelliere, Henrik Wulff Christensen, Diana De Carvalho, Katie De Luca, Alister Du Rose, Andreas Eklund, Roger Engel, Guillaume Goncalves, Jeffrey Hebert, Cesar A. Hincapié, Maria Hondras, Amanda Kimpton, Henrik Hein Lauridsen, Stanley Innes, Anne-Laure Meyer, David Newell, Søren O’Neill, Isabelle Pagé, Steven Passmore, Stephen M. Perle, Jeffrey Quon, Mana Rezai, Maja Stupar, Michael Swain, Andrew Vitiello, Kenneth Weber, Kenneth J. Young and Hainan Yu edited and proofread the commentary. All authors reviewed the letter to the editor and endorsed its content. The author(s) read and approved the final manuscript.

Corresponding author

Correspondence to Pierre Côté.

Ethics declarations

Ethics approval and consent to participate

Not applicable.

Consent for publication

Not applicable.

Competing interests

The following individuals had nothing to declare: CA, IA, JB, DDC, JDC, AE, RE, GG, CAH, MH, SI, AK, AAM, ALM, DN, ADR, HS, MS, AV, JJW. MA Blanchette reports research grants unrelated to this work from University of Quebec-Trois-Rivieres; fees medical-legal expertise from the Canadian Chiropractic Protective Association and Intact Assurance. A Bussières is an Editorial Board member of Chiropractic & Manual Therapies but had no part in the editorial or peer-review process. He reports receiving grants unrelated to this work from the Canadian Chiropractic Research Foundation. C Cancelliere is funded by the Canadian Chiropractic Research Foundation. HW Christensen has received grants unrelated to this work from The Danish Chiropractic Research Fund and my institution has paid for accommodation and travel expenses. P Côté reports research grants unrelated to this work from Aviva Canada, Canadian Institutes of Health Research-Canada Research Chair Program, Canadian Chiropractic Association, Canadian Chiropractic Research Foundation, College of Chiropractors of British Columbia, Et liv i bevegelse” (ELIB), French Chiropractic Association, Financial Services Commission of Ontario, Ontario Ministry of Finance, Ontario Trillium Foundation; travel expenditures unrelated to this work from Griffith University - Whiplash Symposium 2017, Eurospine, Southern Denmark University, Institut Franco-Europeen de Chiropraxie, Karolinska Institutet, North American Spine Society, University of Quebec-Trois-Rivieres, University of Zurich, World Federation of Chiropractic; fees for medical-legal expertise from the Canadian Chiropractic Protective Association. K Deluca reports she has multiple research grants from Australian funding agencies, unrelated to this work. She has received coverage of travel expenditures associated with the Global Summit. SD French is a Deputy Editor-in-Chief of Chiropractic & Manual Therapies but had no part in the editorial or peer-review process of this manuscript. He has received research grants unrelated to this work from Australian and international funding agencies and from chiropractic professional associations. J Hartvigsen reports that he holds multiple research grants from Danish and international funding agencies and charities. He has received coverage of travel expenditures from multiple sources internationally in connection with speaking engagements. Within the past year he has received speaking fees from Parker Seminars and Novartis. J Hébert receives salary support from the Canadian Chiropractic Research Foundation and the New Brunswick Health Research Foundation. He is also an Associate Editor of Chiropractic and Manual Therapies but played no part in the review of this submission and was blinded to the process. G Kawchuk reports active research grants unrelated to this work from The Natural Sciences and Engineering Research, The National Institutes of Health, The Alberta Spine Foundation, The American Orthotic and Prosthetic Association, The New Frontiers in Research Fund and the Canadian Chiropractic Research Foundation. Travel expenditures unrelated to this work in the past year include Kiropraktik i Sverige Live, Et liv i bevegelse” (ELIB), the Nordic Institute of Chiropractic and Clinical Biomechanics, The American Chiropractic Association, The National Institutes of Health, The British Columbia Chiropractic Association, and The World Federation of Chiropractic. Fees for medicallegal expertise unrelated to this work from the Canadian Chiropractic Protective Association. HH Lauridsen holds multiple research grants from funding agencies and charities of which the main contributor is the Danish Foundation for Chiropractic Research and Postgraduate Education. HHL is presently employed as associate professor and head of chiropractic studies at University of Southern Denmark. No funding has been received for the present project. C Leboeuf-Yde reports no competing interests as she obtains no funding from the chiropractic profession. S Mior reports receiving grants unrelated to this work from the Canadian Chiropractic Association, Ontario Chiropractic Association, and Canadian Chiropractic Research Foundation. S O’Neill reports funding by local, regional health authorities and by the Danish chiropractic foundation for research and postgraduate education, but not in relation specifically to this project I Pagé has received grants unrelated to this work. S Perle is an Associate Editor of Chiropractic & Manual Therapies but had no part in the editorial or peer-review process. He has received grants unrelated to this work from the West Hartford Group, Inc. and NCMIC Foundation, Inc. E Poulsen reports research grants from the Danish Foundation for Chiropractic Research and Postgraduate Education, which is funded by the Danish chiropractors, and the Danish Regions. No funding has been received for the present project. J Quon has funding unrelated to this work from the Canadian Chiropractic Guideline Initiative and The Vancouver Foundation. He is a member of the editorial board of Chiropractic & Manual Therapies but had no part in the editorial or peer-review process for this manuscript. M Rezai is a member of the World Federation of Chiropractic Research Disability and Rehabilitation Committee. JS is am active licensed member of Ontario and Canadian Chiropractic Associations. He is currently receiving research support from the Ontario Chiropractic Association. Previously received research support from Canadian Chiropractic Research Foundation and Ontario Chiropractic Association. M Swain holds research grants unrelated to this work. He received the Chiropractic Australia Research Foundation Travel Grant 2019 to cover travel expenses associated with the Global Summit. Within the past year he has received consultation fees unrelated to this work. He is a non-executive director of Chiropractic Australia. He is a governor of the Sydney College of Chiropractic. K Weber was supported by grants from the National Institute of Neurological Disorders and Stroke (K23NS104211 and L30NS108301) of the National Institutes of Health (Bethesda, Maryland, USA). The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health. K J Young received a Personal PhD Grant (2018) from the Royal College of Chiropractors (UK - £1600) that partially funded two papers unrelated to this work.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Côté, P., Hartvigsen, J., Axén, I. et al. Response to Lawrence DJ: the global summit on the efficacy and effectiveness of spinal manipulative therapy for the prevention and treatment of non-musculoskeletal disorders: a systematic review of the literature. Chiropr Man Therap 29, 26 (2021). https://doi.org/10.1186/s12998-021-00380-7

Download citation