- Open Access
Developing key performance indicators for the Canadian chiropractic profession: a modified Delphi study
Chiropractic & Manual Therapies volume 30, Article number: 31 (2022)
The purpose of this study is to develop a list of performance indicators to assess the status of the chiropractic profession in Canada.
We conducted a 4-round modified Delphi technique (March 2018–January 2020) to reach consensus among experts and stakeholders on key status indicators for the chiropractic profession using online questionnaires. During the first round, experts suggested indicators for preidentified themes. Through the following two rounds, the importance and feasibility of each indicator was rated on an 11-point Likert scale, and their related potential sources of data identified. In the final round, provincial stakeholders were recruited to rate the importance of the indicators within the 90th percentile and identified those most important to their organisation.
The first round generated 307 preliminary indicators of which 42 were selected for the remaining rounds, and eleven were preferentially selected by most of the provincial stakeholders. Experts agreed the feasibility of all indicators was high, and that data could be collected through a combination of data obtained from professional liability insurance records and survey(s) of the general population, patients, and chiropractors.
A set of performance indicators to assess the status of the Canadian chiropractic profession emerged from a scientific and stakeholder consensus.
Measuring performance is an important part of improvement in healthcare . Reporting performance indicators helps to monitor function of a system, promote public accountability, and inform change management [1, 2]. Emerging evidence suggests that public reporting of performance indicators may stimulate providers to improve healthcare quality , although it is unclear whether these changes result in improved patient outcomes [4, 5].
In Canada, chiropractic is one of the most frequently sought nonphysician provider groups . The Canadian annual utilisation of chiropractic services slightly increased from 10 to 11.7% between 1980 and 2015 . The majority of chiropractors practice within private clinics , and since the delisting of chiropractic services in some provinces , administrative insurance billing data from most Canadian provincial health insurance plans are not available. Since most patients pay directly for their services , other public insurance plans like workers compensation boards and veteran affairs represent a marginal part of chiropractic activities . Consequently, there are few quality or performance indicators that can be derived from Canadian public medico-administrative databases available to the chiropractic profession.
In response to this limitation in databases, the Canadian Chiropractic Association (CCA) began periodic surveying of their members in order to establish a “statistical portrait of chiropractors to be used in planning, evaluation and policy development” . Unfortunately, the survey has been mainly used for administrative purposes and is not typically provided to the profession or the public. The survey is time-consuming (81 questions), and its response rate has steadily decreased from 70% in 1996 to 39% in 2011. Furthermore, the data is not easily accessible, and sparingly used in research projects .
Recent work suggests important knowledge-to-practice gaps in the care provided by chiropractors for musculoskeletal disorders [11, 12]. Public reporting of chiropractic care performance indicators could be an important component of clinical practice guideline (CPG) implementation  through a learning health system (LHS)  within the chiropractic profession. LHS is defined as a dynamic health ecosystem that synergistically aligns various dimensions of health care delivery and routinized cycles of continuing learning and improvement . However, there is a need for consensus-based and credible indicators to evaluate the status of the chiropractic profession in Canada. A national, scientific, and widely endorsed list of indicators would be an important first step toward a rigorous and credible evaluation of the status of the Canadian chiropractic profession. The purpose of this study was to develop a list of indicators that could be used to assess the status of the chiropractic profession in Canada.
We used a four round modified Delphi design  to identify the best indicators to assess the status of the chiropractic profession in Canada. In the first three rounds, we asked national chiropractic experts to rate each identified indicator by their importance, measurement feasibility, and best source of data acquisition. We conducted a fourth round involving key provincial and national association stakeholders to identify the most important indicators from their perspective. All responses were voluntary. Participants provided informed consent digitally before completing our questionnaires. Research ethics approval (#1802X03) was obtained from the Canadian Memorial Chiropractic College (CMCC).
Participants were purposefully sampled from Canadian chiropractors. We defined “experts” as a Canadian chiropractor with a postgraduate degree in research (MSc and/or PhD). These criteria were used to ensure that the participants had sufficient knowledge of the Canadian chiropractic profession to identify relevant performance indicators. We invited stakeholders, representatives from each provincial and national chiropractic associations, provincial regulatory body and academic teaching institution. Eligible participants were identified by screening websites of relevant Canadian chiropractic organizations (provincial and national associations, foundations, regulatory bodies, and educational institutions). We also used snowball sampling . Invitations to complete the survey were sent by email, with up to three weekly reminders for each round.
Data were collected for each of the first three rounds using online questionnaires distributed via SurveyMonkey® (www.surveymonkey.com, San Mateo, CA, USA) and administered by the Office of Research, CMCC. For the fourth round, feedback was obtained from a web-based survey tool developed by the Université du Québec à Trois-Rivières (UQTR) (https://confluence.uqtr.ca/display/AOPSP/BIQ).
Experts were provided ten open-ended questions and asked to identify potentially relevant indicators related to assessment of: (1) quality of care, (2) financial status, (3) use of chiropractic care, (4) inter-professional collaboration, (5) education of chiropractors, (6) research, (7) public perception, (8) legal status, (9) adverse events, and (10) chiropractor caseloads. Experts were also able to provide a list of relevant indicators that in their opinion, may not have been adequately captured by those listed in the survey. Two researchers (MAB and SM) independently analysed and converted responders’ answers from the first round into a preliminary list of 307 indicators that were subsequently grouped into nine main themes (Additional file 1). They then met with the full team to discuss and reach consensus on the approved themes to be distributed in Round 2.
Experts who completed the first round were invited to rank the importance of each preliminary list of identified indicators on an 11-point Likert rating scale from 0 (strongly disagree) to 10 (strongly agree). They were also asked to provide recommendations for any additional indicators not captured in Round 1, and suggest revisions to any of the proposed indicators. Indicators with median or mean importance scores in the 90th percentile were advanced to Round 3. To ensure that relevant aspects of the profession were measured, the indicators with the highest mean importance score for the themes that had less than two indicators in the 90th percentile were selected by the research team.
Experts who completed the first two rounds were invited to complete the third round. The mean and median importance scores obtained for each indicator at the end of Round 2 were included. Each expert evaluated the importance and feasibility of each indicator on an 11-point Likert rating scale and also identified the best source for the acquisition of data for each indicator from a list of ten suggested sources e.g., survey of: (1) chiropractic patients, (2) Canadian population, (3) chiropractors, (4) medical doctors; or data from: (5) provincial colleges of chiropractors, (6) Canadian Chiropractic Protective Association (CCPA) main Canadian chiropractic malpractice carrier, (7) private insurers, (8) provincial public health plan, (9) legal decision database, 10) CCA. The experts were also able to add non prespecified sources of data. All the indicators assessed during Round 3 were entered into the fourth round.
To consider the perspective of relevant stakeholders on the most important indicators, we invited the Executive Officers of the national and provincial chiropractic associations, regulatory boards, CCPA and Canadian Federation of Chiropractic, as well as representatives of chiropractic academic teaching institutions (CMCC and the UQTR). Each stakeholder was asked to rank the importance of each indicator from Round 3 on an 11-point Likert rating scale, and select the 15 most important indicators from the perspective of their organization/institution.
Descriptive statistics (mean, standard deviation, median, quartiles, mode) were used to assess the importance, feasibility and best source of data of the indicators. We used the independent samples Mann–Whitney U test to compare the importance score obtained during the third round and fourth round. All comparisons were considered statistically significant at p < 0.05 level. All analyses were performed using SPSS for Mac (version 126.96.36.199, IBM Corporation, Armonk, NY, USA).
We identified and invited 131 experts of whom 53 (40.5%) completed the first, 45 (84.9%) the second and 32 (71.1%) the third rounds. We invited 19 stakeholders to complete the fourth round, 10 (52.6%) completed the online survey but only 7 identified the most important indicators from the perspective of their organisation.
Rounds 1 and 2
Round 1 produced 307 potential indicators that were grouped into 9 main themes and 32 subthemes (Additional file 1). The mean importance score for each indicator identified in Round 2 ranged between 3.1 and 9.2, with a mean of 7.5 (standard deviation (SD) of 0.9) (Additional file 1). Taking the 90th percentile of the mean and median importance score, we identified 34 indicators. Despite not being in the 90th percentile, the following 8 indicators were selected because they were rated as the two highest mean importance scores in their respective themes:
Proportion of family health teams (academic or non-academic) including a chiropractor
Proportion of medical doctors with positive attitude towards chiropractors/chiropractic services
Number of hours spent on diagnosis
Average cost per patient paid by Insurance coverage for chiropractic services
Proportion of chiropractors that are satisfied with their job
Proportion of chiropractors involved in multidisciplinary research
Proportion of chiropractic researchers who conduct clinical research
Legislated scope of practice in every province
The theme interprofessional collaboration had three indicators because proportion of family health teams (academic or non-academic) including a chiropractor and proportion of medical doctors with positive attitude towards chiropractors/chiropractic services had the same importance score. After Round 2, the indicator proportion of family health teams (academic or non-academic) including a chiropractor was reformulated prior to the third round to proportion of family or community health teams (academic or non-academic) including a chiropractor.
In the Round 3, the mean importance score ranged between 7.9 to 9.9, with a mean of 9.1 (SD = 0.5) (Table 1). The mean feasibility score ranged between 6.3 to 9.6 with a mean of 8.0 (SD = 0.8) (Table 1). The most common data sources reported to capture the indicators were: survey of chiropractors (31.0%), survey of chiropractic patients (26.2%), survey of the Canadian population (9.5%) and data from the CCPA (9.5%). After Round 3, the indicator proportion of family or community health teams (academic or non-academic) including a chiropractor was reformulated prior to the fourth round to proportion of multidisciplinary medical clinics (e.g., family health teams, health teams etc.) academic or non-academic that include a chiropractor.
During the fourth round, the mean importance score ranged between 6.5 to 9.3 with a mean of 8.2 (SD = 0.8) (Table 2). Most of the importance scores were not significantly different from Round 3, except for nine indicators that were ranked significantly lower. Seven of the ten stakeholders that responded in Round 4 ranked their 15 most important indicators. The following eleven indicators were selected by more than 50% of the stakeholders:
Proportion of chiropractic patients with neck pain
Proportion of chiropractic patients with back pain
Proportion of chiropractic patients with headache
Proportion of chiropractic patients with chronic conditions
Proportion of chiropractors providing evidence-based care
Proportion of chiropractors delivering patient-centred care
Proportion of chiropractic patients who receive an appropriate physical exam
Proportion of chiropractic patients who experience a significant functional improvement
Proportion of the population that perceives chiropractors as credible healthcare providers
Proportion of the population who trust the chiropractic profession
Proportion of chiropractic patients who experienced severe adverse events
Our findings support a recently described theoretically based integrated framework for healthcare performance assessment . The indicators most frequently selected by our stakeholders reflect the framework’s measurable constructs of patients’ needs (proportion of chiropractic patients with headache, chronic conditions, neck and back pain) and expectations (proportion of the population that trust the chiropractic profession and that perceives chiropractors as credible healthcare providers); receipt and experience of healthcare (proportion of chiropractic patients who receive an appropriate physical exam, proportion of chiropractors providing evidence-based care, and patient-centred care); and healthcare outcomes (proportion of chiropractic patients who experience a significant functional improvement, and severe adverse events). Consistent with the framework, the combination of these indicators could provide insight on accessibility, appropriateness, effectiveness, and safety of chiropractic care . Our experts considered these indicators to be of high importance and feasible to collect using a combination of professional liability insurance records and surveys of the general population, patients, and chiropractors. The responding stakeholders confirmed the high importance of most of these indicators. They also reflect the contemporary emphasis on quality of care [11, 12], patient-centred care , safety , and public legitimacy within the chiropractic profession.
To our knowledge this is the first study to identify key performance indicators to assess the status of the Canadian chiropractic profession. Others have identified multiple quality indicators for the management of musculoskeletal disorders in emergency departments . However, these indicators may not be directly applicable as they involve care pathways and pharmacological treatment not commonly used in chiropractic care. Sorensen et al. have developed disease-specific quality indicators for Danish chiropractic patients with low back pain . Their focus was narrower than ours, but their work suggests that it is feasible to measure performance using indicators obtained by surveying chiropractors. More recently, Dutch physiotherapists have measured a core set of healthcare outcomes from routinely collected clinical patient data (patient reported outcomes measures [PROMs]) [22, 23]. Wide implementation of this core set appears promising since the stakeholders involved in collecting these outcomes realized they added value to their clinical practice .
Since previous studies have demonstrated that it is feasible to derive performance indicators from both patient and practitioner surveys [13, 22], we argue in favor of regular public reporting of performance indicators for the chiropractic profession in Canada. Developing an adequate infrastructure for data collection will be challenging given the multiple sources of information required. Considering that provincial and national chiropractic organizations (associations and regulatory boards) regularly survey their members and the population, this data provides an opportunity to optimize and harmonize resources, and contribute to performance metrics. However, collecting survey data to inform performance indicators will require optimizing survey methods to ensure a sufficient response rate to mitigate potential bias . Both the CCA and the Canadian Chiropractic Federation, by nature of their national positions, can be strategic players in rallying provincial organizational participation. Chiropractic regulatory boards periodically audit their members regarding their quality of care within the context of their mandate of public protection. This might be an appropriate context to collect data on the receipt and experience of healthcare. Electronic health records might also facilitate the collection of chiropractors’ and patients’ data [25,26,27].
Negative unintended consequences have been reported following the public reporting of performance indicators [28, 29]; however, the potential for quality improvement is considered to outweigh the risk [1, 3]. Public reporting of performance indicators is uncommon among chiropractic organizations and could potentially increase their transparency and accountability while facilitating the implementation of a LHS.
Among the strengths of our study is the participation from experts and national stakeholders across Canada. Most of the suggested indicators were identified as important by our experts, which may lead to an overabundance of information (“indicator chaos”) . Our study has limitations, such that the stakeholders’ perception of importance of indicators omitted two constructs (healthcare resources and structures, and healthcare processes, functions and context) that would have been highlighted through a theory-based selection . Moreover, the indicators identified in relation to the quality of care require further development to be adequately operationalized [13, 31]. Although our response rate is common among healthcare provider surveys, it is not optimal and suggests that mobilizing stakeholders toward the measurement and public reporting of indicators may be challenging.
We present a set of performance indicators for the Canadian chiropractic profession developed from a consensus of scientific experts and stakeholders. This set of indicators constitute a promising basis for the assessment of the chiropractic profession’s status. Further development regarding the measurement quality of the indicators, the supporting infrastructure, and the stakeholder’s engagement will be necessary prior to implementation.
Availability of data and materials
The datasets generated and analyzed during the current study are not publicly available due privacy restriction but are available from the corresponding author on reasonable request.
Canadian Chiropractic Association
Canadian Chiropractic Protective Association
Canadian Memorial Chiropractic College
Clinical Practice Guidelines
Learning Health System
Patient reported outcome measures
Université du Québec à Trois-Rivières
Levesque J-F, Sutherland K. What role does performance information play in securing improvement in healthcare? A conceptual framework for levers of change. BMJ Open. 2017;7(8): e014825.
Papanicolas I, Smith P. Health system performance comparison: an agenda for policy, information and research: an agenda for policy, information and research. London: McGraw-Hill Education; 2013.
Campanella P, Vukovic V, Parente P, Sulejmani A, Ricciardi W, Specchia ML. The impact of public reporting on clinical outcomes: a systematic review and meta-analysis. BMC Health Serv Res. 2016;16(1):1–14.
Metcalfe D, Rios Diaz AJ, Olufajo OA, Massa MS, Ketelaar N, Flottorp SA, Perry DC. Impact of public release of performance data on the behaviour of healthcare consumers and providers. Cochrane Database Syst Rev. 2018(9).
Arah OA, Westert GP. Correlates of health and healthcare performance: applying the Canadian health indicators framework at the provincial-territorial level. BMC Health Serv Res. 2005;5(1):1–13.
Beliveau PJH, Wong JJ, Sutton DA, Simon NB, Bussières AE, Mior SA, French SD. The chiropractic profession: a scoping review of utilization rates, reasons for seeking care, patient profiles, and care provided. Chiropr Man Ther. 2017;25(1):35.
Blanchette M-A, Rivard M, Dionne C, Cassidy JD. Chiropractors’ characteristics associated with physician referrals: results from a survey of Canadian Chiropractors. J Manipulative Physiol Ther. 2015;38(6):395–406.
Sorbara G. The 2004 Ontario budget. In.: Toronto: Government of Ontario [available online]; 2004.
Kopansky-Giles D, Papadopoulos C. Canadian Chiropractic Resources Databank (CCRD): a profile of Canadian chiropractors. J Can Chiropr Assoc. 1997;41(3):155–91.
Blanchette M-A, Cassidy JD, Rivard M, Dionne C. Chiropractors’ characteristics associated with their number of workers’ compensation patients. J Can Chiropr Assoc. 2015;59(3):202–15.
Brockhusen SS, Bussieres A, French SD, Christensen HW, Jensen TS. Managing patients with acute and chronic non-specific neck pain: are Danish chiropractors compliant with guidelines? Chiropr Man Therap. 2017;25:17.
Bussieres AE, Al Zoubi F, Stuber K, French SD, Boruff J, Corrigan J, Thomas A. Evidence-based practice, research utilization, and knowledge translation in chiropractic: a scoping review. BMC Complement Altern Med. 2016;16:216.
Sorensen LP, Krog BR, Kongsted A, Bronfort G, Hartvigsen J. Development of disease-specific quality indicators for Danish chiropractic patients with low back pain. J Manipulative Physiol Ther. 2011;34(4):204–10.
Menear M, Blanchette M-A, Demers-Payette O, Roy D. A framework for value-creating learning health systems. Health Res Policy Syst. 2019;17(1):79.
Diamond IR, Grant RC, Feldman BM, Pencharz PB, Ling SC, Moore AM, Wales PW. Defining consensus: a systematic review recommends methodologic criteria for reporting of Delphi studies. J Clin Epidemiol. 2014;67(4):401–9.
Johnson TP. Snowball sampling: introduction. Wiley StatsRef: Statistics Reference Online 2014.
Levesque J-F, Sutherland K. Combining patient, clinical and system perspectives in assessing performance in healthcare: an integrated measurement framework. BMC Health Serv Res. 2020;20(1):23–23.
WFC Strategic Plan 2019–2022 In. Toronto ON: World Federation of Chiropractic; 2019.
Funabashi M, Pohlman KA, Mior S, O’Beirne M, Westaway M, De Carvalho D, El-Bayoumi M, Haig B, Wade DJ, Thiel HW. SafetyNET Community-based patient safety initiatives: development and application of a Patient Safety and Quality Improvement Survey. J Can Chiropr Assoc. 2018;62(3):130.
Weeks WB, Goertz CM, Meeker WC, Marchiori DM. Public Perceptions of doctors of chiropractic: results of a national survey and examination of variation according to respondents' likelihood to use chiropractic, experience with chiropractic, and chiropractic supply in local health care markets. J Manipulative Physiol Ther. 2015.
Strudwick K, Nelson M, Martin-Khan M, Bourke M, Bell A, Russell T. Quality indicators for musculoskeletal injury management in the emergency department: a systematic review. Acad Emerg Med. 2015;22(2):127–41.
Verburg AC, van Dulmen SA, Kiers H, Nijhuis-van der Sanden MWG, van der Wees PJ. Patient-reported outcome-based quality indicators in Dutch primary care physical therapy for patients with nonspecific low back pain: a cohort study. Physical Therapy. 2021.
Verburg A, Van Dulmen S, Kiers H, Nijhuis-Van Der Sanden M, Van Der Wees P. Development of a standard set of outcome measures for non-specific low back pain in Dutch primary care physiotherapy practices: a Delphi study. Eur Spine J. 2019;28(7):1550–64.
Edwards PJ, Roberts I, Clarke MJ, DiGuiseppi C, Wentz R, Kwan I, Cooper R, Felix LM, Pratap S. Methods to increase response to postal and electronic questionnaires. Cochrane Database Syst Rev. 2009(3).
Meerhoff GA, van Dulmen SA, Maas MJ, Heijblom K, Nijhuis-van der Sanden MW, Van der Wees PJ. Development and evaluation of an implementation strategy for collecting data in a national registry and the use of patient-reported outcome measures in physical therapist practices: quality improvement study. Phys Ther. 2017;97(8):837–51.
Beaulieu MD, Cloutier AM, Dault R, Éthier JF, Goetghebeur M, Khanji C, Roy D, Vanasse A. Utilisation des données cliniques issues des dossiers médicaux électroniques à des fins de recherche et d’amélioration continue de la qualité des soins et services de première ligne. In: Quebec: Institut national d’excellence en santé et en services sociaux (INESSS). 2018; 52.
Taylor DN. A literature review of electronic health records in chiropractic practice: common challenges and solutions. J Chiropr Humanit. 2017;24(1):31–40.
Heath I, Hippisley-Cox J, Smeeth L. Measuring performance and missing the point? BMJ. 2007;335(7629):1075–6.
Forster AJ, van Walraven C. The use of quality indicators to promote accountability in health care: the good, the bad, and the ugly. Open Med. 2012;6(2): e75.
McGinnis JM, Malphrus E, Blumenthal D. Vital signs: core metrics for health and health care progress. 2015.
Tugwell P, Knottnerus JA. When is a health-care quality indicator ready to use? J Clin Epidemiol. 2014;67(9):961–2.
The authors would like to thank Stéphanie Wouters for her help with the data collection. This study received no specific funding. The article processing charges were covered by the authors respective institutions namely Université du Québec à Trois-Rivières (UQTR) and Canadian Memorial Chiropractic College (CMCC).
This study received no specific funding. The article processing charges were covered by the authors respective institutions namely Université du Québec à Trois-Rivières (UQTR) and Canadian Memorial Chiropractic College (CMCC).
Ethics approval and consent to participate
All responses were voluntary. Participants provided informed consent digitally before completing our questionnaires. Research ethics approval (#1802X03) was obtained from the Canadian Memorial Chiropractic College (CMCC).
Consent for publication
The authors have no conflicts of interest to report.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Blanchette, MA., Mior, S., Thistle, S. et al. Developing key performance indicators for the Canadian chiropractic profession: a modified Delphi study. Chiropr Man Therap 30, 31 (2022). https://doi.org/10.1186/s12998-022-00439-z
- Performance assessment
- Profession: allied health
- Quality assessment
- Quality of care
- Delphi study