Of the approximatively 13.000 articles initially identified as potentially suitable for our review only 13 were included. Although their authors usually avoided to state explicitly that they were studying effect of treatment, they would imply somehow in their text that this was the case, which earned them a place in our review. A wide variety of diseases were examined but, according to the only two studies with a suitable research design to answer our research questions, there was no effect or benefit of chiropractic care in the prevention/early treatment of high blood pressure and no extra benefit was found for manipulative therapy on dysfunctional breathing. In sum, we could not find any evidence in favour of the argument that manipulative therapy/chiropractic care can prevent or stop early disease.
General methodological considerations
Although at first look there appears to be a literature on this subject, it is apparent that most authors lack knowledge in research methodology. The two methodologically acceptable studies in our review were found in PubMed, whereas most of the others were identified in the non-indexed literature. We therefore conclude that it may not be worthwhile in the future to search extensively the non-indexed chiropractic literature for high quality research articles.
One misunderstanding requires some explanations; case reports are usually not considered suitable evidence for effect of treatment, even if the cases relate to patients who ‘recovered’ with treatment. The reasons for this are multiple, such as:
Individual cases, usually picked out on the basis of their uniqueness, do not reflect general patterns.
Individual successful cases, even if correctly interpreted must be validated in a ‘proper’ research design, which usually means that presumed effect must be tested in a properly powered and designed randomized controlled trial.
One or two successful cases may reflect a true but very unusual recovery, and such cases are more likely to be written up and published as clinicians do not take the time to marvel over and spend time on writing and publishing all the other unsuccessful treatment attempts.
Recovery may be co-incidental, caused by some other aspect in the patient’s life or it may simply reflect the natural course of the disease, such as natural remission or the regression towards the mean, which in human physiology means that low values tend to increase and high values decrease over time.
Cases are usually captured at the end because the results indicate success, meaning that the clinical file has to be reconstructed, because tests were used for clinical reasons and not for research reasons (i.e. recorded by the treating clinician during an ordinary clinical session) and therefore usually not objective and reproducible.
The presumed results of the treatment of the disease is communicated from the patient to the treating clinician and not to a third, neutral person and obviously this link is not blinded, so the clinician is both biased in favour of his own treatment and aware of which treatment was given, and so is the patient, which may result in overly positive reporting. The patient wants to please the sympathetic clinician and the clinician is proud of his own work and overestimates the results.
The long-term effects are usually not known.
Further, and most importantly, there is no control group, so it is impossible to compare the results to an untreated or otherwise treated person or group of persons.
Nevertheless, it is common to see case reports in some research journals and in communities with readers/practitioners without a firmly established research culture it is often considered a good thing to ‘start’ by publishing case reports.
Case reports are useful for other reasons, such as indicating the need for further clinical studies in a specific patient population, describing a clinical presentation or treatment approach, explaining particular procedures, discussing cases, and referring to the evidence behind a clinical process, but they should not be used to make people believe that there is an effect of treatment. In fact, there are ‘rules’ for how to deal with case reports, such as the CARE guidelines by Gagnier et al. .
All clinical studies but two suffered from serious methodological problems.
The main problem was that five out of the eight prospective outcome studies did not have a control group. Clearly, in order to find out if a treatment has an effect, a comparison to no treatment must be made or a comparison to another treatment that is known to have an effect. Further, this ‘no treatment’ group must be masked into a sham treatment, to allow for the placebo effect that probably always plays a role in clinical practice.
Interestingly, only two of the five prospective studies without a control group mentioned this as a problem. Nevertheless, instead of discussing this lack of control group, the authors of three articles mentioned that there would be a need for larger studies. However, larger studies will not remedy this fundamental flaw in the study design.
When comparing outcomes between different types of treatment approach, the sham group is not relevant but the study subjects should not have a preference for one type of treatment or the other. Therefore, it is difficult to perform such studies on chiropractors, chiropractic patients and chiropractic students, as study participants should be ‘naïve’. To account for expectation bias, study subjects’ preferences should be elicited prior to the start of the study and taken into account during the analysis.
When establishing effect or benefit of treatment, it is also necessary that the study subjects are captured at about the same period of time, as the disease, the treatment and study subjects may change over time. It is usually not a good idea to simply compare one type of treatment with the results of another type of treatment carried out x number of years ago or at the same time in some other clinic. The reasons why the study subjects should be captured in the same place is that they should be fairly representative of that patient group, and different countries, areas of a country, clinics and clinicians may attract fundamentally different types of patients with inherently different prognoses.
The allocation into one study group or another should be done in a random fashion, in such a way that nobody can guide different patient types into a specific group because they seem more suitable in that group. Random allocation usually avoids clustering of certain patient types in one group, which may have an effect on treatment outcome if these groups react differently to treatment.
Other important aspects are that the person who assesses the outcome should not be the person who treats the patient and should also be blind to which study group the person assessed belongs to. The outcome variables should be objective and relevant in relation to measuring whether the disease improved or not. Further, tests should be reliable when carried out by different examiners and also consistent (reproducible) within the study subject if the test is carried out over several times, to ensure that any changes occurring over time are due to the treatment and not to the instability of the test or inability of the tester.
Theoretically, it would be possible in population-based studies to compare patterns of disease and mortality in relation to various factors, such as the density of various health practitioners in the population. This could be done in epidemiologic studies of randomly selected people from the general population, ecological studies in which people live through a so-called ‘natural experiments’, and data bases from the health insurance industry or those holding socio-economic, morbidity and mortality data.
However, to observe the ‘effect’ of chiropractic care, through a comparison of an area with access to chiropractic care versus an area with less or no such access, as was the attempt in the five reviewed population studies, it would be necessary to take into account factors that might incite chiropractors to settle in a specific area. In poorer areas there would be fewer chiropractors because of the difficulty to run a successful practice but poor people are also sicker and die younger than more financially comfortable people [30, 31]. Therefore, increased morbidity or mortality that seems to be linked with the number of chiropractors would instead depend on other more fundamental factors. None of the five register studies included in this review took into account, properly, the moderating or confounding influence that such variables could have on their initial results. They simply reported the links between chiropractor density and various other predictive factors vs. disease or mortality in independently reported analyses, such as comparing the number of deaths in relation to a) the mean age of the study sample, b) the type of social class in the area, c) the proportion of chiropractors vs. general practitioners but did not combine these using appropriate statistical methods.
In comparison, we reviewed a population-based study on the elderly population in N. America  but could not include it because the outcome was established through questionnaires, not through objective measurements. This study collected a large number of variables at baseline on community dwelling persons who went through a clinical trial. They then compared health outcomes for those who consulted chiropractors and those who did not, and found that there seemed to be a better health profile for the chiropractic subgroup. However, when they statistically controlled on a number of base-line variables which indicated that the two groups were somewhat different (in relation to age, strenuous physical activity, health status, and arthritis status), the difference between the chiropractic group and the other disappeared. The explanation for this is that these additional factors were associated to both the choice of practitioner and the health outcome. However, they were the reasons for health and disease, not the health practitioner. Therefore, the link between the use of chiropractors and better health was only an apparent one. This example is given to illustrate the importance of including relevant ‘competing’ factors when looking at cause-effect in population studies.
Methodological consideration of the review process
The screening of so many titles may result in errors due to fatigue but it was done blindly by two of the authors to avoid mistakes and it was never necessary to consult the third author. In relation to the journal Functional Neurology, Rehabilitation and Ergonomics, in which all published articles were screened, this was done by only one of the authors but it was done blindly at two separate occasions. Although some studies could not be found, it is unlikely that they would have brought any positive evidence for chiropractic care and PP or early secondary prevention, as they were published in the ‘non-indexed’ literature in PubMed.
We designed our own checklists to meet the specific needs of this review. It was not considered appropriate to employ the Cochrane checklist, for example, as preliminary readings of some of this literature indicated that the quality problems would become apparent with a much less sophisticated scrutiny. Another research team would probably have designed a somewhat different list of items. However, this would undoubtedly have identified the glaring methodological problems apparent in most of this literature. Our checklists were easily completed, and the third reviewer was, again, not required to act as a referee, indicating the checklists were user-friendly. The results were interpreted in a narrative way, no meta-analysis has been done, because this was not appropriate.
We reviewed only 13 relevant articles but there is, in fact, a literature on the experimental effects of spinal manipulation in relation to various physiological outcome variables. These were not included, as we were interested in the clinical picture only. Also, the review approach to such articles would have to be different and should therefore be done in separate reviews.
The need for evidence
For groups of chiropractors, prevention of disease through chiropractic treatment makes perfect sense, yet the credible literature is void of evidence thereof. Still, the majority of chiropractors practising this way probably believe that there is plenty of evidence in the literature. Clearly, if the chiropractic profession wishes to maintain credibility, it is time seriously to face this issue. Presently, there seems to be no reason why political associations and educational institutions should recommend spinal care to prevent disease in general, unless relevant and acceptable research evidence can be produced to support such activities. In order to be allowed to continue this practice, proper and relevant research is therefore needed. However, such activities need to be guided by some fundamental concepts, as discussed below.
First, the concept of biological plausibility
In order to proceed to a research study, there must be a credible anatomical, physiological, and/or biological rationale for the link between the treatment and the PP or early secondary prevention of the disease under scrutiny.
Second, the concept of quality of research
In order to show effect of intervention, properly conducted randomized controlled trials should be carried out, as described above. This usually requires the participation of independent and professional researchers and they are costly and therefore require funding. Further, it is unethical to conduct poor quality studies because: they inconvenience subjects on studies with no consequences, they are a waste of money that could have been used on quality projects, and they can be misleading for both chiropractors and their patients. High quality, honest studies evoke admiration and acceptance in scientific and health care environments and will have a good effect on the chiropractic profession, for everybody to enjoy, regardless if the results are ‘positive’ or ‘negative’.
The concept of the three pillars of evidence based practice
The three pillars of evidence-based medicine are often described as (i) the scientific evidence, (ii) the practitioner’s experience, (iii) and the patient’s preference. However, the practitioner’s experience is not objective (please see the description of the problems with case reports above), in particular in relation to ‘effect’ of treatment.
It is therefore not enough to say that ‘it works’. The clinical experience is important in many other ways but not for judging effect of treatment. Therefore, as it has been stated before , “it is important to keep a humble attitude to one’s own clinical experience and not to think that it overrides the evidence obtained in a good quality RCT”.
The need for educating chiropractors on how to read and evaluate research
All chiropractors who want to update their knowledge or to have an evidence-based practice will search new information on the internet. If they are not trained to read the scientific literature, they might trust any article. In this situation, it is logical that the ‘believers’ will choose ‘attractive’ articles and trust the results, without checking the quality of the studies. It is therefore important to educate chiropractors to become relatively competent consumers of research, so they will not assume that every published article is a verity in itself.
Prevention in chiropractic practice
The desire to improve health in general for patients  indicates a common idealistic streak in large groups of the chiropractic profession. In relation to illness in general, all clinicians have a duty to inform and assist patients to avoid preventable disorders, and chiropractors can provide inspiration in this area and also monitor lifestyle changes, as back pain is a recurring disorder that often results in long term clinical relationships. However, prevention of disease through spinal adjustment is, until further notice, futile.