Data extraction

Two authors (MM and CLY) extracted data from each article, independently of each other, into two check-lists, one descriptive and one qualitative. Data were later compared in order to minimize reading errors. The other authors were designated as judges in case of disagreement between the first two reviewers.

Descriptive check-list

Quality checklist

Classifying articles by their quality

For the selected articles, all descriptive and quality items were checked independently by the first two reviewers and a summary quality score was assigned. One point was awarded for each fulfilled quality item.

If the quality item was not satisfied (“no”) or if the article did not mention this condition (”?”), no point was given. However, if the item was not applicable (“N/A”), this was not counted in the denominator. Thus the maximum total score could be different from one article to the next. Finally, the total score was normalized to a 100 point scale in order to compare quality across studies.

Although a summary quality score was calculated, no cutoff-point was defined for acceptable or unacceptable level of quality. This allowed us (and the readers) to use the quality scores and the information on each individual quality item as a guide to determine whether articles would be considered more or less credible. In other words, the quality assessment was meant to be informative rather than proscriptive. Because we decided not to use a scoring system to establish levels of quality, we have not defined any discrete levels of evidence, e.g. high, moderate or low.

Data synthesis

Four data tables were constructed according to the area of the spine where treatment was provided and ROM assessed: 1. cervical treatment and ROM, 2. lumbar treatment and ROM, 3. cervical treatment and range of mouth opening, and 4. sacroiliac treatment and hip ROM. The tables were scrutinized, one by one, to obtain answers to our research question. Results were thereafter interpreted and reported in a narrative fashion.

Study selection

Figure1 shows a flow diagram of the study selection process. A total of 242 articles were identified from the systematic search of MEDLINE, 51 from EMBASE, and 3 from the Cochrane library. Nine other articles were added from an additional hand search of reference lists. After duplicates were removed, 247 articles remained to be assessed for inclusion. Of these, 67 were retrieved in full and examined against our inclusion criteria. In all, 15 studies fulfilled all inclusion criteria and are reviewed herein (see Figure1).

Description of data

Table1 provides a full description of the key characteristics of each study. Articles have been listed chronologically based on the date of publication.

Of the 15 selected studies, two measured the effects of SMT on mouth opening, nine studied effects about the three cervical axes of movement, one measured only the range of cervical rotation, three studied flexion/extension in the lumbar spine, and one investigated the ROM of the hip. There were nine randomized controlled trials and six cross-over studies. All of them measured the ROM before and immediately after treatment. No papers on the thoracic spine satisfied the inclusion criteria.

ROM was measured differently depending upon the spinal region under study. Mouth opening ([34] and[35]) was measured with a universal caliper. Cervical ROM was assessed with a three- dimensional goniometer ([36],[37],[38] and[39]), a strap-on head goniometer ([40]), a postero-anterior assessment device with a head sensor ([41]), an inclinometer with a compass system ([42]), and a magnetic neck brace with three separate inclinometers ([43] and[44]). Lumbar ROMs were measured with a fingertip-to-floor inclinometer[45], a double inclinometer[46] and an electromagnetic tracking device[47]. Hip ROM was measured with a digital goniometer ([48]).

Some researchers (n=11) performed their experiments on people with symptoms, either patients (n=8) or volunteers with symptoms recruited by advertisements (n=3), whereas the remaining four studies dealt with healthy students.

Eight authors performed only one experiment, five reported two experiments, and two studies reported on three and four experiments, respectively.

Only one article did not report whether approval had been received from an ethics committee.

Quality

Table2 presents a description of the quality items. The quality scores ranged from 33/100 to 92/100, with a mean score of 71.4 (SD 10.4) and a median of 69. Articles are listed in the order of decreasing quality score. There were no disagreements between the reviewers in relation to the quality items.

All articles fulfilled the following four quality criteria: “Random allocation”, “SMT well described”, “Estimates reported and shown in tables/graphs” and “Results tested for statistical significance”. The item “Age distribution similar in studies with study sample consisting of people older than 45” was applicable to six studies ([36],[45],[37],[42],[34], and[38]) of which only[45] fulfilled the criterion.

Nine articles dealt with “disinterested/naïve study samples”. Ten studies included what we considered a credible sham treatment in comparison to the spinal manipulation. However, some studies investigated differences in outcome in relation to different types of manipulation (such as manipulating in different planes), i.e. not comparing manipulation to a sham treatment making this issue superfluous. Eleven authors specified that SMT was performed by an experienced person (all except[36, 39, 44, 48]), and twelve assured that pre/post assessment was made by the same person (all except[36, 39, 48]).

Outcome assessments were blinded in thirteen studies, and thirteen assessed ROMs with a validated method. In two of the studies, ([48] and[39]), the validity of the assessment method was not described.

In none of the studies were the experimental conditions stated to have been equal for treatment and control groups or before and after the manipulation. Only six studies reported if there were any exclusions or losses during the study.

Outcomes reported

Outcomes were reported as either degrees or millimeters and there were generally no large improvements in ROM. In fact, although a statistically significant effect of SMT was seen in several studies, the differences between pre and post treatment, when clearly reported, were, for the best results, in the order of 3.5mm for mouth opening ([34]) and 10° ([37]) for unilateral cervical rotation.

A positive effect of SMT was reported in both of the studies where mouth opening was assessed after cervical manipulation (+3.5mm ([34]) and 1.5mm ± 1.5[35]). In five of the nine studies on cervical ROM ([40],[37],[42],[43], and[39]) a positive effect was reported, whereas the remaining four studies ([36],[41],[38], and[44]) did not show improvement. See Table4 for examples of increased ROM in the studies with significant results.

None of the three studies of the lumbar spine (Table5) showed an effect of SMT on lumbar ROMs ([46],[47], and[45]) and one study ([48]) of sacroiliac manipulation (Table6) reported no effect on the ROM of the hip joint.

The lack of homogeneity in the various reports made a meta-analysis impractical. For example, different areas of the spine were studied, all of which have different movement patterns. Concerning the cervical region, study[42] measured only rotation and not all three planes; study[40] showed only the baseline values and not the post-manipulative values. Study[41] did not report exact measures but showed the results in a graph. Finally, study[39] reported only exact numerical values for the significant findings but provided no estimates for those with non-significant differences. Also the study populations differed between papers, some including patients with back problems and others using healthy people. Only two studies, ([41] and[35]), compared the outcome of SMT in three groups: treatment, sham procedure and a control group which did not receive any type of treatment. The first of these two studies, which examined cervical ROM, did not find an effect, whereas the second, which examined mouth opening, did.

Nine studies compared the action of SMT to only a sham procedure ([36],[48],[40],[37],[43],[34],[38],[44], and[47]). Five of them ([40],[37],[43],[34], and[47]) found the outcome of SMT to be statistically significant but four ([36],[48],[38], and[44]) did not.

Four authors compared the action of SMT to the outcome in a control group which did not receive any treatment ([45],[46],[42], and[39]). The first of these studies ([45]) did not find any difference between the outcomes after SMT and the control process, whereas the three others did ([46],[42], and[39]).

Finally, if articles are listed by total quality score, the seven highest ranked studies ([35],[43],[46],[40],[47],[42], and[37]), with scores ranging from 92 to 77, showed significant positive effects of SMT on ROM. Continuing down the list from scores of 69 to 33, studies[45],[38],[36],[41],[44], and[48] reported no significant differences in the outcomes between groups, while studies[34] and[39] did report significant effects of SMT.

Summary of results

This appears to be the first systematic critical literature review of the immediate effect of SMT on ROM. The fifteen articles that met our inclusion criteria for review did not provide a coherent picture; some studies found a small effect, others none. The differences in results between studies were related to the quality of studies, with the better studies reporting significant findings. Interestingly, it did not matter if SMT was compared with a sham treatment or if it was compared with some type of control procedure, suggesting that results were unaffected by expectation or observation bias. The only studies showing a positive effect in the spine were those performed in the cervical spine, while no effect was discerned after pelvic or lumbar treatment. When positive findings were present, they were generally modest.

Heterogeneous methods make meta-analysis unsuitable

Many different tools are used in research and in clinical practice to measure ROM: single/double/triple inclinometers, goniometers, a rangiometer, tape measures, visual estimation, spine motion analyzer, etc. It was not the purpose of this review to describe and compare these methods, but we refer to the literature concerning their reliability and validity[33, 49–52]. The main point is that there are quite a number of studies on the topic but that the measurement tools used often differed from one study to another, making it difficult to compare results between studies. In addition, this multitude of outcome variables plus other differences such as choice of study population and area of manipulation make it impossible to combine results in a meaningful meta-analysis.

Why such small effects- if any?

The small improvement in ROM, when an improvement was seen at all, may come as a surprise to those who have clinical experience with SMT. However, perhaps in clinical practice, the biomechanical effect of SMT is influenced by a reduction in pain. In this review, we did not take into account changes in level of pain, but rather focused on ROM, which is a less common outcome variable in randomized clinical trials on SMT and back pain[11, 46]. The choice of researchers to avoid measures of ROMs as outcome variables might well be because SMT does not generally produce substantial changes in ROMs.

Another possibility for this discrepancy between the clinical experience and the results of this review is that ROMs may improve gradually as treatment progresses over days or weeks. Our review dealt only with the immediate effects of SMT, in order not to confuse the effects of SMT with those of other factors that could come into play over a period of time. Other possible explanations are that SMT might produce changes in kinematics (i.e. how the spine moves) rather than changes in the total ROM. It is also possible that changes in one manipulated joint among several in a spinal region would not contribute greatly to changes in regional movement.

Significant effects only in the cervical spine

In this review, SMT only showed discernible effects in the cervical spine. There may be several explanations for this. First, gross ranges of motion are largest in the cervical spine and so there is the potential for treatment in that area to cause a larger, hence measureable, effect. Secondly, more than 50% of the total rotation of the cervical spine is attributed to movement at a single level, the atlanto-axial joint[53]. Hence, SMT targeting the atlanto-axial joint could potentially lead to a substantial overall increase in the rotation of the entire cervical spine. In the lumbar and thoracic spine no single segmental level is responsible for a substantial proportion of movement of the entire region. Therefore, an increase in movement at a single level below the neck is less likely to result in a large change in regional ROM, as shown in this review.

Post hoc analysis

In order to investigate this matter further, all articles that tested the effect of SMT specifically in the upper cervical spine were re-analyzed, in relation to the size of the effect. Two studies[39, 40] treated C1-C2, with different results: the first one[39], showed a difference pre/post treatment (3.75 degrees) only for right neck rotation. The other[40], showed that SMT significantly increased neck flexion and rotation (see Table4). Studies[37, 41] treated at the C3-C6 level and they also had different results:[37] showed that SMT was more effective than control mobilization, whereas[41] showed no significant effect on ROM. One study[42] was conducted at the thoracic level and showed improved cervical ROM. Finally four studies[36, 38, 43, 44] treated a clinically determined area and only one[36] showed improved cervical ROM with SMT. These results, therefore, do not corroborate the theory that upper cervical SMT has a particularly obvious effect on cervical rotation. Further study is obviously needed on this aspect of spinal ROM.

Technical challenges in measuring ROM

There are also some technical issues that have to be taken into account in interpreting the results presented herein. For example, measurements of ROM face the problem of where to start and stop; i.e. how to identify the neutral or starting position. Therefore, measurements are probably more accurate when the full ROM is reported. Although most studies in our review did measure the full range, for some reason they reported on each individual movement component (e.g. left rotation and right rotation).

It is also possible that ROM does increase immediately following treatment but that the methods used in the reviewed studies did not capture this effect. In this regard, most of authors either claimed to be using a previously tested and acceptable instrument in relation to reliability, or they presented their own reliability study with acceptable results.

Interestingly, many studies of cervical ROM seem to concentrate on the reliability of measurement tools, but construct validity and what has been referred to as ‘responsiveness’ - the ability to capture change - are studied less often[54]. Whether a particular device is capable of accurately measuring ROM with appropriate precision is of utmost importance. However, it has been pointed out[33] that certain methodological procedures, rather than the device itself, are problematic, at least for cervical ROMs. Such items are wide variations in study designs, the characteristics of the observers and study populations, whether warm-ups were allowed, whether the movements were active or passive, and the time intervals between repeated measurements. Furthermore, it is difficult to assess ROMs, and outcomes may not be reliable if there is pain in the examined area[55]. These were all items that we examined in our critical review of articles, and it was apparent that a number of studies did not take these concerns into consideration.

Methodological considerations of this review

As with all systematic literature reviews, it is likely that not all relevant articles were found, and the review process itself has a subjective element even with the systematic approach of using checklists. Other researchers may prefer to scrutinize other methodological items and may judge the contents differently. However, as there were no disagreements between the two primary reviewers, the protocol appears to have been appropriate and clearly defined.

Research findings are of course dependent on the study methodology; it is often seen that the weakest studies have the “best” results. However, in this review, the methodological score was, generally, associated with study outcome in the opposite way, i.e. studies with better quality scores were more likely to show statistically significant differences, suggesting that the effects of SMT on ROM are robust, albeit not very large. Nevertheless, the fact that there was no set cut-point for methodological acceptability and because the evidence tables report on both descriptive and quality items, the reader is able to make his/her own additional analysis of the data, should this be relevant.

Implications of findings for future research

It has been shown that it is difficult to evaluate ROM if there is pain[55]. On the other hand, it does not seem relevant to manipulate non-painful articulations in order to study the effect on movement, as these likely already have a full ROM. Therefore, it is difficult to know whether clinical improvements following SMT in patients with neck pain[11] or patients with low-back pain[1] are due mainly to improved biomechanical status, a direct reduction of pain, or a combination of the two. A previous review concluded that there is a direct action of SMT on experimentally induced pain[26] but the present review cannot isolate a general direct effect of SMT on ROM. Consequently, other information seems to be necessary to better understand the relationship between pain and motion in relation to SMT.