Reliability of diagnosis of somatic dysfunction among osteopathic physicians and medical students

Katrine Bengaard, DO, Florida Hospital East Orlando, Family Medicine Residency, 7975 Lake Underhill Road, Suite 200, Orlando, FL 32822.

E-mail address: kbengaarddo@gmail.com

This study assessed the reliability of the diagnosis of somatic dysfunction (SD) through palpation in a group of osteopathic practitioners. Somatic dysfunction is defined as “impaired or altered function of related components of the somatic system: skeletal, arthrodial, and myofascial structures, and related vasculature, lymphatic and neural elements.”1 Specific components used to identify the presence of SD include one or more of the following: tenderness, asymmetry, restriction of range of motion, and tissue texture change (TART).1,2 Osteopathic medical students are taught early in their training how to diagnose SD. Despite these definitions, there is often a discrepancy between two practitioners on what the SD diagnosis actually is. Several studies have investigated this inconsistency, and each has shown that there is some degree of disagreement among practitioners.

The palpation of the anterior superior iliac spine (ASIS) is a basic skill that is taught early in osteopathic medical schools. All osteopathic students and physicians are taught to compare the position of one ASIS with the other. If there is a discrepancy in the level of the two ASIS’, a SD is considered present (checking for the asymmetry in TART). However, even an apparently simple diagnosis to determine which ASIS is more superior is not always agreed upon among a group of examiners. This study was designed to investigate the degree of concordance among a larger group of examiners regarding their diagnosis of ASIS symmetry and location.

‌Materials and methods

To investigate the concordance of diagnosis of SD among a group of individuals, 151 osteopathic students, residents, and attendings were recruited to palpate the ASIS’ of three different models. The study design was to recruit 50 stu- dents, 50 residents, and 50 attendings. However, the actual breakdown of volunteer examiners was 95 students, 26 residents,18 attendings, and 12 unspecified. The examiners then indicated whether they believed that one ASIS was more superior when compared with the other, or whether they were equal.

Data were collected over several evenings at the American Academy of Osteopathy (AAO) Convocation in Little Rock, Arkansas, March 25-29, 2009. Twelve conference participants were recruited to serve as live models. Each completed and signed an informed consent allowing the examiners to palpate their ASIS’. One fixed, plastic model was also used as an independent type of check on interexaminer diagnostic agreement. Volunteer examiners were recruited to participate in this study either by word of mouth or with a poster set up next to the study area. This study was approved by the Florida Hospital Institutional Review Board.

Two live models per shift were positioned lying supine on examination tables. The live models were asked to lie as still as possible, being conscious not to shift their position, thereby potentially changing the position of their ASIS’. There were 25 volunteer examiners palpating each live model at any given time before the models were switched for two new, unpalpated models (12 live models, 6 shifts, up to 25 examiners per live model).

Volunteer examiners were briefly instructed on the pro- cedure of palpation. It was explained to them that they were to diagnose using static rather than dynamic palpation of the ASIS of both the live and fixed models. They were also asked to examine the model while standing on the same side of the table as the location of their dominant eye. The definition of SD was defined as a greater than or equal to 1 cm difference in the position of the ASIS. If the ASIS’ were within 1 cm of each other, they were to be designated as equal.

The examiners were given a form on which they identi- fied themselves as either a medical student, resident, or attending physician. Once in the examination area, each examiner had up to one minute to palpate the ASIS’ of the model and indicate on their form whether they believed the right or left ASIS was superior, or if they were equal. Each examiner palpated two live models as well as the fixed model and marked their forms appropriately (see Appendix 1).

‌Results

‌Concordance in palpation results across examiners of live subjects

A casual review of these data might draw a reader to conclude that, at least for some cases, there was a high degree of agreement among examiners. In Case 5, for ex- ample, examiners identified ASIS as left superior 79.2% of the time. From a clinical standpoint, 79.2% may or may not be a sufficiently high level of agreement depending on the condition under examination and the importance of an ac- curate result.

From a statistical standpoint, however, the standard is more straightforward. By chance, 33.3% of examiners would select left, 33.3% would select right, and 33.3% would select equal. Because we do not know the “correct” result, the degree of concordance is the degree of systematic variation from random chance. What is being tested using Kappa is whether students and physicians trained to use palpation to come up with a diagnosis will arrive at a finding that is consistent with results attained by their peers using the same procedure with the same patient.

Tests of agreement across students and physicians did not support the idea that palpation produces consistent re- sults when assessing the symmetry of the ASIS’. The Fleiss Kappa of 0.028 suggests a very modest degree of consis- tency that is well below general guidelines for moderate (0.41-0.60), good (0.61-0.80), or excellent (0.81-1.00) inter- rater reliability. Among this group of students and physi- cians, examinations did not produce consistent results for the 12 cases. Assessing the 190 student ratings produced slightly higher concordance, but that was still well below any indication that palpations were reliable in determining ASIS superiority (Fleiss’ kappa = 0.038).

‌Palpation accuracy relative to a known fixed result

Live subjects may add unmeasured variation to the assessment of inter-rater agreement.6This study also asked trained examiners to palpate a fixed model. The research team used a screw to fix the plastic pelvic model with bothASIS’ at equal levels. Because the model was set at equal(neither left nor right as superior), perfect accuracy would find that all 151 of the examiners would choose “equal.”However, statistical examination of the distribution of results revealed that this did not occur. Only 15 of 139 of the examinations with the fixed, equal model produced a result of equal, as shown inTable 2. These medical students and physicians did not accurately choose equal after palpation when examining a model fixed to be equal. Palpation results were very strongly different from the known fixed expected results. For this group of examiners, palpation appears to not be an accurate method of assessing ASIS symmetry.

Table 1 Palpation results (n = 278) across 12 cases Counts Percentages

Left Right Equal Left Right Equal

‌Students versus residents/attendings in successes with the fixed result

Disentangling the palpation results of students versus resident or attending physicians might reveal that more experienced practitioners could more accurately assess the model fixed at equal symmetry for ASIS.Table 3shows that 11 of 95 student assessments succeeded in producing a correct result of equal, as did four of the 44 assessments by resident or attending physicians. There were not enough attending volunteers in the sample to do a separate analysis. Therefore, they were analyzed together with the residents. Fisher’s exact test demonstrated no statistically meaningful difference between students and residents/attendings in the proportion of results successfully identifying the fixed model as equal.

‌Discussion

This study was originally conceived after reviewing th earticle by O’Haire and Gibbons.7The article showed that even a simple palpation of three landmarks (posterior superior iliac spine, sacral sulcus, and sacral inferior lateral angle) could not be agreed upon among 10 examiners.

The results of our study show that among a group of medical students, residents, and attendings, palpation for diagnosis of ASIS superiority is not reliable. This may seem puzzling, because we are all taught how to diagnose the innominates during our first semester of osteopathic medical school. However, 151 examiners were unable to consis- tently agree on the diagnosis of the three models they palpated. Therefore, although the palpation of landmarks such as the ASIS is taught universally in osteopathic med- ical schools, the possibility that practitioners cannot do it reliably casts doubts on how it is taught, how it is done, and its value in assessing patients.

It is also important to note that repeated palpation of the subjects (up to 25 palpations) may have influenced the location and symmetry of the ASIS. Although instructed not to motion test, several of the live model subjects reported that the ASIS compression test was used. Any motion in- duced into the pelvis may have, however inadvertently, delivered treatment that influenced the location of the ASIS.

Such dynamic palpation could certainly have changed the outcome of subsequent examiners’ diagnoses. In short, the first examiners may have been palpating a very different pelvis than the last examiner palpating the same subject.

One result of special interest was that, even with the fixed model, the predominant diagnosis chosen was left ASIS superior (presumably indicating either a left posterior innominate or a right anterior innominate). Because left was more often chosen in the live models (148 of the 278 palpations, or 52.2%), it is possible that either our models all had a left superior ASIS or left superior ASIS is a more common diagnosis based on examiner perspective.

J. Gordon Zink, DO, observed a similar finding in his research in the 1970s, when he coined the term common compensatory pattern (CCP).16-18 He noted that 80% of healthy subjects had a left/right/left/right (LRLR) rotational pattern. That is to say, 80% of people he examined that were free of illness or injury would, in fact, have a left superior ASIS. Zink and Lawson explain that “when the patient’s pelvis and thorax are not found in the physiologic (even) or the common compensatory pattern (left ASIS superior), the findings are identified as being disparent.”*,19

Student doctor Sepehri examined 40 Guatemalan pa- tients to determine whether those examined would fit into Dr. Zink’s LRLR pattern. She found that only 26% fell into that category, and that 66% did not fit any pattern. She suggests that the observed Zink patterns may only represent the population studied, and that examiner reliability should be assessed and documented in future research.20

Assuming that the live models used in our study were healthy, our examiners’ frequent finding that left ASIS was superior is in agreement with Dr. Zink’s LRLR pattern. However, this does not explain why, in the even, fixed model, they still chose left.

The examiners were asked to stand on the side of their dominant eye; however, there was no instruction on how to determine which eye is dominant. It is therefore likely that several of the examiners were unaware of which eye was dominant and, because the tables were set up right to left, stood on the right side, using their left eye as dominant. Their opinion may have changed had they then palpated from the left side of the table. Although it is of possible importance for future studies, this study was not designed to test this issue.

Another limitation of this study is the sample size. Also, as outlined in the Material and Methods section, we did not fulfill our study design objective, which was to recruit an equal number of students, residents, and attendings. It was particularly difficult to find residents and attendings that would participate, although students seemed more than will- ing to volunteer. Of course, one of the obvious biases was that we performed our study at the AAO Convocation. Attendees of this gathering have a particular interest in osteopathic manipulation, and should, in theory, be more intune to palpatory skills. Therefore, the findings of this study are even more interesting, because even among those who attended Convocation, data still showed quite poor interex- aminer reliability.

‌Conclusion

In both the live and fixed models, there was low reliability in the diagnosis of somatic dysfunction among trained ex- aminers. Because we are taught that SD is present when asymmetry is palpated, it is important to be in agreement about what type of asymmetry exists. This is considered an objective finding, yet “objective” infers that it is true and would not change if another examiner were also to palpate. Because we are unable to do so reliably, it may be true that asymmetry is not a good way to define SD of the innomi- nates.

‌References

1. ‌Ward RC, Hruby RJ, Jerome JA, et al: Foundations for Osteopathic Medicine. 2nd ed. Philadelphia: Lippincott Williams & Wilkins, 2002

2. ‌DiGiovanna EL: Somatic dysfunction. In: Nelson KE, Glonek T, editors. Somatic Dysfunction in Osteopathic Family Medicine. Phila- delphia: Lippincott Williams & Wilkins, 2006, pp 35

3. Fleiss JL, Cohen J: The equivalence of weighted kappa and the intraclass correlation coefficient as measures of reliability. Educ Psy- chol Meas 33:613-619, 1973

4. Gwet KL: Computing inter-rater reliability and its variance in the presence of high agreement. Br J Math Stat Psychol 61:29-48, 2008

5. ‌Li MF, Lautenschlager G: Generalizability theory applied to categor- ical data. Educ Psychol Meas 57:813-822, 1997

6. Galewaler JE: Motion in health and disease. JAOA 69(2):173/101- 179/107, 1969

7. ‌O’Haire C, Gibbons P: Inter-examiner and intra-examiner agreement for assessing sacroiliac anatomical landmarks using palpation and observation: pilot study. Man Ther 5:13-20, 2000‌

8. Paulet T, Fryer G: Inter-examiner reliability of palpation for tissue texture abnormality in the thoracic paraspinal region. Int J Osteopath Med 12:92-96, 2009

9. ‌Halma K, Degenhardt B, Snider K, et al: Intraobserver reliability of cranial strain patterns as evaluated by osteopathic physicians: a pilot study. J Am Osteopath Assoc 108:493-502, 2008

10. ‌Kmita A, Lucas N: Reliability of physical examination to assess asymmetry of anatomical landmarks indicative of pelvic somatic dys- function in subjects with and without low back pain. Int J Osteopath Med 11:16-25, 2008

11. ‌Stovall B, Kumar S: Reliability of bony anatomic landmark asymme- try assessment in the lumbopelvic region: application to osteopathic medical education. JAOA 110:667-674, 2010

12. ‌Bombei B: Students experience virtual reality training to develop palpatory skills. Osteopathic Family Physician News Nov/Dec:24-25, 2008

13. ‌Howell J, Conatser R, Williams R, et al: Palpatory diagnosis training on the virtual haptic back: performance improvement and user evalu- ations. J Am Osteopath Assoc 108:29-36, 2008

14. ‌McConnell DG, Beal MC, Dinnar U, et al: Low agreement of findings in neuromusculoskeletal examinations by a group of osteopathic physicians using their own procedures. J Am Osteopath Assoc 79:59-68, 1980

15. ‌Degenhardt B, Snider K, Snider E, et al: Interobserver reliability of osteopathic palpatory diagnostic tests of the lumbar spine: improvements from consensus training. J Am Osteopath Assoc 105:465-473, 2005

16. Dalton E: Short right leg syndrome [page 139-153]. Available at: http://www.somatics.de/Dalton/ShortRightLeg1.pdf. Accessed De- cember 28, 2010.

17. Pope RE: The common compensatory pattern: its origin and relation- ship to the postural model. AAO Journal 13:19-40, 2003

18. ‌Quinn A: The compensatory pattern, as seen in art and osteopathy. AAO Journal 10:21-23, 2000

19. ‌Zink J, Lawson W: An osteopathic structural examination and func- tional interpretation of the soma. Osteopath Ann 7:12-19, 1979

20. Sepehri DS: Osteopathic survey of somatic dysfunction and Zink compensatory patterns in Sololá, Guatemala [lecture]. Available at: http://www.osteopathic.org/inside-aoa/development/internationl- licensure/Documents/biomea-2009-sepeheri.pdf. Accessed Decem- ber 28, 2010. [Abstract found in JAOA 110:25, 2010)