Burnout, Depression, Non-Modifiable Factors, & Work Environment in Osteopathic Family Medicine Residents

To improve the well-being of residents, several initiatives, including work-hour restrictions, have been implemented. The efficacy of these initiatives has not been widely studied. As such, the purpose of the current study was to evaluate burnout and depression in Osteopathic family medicine residents, examine non-modifiable factors influencing burnout, and assess the relationship of the work environment as it relates to burnout.

Methods: The current study used a cross-sectional study design and an anonymous, web-based survey to assess burnout and depression in Osteopathic family medicine residents. Residents received invitations to participate in the survey via e-mail. The survey was created specifically for the current study.

Results: In total, 316 Osteopathic family medicine residents completed the survey. Burnout was present in 69.0% of residents, and 87.9% met criteria for depression. Females were 1.8 times more likely than males to be burned out. No significant difference was found for overall burnout when examining, age, sexual orientation or relationship status. Residents who worked more than 80 hours per week had increased emotional exhaustion and decreased personal accomplishment. Finally, 23.0% of residents reported being very satisfied about balance between personal and professional life and 58.3% reported being very satisfied about family medicine as a career choice.

Conclusions: The current study suggested that Osteopathic family medicine residents experience high burnout and depression. These negative constituents of mental well-being still exist despite the changes instituted for work-hour restrictions. Additional research is needed to determine effective interventions for this ongoing problem.

Jessica Lapinski, BS, BA, OMSIV | [email protected]

This article was designed to be viewed and distributed as a PDF. Please download the PDF for easiest reading.

INTRODUCTION

Within the past decade, changes have been made to residency training programs to reduce the number of hours worked per week.¹ These work-hour restrictions were implemented to improve the health and well-being of residents and to improve the quality of patient care.^1-4 Current standards for work hour restrictions are similar between Osteopathic and Allopathic residency programs. ^1,5-6

After these policies were implemented, scarce research was conducted to examine the efficacy of these changes. From the studies that were conducted, a majority occurring within the first 3-4 years after the restrictions were implemented, a few trends started emerging. First, studies of Allopathic residency programs showed no change in patient mortality following the implementation of these policies in July 2003.^1,7-9 Secondly, studies did not find a significant difference in burnout when comparing levels before and after the change in work-hour restrictions.^7-8,10-12 However, one study suggested that, even though the standards for work hours changed according to the Accreditation Council for Graduate Medical Education & The American Osteopathic Association, changes were not necessarily implemented.^1,8-9 There appeared to be an underlying culture in which residents were not properly logging all the hours they worked to remain within the new restriction standards.¹ With the recent publicity and awareness regarding burnout, depression, dissatisfaction and suicide among today’s physicians, it is evident that these issues are still plaguing the profession.¹²

Three factors are used to assess burnout: emotional exhaustion (EE), depersonalization (DP), and decreased sense of personal accomplishment (PA).^9,11-16 Burnout has been associated with poor work performance, increased error rate, and decreased commitment.^11,17 It has also been associated with increased health problems, substance abuse, suicidal ideation, and depression.^10-11 These factors can greatly affect health care, leading to increased medical errors that can affect patient morbidity and mortality.

With the rising rates of physician dissatisfaction and suicide, it has become evident that more research is needed to address this issue. Specifically, few studies have investigated family medicine or Osteopathic residency programs.^10-11 Therefore, the purpose of the current study was to evaluate burnout and depression in Osteopathic family medicine residents, examine non-modifiable factors influencing burnout, and assess the relationship of the work environment as it relates to burnout.

METHODS

SURVEY DISTRIBUTION

For the current study, a cross-sectional study design was used to assess burnout and depression in Osteopathic family medicine residents using an anonymous, web-based survey. In January 2015, the American College of Osteopathic Family Physicians (ACOFP) was approached about helping with the distribution of our survey because it has a complete list of e-mails for all current Osteopathic family medicine residents and because residents must subscribe to ACOFP. The ACOFP was emailed with a prompt and link to the survey and asked to forward the e-mail to all Osteopathic family medicine residents. To avoid response bias, the phrase “personal and professional satisfaction” was used in the survey prompt instead of burnout and depression. Residents interested in participating in the survey were asked to click on a link that forwarded them to an informed consent page. After reading the linked paged containing the informed consent and providing consent, the residents were taken to the survey questions. The local institutional review board and ethics committee approved all study procedures.

Because the above distribution method resulted in a poor response rate, we e-mailed the Osteopathic program directors directly and asked them to invite their residents to complete the study survey. When contacting the program directors, we emphasized that the survey would be anonymous, and results would not be seen by the resident’s program director or anyone related to the program. The entire data collection period ran for a total of five months, during which time the residents could complete the survey.

As an additional incentive for all residents who completed the survey, residents were given the opportunity to be entered into a drawing. By providing their e-mail address through a secure website, they would be entered to win an Amazon gift card; 10 gift cards were available (2 for $100, 4 for $50, and 4 for $25).

SURVEY MEASURES

The study survey consisted of 32 questions and took approximately 5-7 minutes to complete. All survey questions were required, however, respondents were allowed to put “refuse to answer” for any questions they did not feel comfortable answering. Basic demographic questions were included requesting information about gender, age, year in training, relationship status, the number of children and sexual orientation. The Maslach Burnout Inventory-Human Services Survey (MBI-HSS)13 was also included to quantify burnout in survey respondents. This scale assesses three specific variables to quantify burnout: EE (9 questions), elevated DP (5 questions) and decreased sense of PA (8 questions). These three variables were analyzed as both categorical and continuous variables. The MBI-HSS was graded on a 7-point scale with 1 being “never” and 7 being “every day,” resulting in the following ranges: EE (0 to 63 points), DP (0 to 35 points), and PA (0 to 56 points). Arithmetic mean was used to find the average for each subscale using each resident’s response to questions within that subscale. Categorical variables, based on a cut-off provided by the developer, were used to classify survey respondents as having low, moderate, or high on the burnout scale.¹³ For the current study, we created a burnout category which has been used and validated in previous studies.^14-15 This included individuals with high EE or de- creased the sense of PA, which allowed us to look more closely at the overall burnout rate.

The Patient Health Questionnaire (PHQ-9)18 was also included in the study survey to assess depression and severity of depression. Responses to this 9-question survey are scored on a scale of 0 for “not at all” to 3 for “nearly every day,” with ranges from 0-27 points. The following cutoffs were used in the current study to classify depression: minimal (1-4 points), mild (5-9 points), moderate (10-14 points), moderately severe (15-19 points), and severe (20-27 points). We used categorical and continuous variables for our statistical and descriptive analyses. Responses to this part of the survey were not meant to diagnosis someone with “clinical depression,” but rather to provide a quantitative measure of depression.

Questions were also included in the study survey to assess the resident’s work environment. These questions, which used a Likert-like scale, included factors such as stress level at work and how the 80-hour work limit affected the resident’s stress level and performance. We also included questions about on-call schedule, night/weekend schedule, work schedule, and sleeping habits. Finally, we included questions that asked residents about personal and professional life satisfaction. Specifically, using a Likert scale, residents were asked how satisfied they were with the balance between their personal and professional life and how satisfied they were with their choice of family medicine as a career.

STATISTICAL ANALYSIS

SPSS version 18.0 predictive analytic program (IBM, Chicago, IL) was used for all statistical analyses. Percentage, mean, confidence interval (CI), and standard deviation (SD) were reported when applicable. A 2 test of independence was performed to examine the relationship between burnout and gender, age, relationship status, sexual orientation, depression, on-call schedule, night/weekend schedule, work schedule, and sleeping habits. To further examine the subscales of burnout (EE, DP, and PA), either a 1-way analysis of variance in conjunction with Tukey post hoc comparisons or an independent samples t-test was used for comparisons between burnout subscales and gender, age, relationship status, sexual orientation, depression, on-call schedule, night/weekend schedule, work schedule, and sleeping habits. Cronbach  was used to examine the reliability of the scales used in the current study. A p≤.05 was considered statistically significant.

RESULTS

RESIDENT CHARACTERISTICS

Approximately 1700 Osteopathic family medicine residents were e-mailed an invitation to complete the study survey. From this total, 316 residents responded to the survey, resulting in a response rate of 18.6%: 145 residents responded from the initial e-mail invitation sent by the ACOFP, and 171 responded to the e-mail sent by the Osteopathic program directors. Table 1 (page 14) presents the demographic characteristics of the residents who responded to our survey. Approximately an equal number of male (45.9%) and female residents completed our survey, with a large percentage (87.3%) falling within the 26 to 35 years age group. We had a slightly higher number of first-year residents (38.3%) respond than second-year or third-year residents (30.4% and 31.3%). Further, 60.8% of residents were married or living as married, and the majority (64.6%) had no children. Finally, 6.0% of residents self-identified as lesbian / gay / bisexual / asexual.

TABLE 1: Characteristics of Osteopathic Family Medicine Residents (N=316)

a Refers to the number of residents who completed the survey but did not provide an answer to this specific question or responded “refuse to answer.” Abbreviation: OGME, Osteopathic Graduate Medical Education.

FIGURE 1:

Refer to PDF for diagram

 

BURNOUT & DEPRESSION

Based on the MBI-HSS criteria, 218 (69.0%) of Osteopathic family medicine residents were burned out. Figure 1 displays the frequency distribution of the burnout subscales. Specifically, 192 residents (68.6%) had high EE, 171 (60.6%) had high DP, and 205 (75.1%) had a low sense of PA. Mean (SD) scores for EE (33.33 [11.67] points) and DP (15.66 [6.73] points) were both within the high range, and PA (43.52 [7.60] points) was in the low range.

In the current study, 240 residents (87.9%) met the PHQ-9 criteria for some level of depression. Specifically, 33 (12.1%) residents were not depressed, 125 (45.8%) had minimal depression, 61 (22.3%) had mild depression, 28 (10.3%) had moderate depression, 16 (5.9%) had moderately severe depression, and 10 (3.7%) had severe depression.

NON-MODIFIABLE FACTORS INFLUENCING BURNOUT

GENDER

When comparing burnout and gender, 90 (62.1%) males and 127 (75.1%) females met criteria for burnout. The odds of having burnout were 1.8 times greater for females than for males (95% CI, 1.14-2.99; p=.01). The mean (SD) EE in males (31.14 [11.91] points) was statistically different than in females (35.15 [11.19] points, t(277)=2.90, p=.004). The mean DP and PA were not significantly different for males and females (15.62 vs 15.68 points and 42.62 vs 44.29 points).

AGE & SEXUAL ORIENTATION

No statistically significant difference was found for burnout between the different age groups (2(3, 311)=5.67, p=.13) or based on sexual orientation (2(1, 311)=0.02, p=.89). Examination of the burnout subscales also found no statistical differences for age (2(3, 277)=158.44, p=.43 for EE; 2(3, 279)=95.68, p=.25 for DP; 2(3, 270)=71.31, p=.98 for PA) or sexual orientation (2(1, 278)=39.66, p=.90 for EE; 2(1, 280)=37.04, p=.15 for DP; 2(1, 271)=24.49, p=.86 for PA).

RELATIONSHIP STATUS

No statistically significant difference was found for overall burnout between the different relationship status groups 2(2, 313)=3.17, p=.21). Examination of the burnout subscales Frequency Distribution of the Burnout Subscales of Emotional Exhaustion (EE), Depersonalization (DP), and Personal Accomplishment (PA) found a statistical difference for EE (F(2, 277)=6.01, p=.003), DP (F(2, 269)=4.77, p=.009), and PA (F(2, 275)=5.72, p=.004). A post-hoc Tukey multiple comparisons test found those who were never married reported the highest DP, and those who were divorced or separated reported the lowest DP. Those who were married or living as married reported the highest PA, and those who were never married reported the lowest PA. Finally, those who were divorced or separated reported the highest EE, and those who were married or living as married reported the lowest EE.

DEPRESSION

When comparing burnout and level of depression, residents with higher levels of depression were more likely to have burnout than those who were not depressed (2(5, 273)=64.17, p<.001). Examination of the burnout subscales found a statistical difference for all three subscales: EE (F(5, 261)=39.92, p<.001), DP (F(5, 261)=11.65, p<.001), and PA (F(5, 253)=6.54, p<.001). A post-hoc Tukey multiple comparisons test found that those residents who had higher levels of depression reported more EE and DP and less PA.

WORK ENVIRONMENT & BURNOUT

When asked if the 80-hour work limit had improved their overall stress level, 65 residents (21.6%) indicated that they strongly agreed, 87 (28.9%) agreed, 102 (33.8%) were neutral, 31 (10.3%) disagreed, and 16 (5.3%) strongly disagreed. When asked if the 80-hour work limit had improved their overall work performance, 55 (18.3%) strongly agreed, 86 (28.6%) agreed, 111 (36.9%) were neutral, 33 (11.0%) disagreed, and 16 (5.3%) strongly disagreed.

Table 2 summarizes survey responses about work environment factors, including on-call schedule, night/weekend schedule, work schedule, and sleeping habits. In general, 73.5% of Osteopathic family medicine residents were on call less than five days per month, and 69.7% worked nights/weekends less than five days per month. For work schedule, 58.3% reported working between 60-80 hours. For sleeping habits, 78.6% slept between 6-8 hours a night.

ON-CALL SCHEDULE

No statistically significant difference was found between burnout and the different on-call schedule groups (2(3, 309)=1.46, p=.69). Examination of the burnout subscales found a statistical difference for EE (F(3, 274)=3.47, p=.02), but no difference for DP (F(3, 276)=0.98, p=.40) or PA (F(3, 268)=0.30, p=.83). Post-hoc comparisons found that residents who were on call for more than 16 days per month had the highest EE.

NIGHT/WEEKEND SCHEDULE

No statistically significant difference was found between burnout and the different night/weekend schedule groups (2 (3, 310)=2.52, p=.47). Examination of the burnout subscales also found no statistical differences (F(3, 274)=1.14, p=.35 for EE; F(3, 276)=1.52, p=.21 for DP; F(3, 268)=1.41, p=.24 for PA).

WORK SCHEDULE

No statistically significant difference was found between burnout and the different work schedule groups (2(3, 312)=3.15, p=.37). Examination of the burnout subscales found a statistical difference for EE (F(3, 276)=4.40, p=.005) and PA (F(3, 269)=2.60, p=.05), but not for DP (F(3, 278)=1.69, p=.17). Post-hoc comparisons found that residents who worked more than 80 hours per week had the highest EE, and those who worked between 40-59 hours had the lowest EE. Residents who worked 60-80 hours a week had the highest PA, and those who worked 20-39 hours a week had the lowest PA.

TABLE 2: Work Environment Factors in Osteopathic Family Medicine Residents (N=316)

SLEEPING HABITS

No statistically significant difference was found between burn- out and the different sleeping habits groups (2(4, 312)=7.41, p=.12). Examination of the burnout subscales found a statistical difference for EE (F(4, 275)=5.83, p<.001), but no differences for DP (F(4, 277)=1.95, p=.10) or PA (F(4, 268)=0.41, p=.80). Post-hoc comparisons found that residents who slept less than five hours per night had the highest EE, and those who slept more than eight hours had the lowest EE.

PERSONAL AND PROFESSIONAL LIFE SATISFACTION

For survey questions about balance between personal and professional life, 70 (23.0%) reported being very satisfied, 127 (41.6%) reported being somewhat satisfied, 24 (7.9%) reported feeling neutral, 60 (19.7%) reported being somewhat dissatisfied, and 24 (7.9%) reported being very dissatisfied. For survey questions about career satisfaction, 176 residents (58.3%) reported being very satisfied, 70 (26.2%) reported being somewhat satisfied, 15 (5.0%) reported being ambivalent, 23 (7.6%) reported being somewhat dissatisfied, and 9 (3.0%) reported being very dissatisfied.

RELIABILITY OF THE SCALES

A Cronbach  correlation coefficient was used to assess the internal consistency of the scales. For the MBI-HSS, the Cronbach  for the burnout score was 0.93. The Cronbach  for the three burnout subscales was 0.92 for EE, 0.82 for DP, and 0.82 for PA. The PHQ-9 had a Cronbach  of 0.90.

DISCUSSION

The purpose of the current study was to evaluate burnout and depression in Osteopathic family medicine residents, examine non-modifiable factors influencing burnout, and assess the relationship of the work environment as it relates to burnout. To our knowledge, since changes were implemented for work-hour restrictions, no studies have examined Osteopathic family medicine resident burnout. Therefore, results of the current study may provide additional information about the consequences of those changes and how they have impacted Osteopathic family medicine residents and, as such, the patients that they care for.

In the current study, 69.0% of Osteopathic family medicine residents met the MBI-HSS criteria for burnout. For the burnout sub- scales, 68.6% of residents had high EE, 60.6% had high DP, and 75.1% had a low sense of PA. Previous research has suggested that overall burnout rates in all resident specialties range from 27%-75%.2-4,10 Martini et al¹⁹ found that the overall rate of burn- out was 27% in family medicine residents. Results of the current study suggested current Osteopathic family medicine residents had a higher rate of burnout than residents in previous studies. Given that this is the first study specifically targeting Osteopathic family medicine residents more studies are needed to solidify the actual rates of burnout in this population.

Several factors may explain our higher burnout results. For example, current residents may have increased time constraints from work-hour restrictions because they are required to perform the same amount of work in less time. As a result, work not completed during work hours may be expected to be done on the resident’s own time, causing increased pressure and stress.

Additionally, current residency programs may be enforcing standards for work hours and other restrictions, whereas previous research seemed to indicate these standards were not being followed.8 Therefore, enforcement of these standards could be contributing to the increased burnout we observed because residents may be precluded from some events or educational opportunities (such as performing time-sensitive procedures or missing interesting cases) due to work-hour or other time restrictions.

When comparing burnout results with the non-modifiable factors of gender, age, sexual orientation, and relationship status, some interesting results emerged. Gender significantly impacted burnout and all three burnout subscales. Female residents were 1.8 times more likely to be burned out compared with male residents. Other studies have shown mixed results regarding burnout and gender, but there is no current consensus.^10-11 When comparing burnout with age and sexual orientation, we found no significant differences. The sexual orientation finding contradicts previous research, which suggests that sexual minorities are at increased risk of burnout due to minority stress.²⁰ Finally, for relationship status, we found no differences for burnout, but there were significant differences for the burnout subscales. Specifically, residents who were never married had the highest DP and the lowest PA. Residents who were married or living as married had the highest PA and the lowest EE, and those who were divorced or separated reported the lowest DP and the highest EE. Research is mixed about the role of relationship status and burnout.^10-11 More research is needed to elucidate the potential differences.

When evaluating depression in the Osteopathic family medicine residents of the current study, 87.9% of residents met criteria for some level of depression, and 3.7% were classified as having severe depression. Our analyses found a higher rate of burnout in those residents who were depressed, and comparisons with the burnout subscales were also statistically significant. These results are comparable to a study examining Osteopathic medical students that found 77.7% of students met criteria for some level of depression, and 2.4% had severe depression.²¹

When comparing burnout with the work environment, we found no statistical differences. However, when comparing the burnout subscales with the work environment, some interesting trends emerged. For on-call schedule, residents with the highest number of on-call days had the highest degree of EE. Residents who worked more than 80 hours per week also had the highest EE, while those working 40-59 hours had the lowest. Personal accomplishment was highest in residents who worked 20-39 hours a week and lowest in those who worked 60-80 hours. Finally, for sleeping habits, residents who slept more had lower EE. This finding was consistent with previous research that showed decreased sleep was related to high burnout in both medical students and residents.^1,22

The current study had several limitations, including response bias, unacceptability bias, and selection bias. Residents who were unhappy may have been more responsive and motivated to return our study survey, despite our efforts to minimize these response biases. Therefore, the survey responses from participating residents may not represent the overall Osteopathic family medicine population. Residents who did not respond to our survey may have had either higher or lower burnout rates compared with those who did respond. Because depression and burnout are viewed in a negative light, those residents who did respond to our survey may not have been completely honest when answering survey questions due to feelings of embarrassment. This unacceptability may have been especially problematic for those residents who are familiar with the tools used to screen patients for depression. As our study design was cross-sectional, inferences about the progression of mental health during medical education may be limited.

Future research is needed to further elucidate several important factors. First, studies should examine why Osteopathic family medicine residents appear to have higher rates of burnout then residents in previous research, and what specific factors may be contributing to these higher rates. Further research is also needed to better examine the role that non-modifiable factors play in regard to burnout and how specific interventions may be needed to target individual subgroups of the population. Finally, more research is needed in regards to the new work environment that our residents are functioning within and how this environment is impacting current residents’ well-being.

CONCLUSIONS

Results from the current study suggested that factors, including non-modifiable factors and work environment, may impact burnout in Osteopathic family medicine residents. As our findings suggest and despite mandated work-hour restrictions, burnout and depression still seem to be a prevailing issue experienced by today's residents. With the rising rates of physician dissatisfaction and suicide, it becomes imperative to discover ways of combating this serious issue.

FUNDING / SUPPORT:

The current study was funded by a Still-OPTI grant.

ACKNOWLEDGEMENTS:

The authors would like to extend their deepest gratitude to the following individuals: Anita Franklin for assistance with survey set- up, Dr. Timothy Buffey for serving as our mentor and the Still-OPTI for providing us with funding.

REFERENCES

1. Lisa Rosenbaum, M.D., and Daniela Lamas, M.D. Residents' Duty Hours — Toward an Empirical Narrative. N Engl J Med 2012; 367:2044-2049November 22, 2012DOI: 10.1056/NEJMsr1210160

2. Ripp JA, Bellini L, Fallar R, Bazari H, Katz JT, Korenstein D. The impact of duty hours restrictions on job burnout in internal medicine residents: a three-institution comparison study. Acad Med. 2015;90:494-499.

3. Jennings ML, Slavin SJ. Resident wellness matters: optimizing resident education and wellness through the learning environment. Acad Med. 2015;90:1246-1250.

4. Earle L, Kelly L. Coping strategies, depression, and anxiety among Ontario family medicine residents. Can Fam Physician. 2005 Feb;51:242-3. PubMed PMID: 16926935; PubMed Central PMCID: PMC1472973.

5. ACGME resident duty hours in the learning and working environment comparison of 2003 and 2011 standards. http://www.acgme.org/ acgmeweb/Portals/0/PDFs/dh-ComparisonTable2003v2011.pdf. Accessed September 23, 2015.

6. AOA the basic documents for postdoctoral training. http://www. Osteopathic.org/inside-aoa/accreditation/postdoctoral-training- approval/postdoctoral-training-standards/documents/aoa-basic- document-for-postdoctoral-training.pdf. Updated July 1, 2015. Accessed November 3, 2015.

7. Volpp KG, Rosen AK, Rosenbaum PR, et al. Mortality among patients in VA hospitals in the first 2 years following ACGME resident duty hour reform. JAMA. 2007;298(9):984-992.

8. Shetty KD, Bhattacharya J. Changes in hospital mortality associated with residency work-hour regulations. Ann Intern Med. 2007;147(2):73-80.

9. Thomas NK. Resident Burnout. JAMA.2004;292(23):2880-2889. doi:10.1001/jama.292.23.2880.

10. Ishak WW, Lederer S, Mandili C, et al. Burnout during residency training: a literature review. J Grad Med Educ. 2009;1(2):236-242.

11. McCray LW, Cronholm PF, Bogner HR, Gallo JJ, Neill RA. Resident Physician Burnout: Is There Hope? Family medicine. 2008;40(9):626-632.

12. Cedfeldt AS, Bower E, Flores C, Brunett P, Choi D, Girard DE. Promoting resident wellness: evaluation of a time-off policy to increase residents' utilization of health care services. Acad Med. 2015;90:678-683.

13. Maslach C., Jackson SE, Leither M. Maslach Burnout Inventory. Evaluating Stress: A Book of Resources. 3rd ed. Lanham: Rowman & Littlefield Publishers; 1997:191-218.

14. L. N. Dyrbye, C. P. West, D. Satele, S. Boone, L. Tan, J. Sloan, and T. D. Shanafelt, “Burnout Among U.S. Medical Students, Residents, and Early Career Physicians Relative to the General U.S. Population.,” Acad. Med., vol. 89, no. 3, pp. 1–9, Jan. 2014.

15. T. D. Shanafelt, S. Boone, L. Tan, L. N. Dyrbye, W. Sotile, D. Satele, C. P. West, J. Sloan, and M. R. Oreskovich, “Burnout and satisfaction with work- life balance among US physicians relative to the general US population.,” Arch. Intern. Med., vol. 172, no. 18, pp. 1377–85, Oct. 2012.

16. Joules, N. , Williams, D. and Thompson, A. (2014) Depression in Resident Physicians: A Systematic Review. Open Journal of Depression, 3, 89-100. doi: 10.4236/ojd.2014.33013.

17. Shanafelt T, Bradley K. Burnout and self-reported patient care in an internal medicine residency program. Intern Med. 2002;136(5);358-367.

18. Kroenke K, Spitzer RL. The PHQ-9: a new depression diagnosis and severity measure. Psychiatry Ann. 2002;32(9):509-521.

19. Martini S, Arfken CL, Churchill A. Burnout comparison among residents in different medical specialties. Acad Psychiatry. 2004;28(3):240-242.

20. Dyrbye LN, Massie FS, Eacker A, et al. Relationship between burnout and professional conduct and attitudes among US medical students. JAMA. 2010;304(11):1173-1180.

21. Lapinski J, Yost M, Sexton PS, LaBaere RJ. Factors modifying burnout in Osteopathic medical students. Acad Psychiatry. 2015 [Epub ahead of print].

22. Yost MG, Johnson JC, Johns MM III, Burchett KD. Burnout among Osteopathic otolaryngology residents: identification during formative training years. J Am Osteopath Assoc. 2014;114(8):632-641.