Effectiveness of a Clinically Oriented Motivational Interviewing Training Program in Increasing Skills & Changing Perceptions

Leslie Spencer, PhD, MS | spencer@rowan.edu

ACKNOWLEDGEMENTS

Sima Bennett and Rose Rossiter, Coordinator and Administrative Assistant of the Clinical Education & Assessment Center – Standardized Patient Lab, Rowan University School of Osteopathic Medicine, for assistance in training and scheduling of the patient actors.

INTRODUCTION

Motivational interviewing (MI) is an evidence-based strategy that can be used by health practitioners to help patients make quality treatment decisions, comply with treatment recommendations, and change their health-related behaviors to increase their over- all quality of life.1 Although clinical encounters with patients are brief (often less than 15 minutes), modified MI can effectively create a collaborative environment between the health practitioner and the patient where the patient feels empowered to make decisions that are in his/her best interest, rather than merely following a healthcare provider’s prescribed action plan. When shared decision-making is utilized, it is more likely that patients will comply with a treatment strategy.2,3

Using MI in the clinical setting incorporates establishing an agenda and rapport, identifying ambivalence, asking open-ended questions, reflective listening, and tailored advice giving/education as the main techniques for engaging the patient.1 The goal of MI in this brief encounter is to empower the patient to identify the need for change and express the desire to change him or herself, rather than being told to do so by a health practitioner.2 Typically, medical practitioners fall into this habit of simply giving advice to their patients, hoping this will be an effective strategy in decision-making. Unfortunately, this counseling style has been found to be effective only 5-10% of the time in the areas of smoking cessation and addiction management.3 In addition, practitioners are rarely trained in lifestyle management and behavior change, so treatment is often unsuccessful, reinforcing the idea that treatment is not worthwhile.4 MI is a more effective method for helping people become motivated to change that is patient-centered and is practically and economically feasible, given that it can occur within the time frame of a typical 15 minute patient visit with a practitioner, with multiple encounters increasing the positive effect.5 In a systematic review and meta-analysis, 72 randomized controlled trials were assessed for effectiveness of MI training on patient outcomes. All studies used indirect measures (e.g. questionnaires) to measure effect; 46% also used direct measures (health outcome, direct/indirect indicators and utilization of healthcare services). The results indicated that physicians obtained a positive effect on direct and indirect measures in 75% of the studies.6 In a randomized controlled trial on the effectiveness of clinicians trained in MI on changing patients behavior related to diabetes control, results indicated that patients that were treated by trained MI clinicians were significantly more motivated to change their behavior versus those treated by non-trained clinicians.5

A significant body of research indicates that physicians who have been trained in MI have used it successfully with their patients. In brief, Soderlund et al.6 report in their systematic review of 10 studies on MI training that general health care practitioners view MI training as favorable. Rubak et al.7,8 report that those trained in MI believe that MI is a practical approach that physicians can use as part of routine care and that the outcomes will be more favorable than the traditional method of simply telling patients what to do. Saitz et al.9 reported in their study that 91% of the clinicians felt that the training affected their practice in a positive manner and physicians in a study by Rubak et al.5 reported that MI is “realistic and usable in daily work” and is “more effective than traditional advice giving.” Lastly, and, most importantly, in a randomized con- trolled trial comparing the patient outcomes of physicians trained in MI to those in the control group, at one year the patients of the MI trained physicians were more motivated to change their behaviors compared to control participants.5

While primary care physicians play a central role in counseling patients in health matters, a body of research shows that they do not have the skills in behavior change counseling, nor do they feel confident in their ability to help patients change their health-related behaviors.10 Therefore, it would be beneficial to begin MI training early in the physician preparation process. Several studies have shown success in this area, with Haeseler et al.11 recommending training in MI as early as year three of medical school. In a randomized controlled trial of 131 medical students, the MI trained group showed significantly better MI skills than did the control group.10

Rationale for The Present Study

While the research is clear that MI has the potential to change both the counseling behaviors of the clinician and the health behaviors of the patient, Barwick et al.12 state that more work is needed to understand how delivery of MI training can best be implemented. Effective training programs tailored for using MI in brief clinical encounters are needed. Additionally, previous studies have relied on self-reported use of MI strategies through post-intervention surveys of participating physicians as the sole means of demonstrating the success of the training and effective use of MI strategies. In the present study, we used a more rigorous method to demonstrate the effectiveness of the training program for brief clinical encounters, which includes use of standardized patients. This offers an objective measure of skill improvement and is a verifiable means to assessing skills.13

Purpose of The Present Study

The purpose of this study was to assess: 1) the effectiveness of a motivational interviewing (MI) training program for use in clinical encounters to improve the skills of family practice residents, and

2) resident and standardized patient perception of skills, resident knowledge, perceptions of the effectiveness of training, and beliefs about MI in clinical practice. Specifically, changes in the following outcomes were measured:

1. Skills of using MI using the Behavior Change Counseling Index (BECCI)

2. Perceptions of resident and standardized patient regarding use of MI skills

3. Knowledge of MI core skills in brief, clinical encounters

4. Perceptions of the effectiveness of training

5. Beliefs about MI in clinical practice

METHODS

Participants

Family practice residents were recruited from the family practice residency program of an osteopathic medical school. Residents in this program participated in required weekly educational sessions as part of their program; therefore it was convenient to incorporate the MI training into their previously established curriculum with minimal burden to the residents. Before the training occurred, we received full approval from the Institutional Review Board for our study.

Instruments

To assess the effectiveness of the MI training in increasing skills, resident standardized patient encounters were scored using the Behavior Change Counseling Index (BECCI) at baseline, end-of- intervention and 3-month follow-up. Self-report perceptions of resident use of MI skills in the standardized patient encounter were assessed using electronic reflection forms at baseline; end-of-intervention and 3-month follow–up. standardized patients also completed a self-report electronic reflection form following each encounter.

BECCI - All standardized patient encounters were videotaped and independently scored by two of the researchers using BECCI, a previously-validated standardized tool that has demonstrated statistically acceptable levels of internal consistency, inter-rater reliability, intra-rater reliability, and responsiveness (i.e. sensitivity to changes in subjects from pre- to post-test).13 We selected BECCI because it provides a quantitative score that can be used to com- pare outcomes at multiple points over time. Each of the 11 items on the index are scored using a Likert scale, indicating to what ex- tent the practitioner carried out the action (0 = Not at all, 1 = Minimally, 2 = To some extent, 3 = A good deal, 4 = A great extent). Each item on the BECCI scale is listed in Table 1 (page 12).

We completed the required training for using BECCI as a scoring tool, which included three readings, a training video, and thorough review of the manual. The two members of the research team per- forming the scoring also practiced by independently scoring MI encounters not associated with this study, and reviewing each other’s scores until mutual agreement and understanding of the scoring mechanism was reached. An 84% inter-observer agreement rate was achieved for each of the 11 items between the raters.

MI Skills Reflection Forms - The residents completed a self-report electronic reflection form that asked them to rate their effective- ness with the patient on the following core skills of MI: reflective listening, showing empathy, asking open-ended questions, resisting the righting reflex, and giving advice in an MI style. Each of the skills were rated using a 5-point Likert Scale (5=Strongly Agree, 4=Agree, 3=Neutral, 2=Disagree, 1=Strongly Disagree). They were also asked two open-ended questions: (1) list at least one thing they felt they did well in the interaction and (2) list at least one thing they felt they could have done better and would like to improve upon.

Standardized patients completed an electronic reflection form assessing their perception of resident skills in the following areas: asking open-ended questions, using reflective listening, showing empathy and respect for patient choice, and giving tailored advice. Each of the skills was scored using a dichotomous scale (Agree/ Disagree).

Post-Training Survey - The residents completed an electronic sur- vey on the final day of training. The survey assessed three areas: (a) resident knowledge of the righting reflex, advice giving using an MI style, identifying ambivalence and how to respond, and proper use of behavior change scales (4 items), (b) perceptions of the effectiveness of the training (6 items) and (c) beliefs about MI in clinical practice (5 items).

Standardized Patients

Standardized patients were trained by the staff of the University Clinical Education and Assessment Center/Standardized Patient Lab using cases developed by the research team. Each interaction included typical family practice patient interactions, and was focused on changing health behaviors such as improving dietary habits, increasing exercise or smoking cessation. Two males and two females were used in the interactions, and all presented as middle-aged relatively healthy patients in need of lifestyle changes to prevent or treat chronic diseases such as diabetes, high blood pressure, and obesity. Standardized patients were paid their typical fee from the Clinical Assessment Center for their participation in the study.

Procedures

Seventeen family practice medical residents completed one 15-minute standardized patient interaction during the month prior to the start of training. The encounters were videotaped and independently scored by two of the researchers using BECCI. Res- idents and standardized patients completed the MI Skills Reflection Form immediately following the encounter.

Following this encounter, two members of our research team met with the residents four times over a two-month period to provide eight total hours of MI training. Sessions took place every other Friday afternoon for two hours from October 2014 - November 2014. Training included short didactic lessons, case studies, large and small group discussions, role plays, and individualized feed- back and coaching to help residents develop the following skills related to MI: establishing an agenda and rapport, identifying 

TABLE 1:

Teaching Activities Utilized

Session	Core Skills / Technique	Methodology
One	Overview of Motivational Interviwing Establishing Rapport	Didactic Lecture Video examples of MI in brief patient encounters Role plays
	Agenda Setting	Group discussion
	Open-Ended Questions	Debriefing of use of skills with patients with Q/A Short didactic lecture
Two	Identifying Ambivalence	Sample patient case scenarios with role play and small group
	Identifying Change Talk	coaching Worksheets with sample open-ended questions
Three	Reflective Listening	Debriefing of use of skills with patients with Q/A Short didactic lecture Sample patient case scenarios with role play and small group coaching Video examples
Four	Informing “MI Style” Goal Setting Individual Skill Evaluation	Debriefing of use of skills with patients with Q/A Short didactic lecture Individual Feedback and Coaching based on SP encounter #2

ambivalence and change talk, asking quality open-ended questions, reflective listening, and tailored advice giving/education. Table 1 provides more detail on the content of the training sessions. Following the last training session, the residents electronically completed the Post-Training Survey.

Residents completed a second standardized patient interaction during the month following the eight weeks of training to allow for a post-program evaluation of his/her use of MI strategies. The 15-minute encounters were videotaped and again independently scored by two of the researchers using BECCI. Similar to baseline, the residents and standardized patients completed the MI Skills Reflection Form immediately following the encounter.

Following the second encounter, two members of the research team met with the residents on four Friday afternoons between February and March of 2015 to follow-up with the training program discuss their experiences with the use of MI in practice. They were invited to share their confidence in using MI, how frequently they were using it with their patients, the barriers they faced to 

TABLE 2:

Itemized and Total BECCI Scores at Baseline, End-of-Intervention and 3-Month Follow-Up (Mean Scores on a 1- 4 Scale)+

MI Skill	Baseline (n=17)	Post (n=13)	Change from Baseline	Follow-up (n=13)	Change from Baseline
Invited patient to talk about behavior change	0.61	1.61	1	2.67	2.06
Demonstrated sensitivity to talking about other issues	0.81	2.42	1.61	2.08	1.27
Encouraged patient to talk about current behavior or status quo	1.84	2.71	0.87	2.71	0.87
Encouraged patient to talk about behavior change	0.32	2.93	2.61	2.92	2.60
Asked questions to elicit how patient thinks and feels about topic	0.87	2.85	1.98	3.00	2.13
Used empathetic listening statements when patient talks about the topic	0.41	2.79	2.38	1.67	1.26
Used summaries to bring together what the patient says about the topic	0.06	1.58	1.52	0.79	0.73
Acknowledged challenges about behavior change that the patient faces	0.35	2.44	2.09	1.79	1.44
When providing information, it is sensitive to patient concerns and understanding	1.31	2.63	1.32	2.08	0.77
Actively conveyed respect for patient choice about behavior change	1.08	3.04	1.96	3.29	2.21
Exchanged ideas about how the patient could change current behavior	0.47	2.44	1.97	1.83	1.36
Total	0.74	2.49	1.75	2.26	1.52

+ 0 = Not at all 1 = Minimally 2 = To some extent 3 = A good deal 4 = A great extent

using it, and their intentions to continue using it. Written notes were made at these meetings to document the feedback offered by the residents. Individualized coaching and feedback was given to each resident in the form of mutual review of their second standardized patient encounter.

Residents completed a third (and final) standardized patient interaction during the month following the final Friday afternoon session to assess their use of MI strategies. The 15-minute session was videotaped and independently scored by two of the researchers using BECCI. Residents and standardized patients completed the MI Skills Reflection Form immediately following the encounter.

Data Analysis

Given that this was a pilot test of the training program with a small sample size and no control group, we limited our analyses to descriptive statistics. Inferential statistics (paired sample t-tests) would not be appropriate due to the power of the test being too low. Therefore, p-values of the differences in means from baseline, end-of-intervention and 3-month follow-up are not reported. For each resident, the BECCI scores from both researchers were com- bined to find the average score of each of the 11 items. The total BECCI score was calculated by taking the average of each of the 11 items on the index, as directed by the BECCI manual. For each of the survey items on the Reflection Forms and Post Training Survey, percentages were calculated.

RESULTS

Participants

Of the 17 residents who began the training, 13 completed all training sessions, standardized patient encounters, and baseline, end-of-intervention and 3-month follow-up surveys. Ten (56%) residents reported no previous structured training in counseling; two of the residents reported having a bachelor's degree in psychology, four reported undergraduate medical school training using standardized patients and one gained experience in counseling as a research assistant.

Change in MI Skills as Assessed by BECCI

BECCI evaluates the extent to which the practitioner carries out each of 11 separate action items. Baseline, end-of-intervention and 3-month follow-up scores of each of these items are reported in Table 2. Each item is rated on a scale of 0 to 4, with 4 being the highest rating. Overall, the average score of the residents on all 11 items combined increased from 0.74 to 2.49 at end-of-intervention, and decreased to 2.26 at 3-month follow-up. This indicates that overall, residents increased from using MI skills less than “minimally” at baseline to between “some extent” and “a good deal” at 3-month follow-up.

As reported in Table 2, residents saw the most improvement in the following areas: (a) encour- aging patients to talk about change, (b) asking good open-ended questions, (c) reflective lis- tening, (d) acknowledging challenges to mak- ing changes, (e) conveying respect for patient choice, and (f) exchanging ideas for change with the patient. Each of these items was specifically covered in the training, indicating residents may have learned these skills from the training program. At 3-month follow-up, all skills were maintained (indicated by improvement from baseline), with some skills showing further improvement from end-of-intervention: (a) inviting to talk about change (agenda setting), (b) asking good open-ended questions, and (c) conveying respect for patient choice. Additional coaching and feedback was given to each resident following the end-of-intervention, which may have helped increase these skills. Although no skills were rated as a 4 (“to a great extent”) at end-of-intervention or 3-month follow-up, at end-of-intervention 82% of the scores fell in

FIGURE 1:

Changes in Core Motivational Interviewing Skills as Assessed by BECCI at Baseline, End-of-Intervention and 3-month Follow-Up

 

the range of “to some extent” to “a good deal.” At 3-month follow-up, 36% of the scores were in this range, indicating the need for further coaching and feedback to maintain skills. Figure 1 shows improvement in the core skills specifically covered in the training program.

Results of Self-Report Reflections of Resident & Standardized Patient MI Skills

Resident MI Skills Reflection - Table 3 shows the self-perceptions of the residents regarding their ability to use MI skills and how these perceptions changed among the group from baseline, end-of-intervention, and 3-month follow-up. In general, perceived skills improved for all of those surveyed at post-test, with the skill of asking “quality open-ended questions” showing the most improvement and “using an approach that was supportive and encouraging for the patient to make positive lifestyle changes” showing the least improvement. At follow-up, perceived skills improved for all of those surveyed from baseline, and all but 2 perceived skills improved further from post-test: “quality open ended questions” and “showing respect for patient choice”. The most marked improvement in perceived skills occurred in reflective listening and showing empathy. Additionally, there was a 15% increase in the belief that the encounter they had with the patient will actually lead to positive changes in behavior. The open-ended questions (not included in Table 2) reflected that they would like to work on giving advice that is aligned with patient readiness to make a lifestyle change and asking the patient quality open-ended questions. They felt they did well on listening reflectively to the patient.

Standardized patients MI Skills Reflection - Table 4 shows the perceptions of the standardized patients regarding the residents’ ability to use MI skills and how these perceptions changed among the group from baseline, end-of-intervention, and 3-month follow-up. As noted in the table, the standardized patients felt that the most improvement from baseline to end-of-intervention was made in showing empathy, using a supportive approach, and showing respect. One skill, understanding what the patient values in terms of their health, was rated lower at end-of-intervention. At the 3-month follow-up, standardized patients reported that the resident’s maintained or improved all skills.

Results of Post-Training Survey

Knowledge - Of the 17 medical residents who began the training, all completed it and 13 (81%) participated in the end-of-intervention survey. In four survey items designed to test their knowledge, 12 (92%) of the 13 survey completers were able to identify ambivalence in a patient, how to respond appropriately to ambivalence, and describe the “righting reflex” (i.e. the habit of arguing for change for the patient vs. allowing the patient to argue for change). Ten (77%) of the 13 were able to identify how to appropriately use a readiness scale and describe at least two examples of how to give advice using an MI approach.

Perceptions of Effectiveness of Training - As shown in Table 5 (page 16), 12 (90%) of the residents agreed with most of the statements regarding the effectiveness of the training program. Ten (77%) of the residents felt confident in their abilities to use MI when talking to patients as a result of the MI training and 9 (69%) felt that MI helps them in patient care.

Beliefs about MI in Clinical Practice - Twelve (90%) of residents believed MI offers an advantage over “advice giving”, and 11 (85%) believed “the methods of MI are realistic and usable in daily work”. Approximately 70% agreed that “MI is more effective than traditional advice giving” and that “the methods of MI are time consuming.” Very few residents (n=4, 31%) agreed, “it is difficult to change my ways in the patient-doctor relationship.”

Qualitative Feedback from Residents - At each training session, residents’ comments and suggestions were recorded by the research assistant. The main themes that were evident in these comments were that they intended on using MI in their clinical practice, they believed the training was effective but could use more one-one coaching, the training should be more readily available (i.e. online), and that training should occur earlier in medical education so MI becomes standard practice and not a “new skill.”

TABLE 3:

Resident Reflections at Baseline, End-of-Intervention, and 3-month Follow-Up+

I believe that...	Baseline (n=17)	Post (n=13)	Change from Baseline	Follow-up (n=13)	Change from Baseline
I listened attentively to the patient.	81	92	+11	96	+15
I showed empathy to the patient by acknowledging their emotions, concerns or point of view related to making lifestyle changes.	81	92	+11	96	+15
I asked the patient quality open-ended questions that encouraged them to share what they value in terms of health.	62	77	+14	72	+10
I showed respect for the patient’s right to make his/her own choice, even if I didn’t agree with the choice.	81	92	+11	88	+7
The advice I offered the patient was aligned with what they shared with me in terms of their readiness to make a lifestyle change.	87	92	+5	96	+9
I used an approach that was supportive and encouraging for the patient to make positive lifestyle changes.	75	77	+2	88	+13
The patient will take positive steps to address his/her health risks after this encounter.	69	77	+8	84	+15

+ Numbers represent percent of residents that strongly agreed or agreed with each statement as measured by a 5-point Likert scale.

COMMENT

There were two primary purposes of this study. First, we assessed the effectiveness of an MI training program designed specifically for family practice residents and the possible impact it can have on patient interactions using an objective measure. Second, we assessed changes in perceptions, knowledge, and beliefs about MI in clinical practice. In response to the first purpose statement, we found that an objective measure indicated that residents improved in their use of MI strategies with patients over the course of a training program. While this study design does not permit the inference of causality, these pilot data suggest that the training program could be related to the improvement in the MI skills of the residents.

In response to the second purpose statement, we found that the majority of residents were ready to improve their skills in giving advice that is aligned with a patient’s readiness

TABLE 4:

Standardized patient Reflections at Baseline, End-of-Intervention, and 3-month Follow-Up+

I believe the physician...	Baseline (n=17)	Post (n=13)	Change from Baseline	Follow-up (n=13)	Change from Baseline
Used quality open-ended questions	94	100	+6	100	100
Used reflective listening statements	88	92	+4	95	95
Showed empathy	76	100	+24	100	100
Showed respect for my choices	59	75	+16	83	83
Gave good tailored advice to my needs	82	83	+1	83	83
Understood what I value in terms of my health	88	83	-5	92	92
Used a supportive approach	82	100	+18	100	+18

+ Numbers represent percent of standardized patients that agree with each statement as measured by a dichotomous Agree/Disagree scale

TABLE 5:

Resident Perceptions on the Effectiveness of the Training Program+

Statement	% Agree (n=13)
I believe the MI training was clear in explaining and demonstrating the principles and skills of motivational interviewing.	100
I believe the training was effective in preparing me to deliver MI to patients.	92
I am confident in my abilities to use MI ele- ments when talking to patients as a result of the MI training.	77
The methods of MI from the training help me in my patient care.	69
I understand the principle rules of MI from the training.	100
I feel trained adequately to use MI in daily work.	92

+Numbers represent the percentage of residents that strongly agreed or agreed with each statement as measured by a 5 –point Likert scale

to make a lifestyle change. They had difficulty asking the patient quality open-ended questions, although they demonstrated good reflective listening skills. We learned that more individualized guidance and feedback from the instructor as s/he observed them in role-play scenarios would help the residents improve their skills and confidence in using MI.

A strength of the present study is that we used an objective and verifiable measure to assess the effectiveness of the training program on increasing skills in effective counseling techniques for chronic conditions. Previous studies have not included an objec- tive measure, but have solely relied on self-reported experiences by the students. One limitation of this study is that standardized patients do not respond exactly as real patients would respond; the experience of these medical residents may have been different if they had worked with real patients, who may have presented more challenges. A second limitation is the small sample size (N= 13) of residents who completed the study. While the findings are useful as an initial, pilot-study, research on this topic is needed with a larger sample size of residents and in which a comparison group is utilized.

The findings from this study support the benefit of incorporating MI into the training of residents. Future studies should evaluate a feasible approach by which medical schools could incorporate MI concepts and strategies into the training of all students, including those in years 1 to 4. They should also evaluate the impact that use of MI by a medical resident or physician has on the health out- comes of patients, and not just the skill with which MI is used.

CONCLUSION

The majority of chronic disease is influenced by lifestyle behaviors, yet most physicians don’t receive appropriate training to assist patients in making these appropriate behavioral changes. The evidence is clear that traditional methods of instructing patients to change their behavior do not lead to effective behavior change. Preliminary evaluation of the use of MI appears to enhance a clinician’s skills in communicating with her or his patients and may achieve necessary behavior changes to improve health outcomes.

Training in MI in an early stage of medical education can enhance the clinician’s skills to foster positive changes in the patient’s lifestyle and health status. This may be an economical and efficient strategy to help patients change their behavior to prevent and/or reduce the impact of chronic disease on health care costs and the quality of life for a significant portion of the population.

REFERENCES

1. W. R. Miller, S. Rollnick, Motivational Interviewing: Preparing People for Change, second ed., Guildford Press, New York, 2002.

2. A. Baker, J. Cambridge, MINT: Enhancing engagement in treatment for mental health problems, Behavior Change. 19 (2002) 138-145.

3. E. Britt, S. M. Hudson, N. M. Blampied, Motivational interviewing in health settings: A review, Patient Education and Counseling. 53 (2004) 147-155. http://www.sciencedirect.com/science/article/pii/S0738399103001411

4. J. Wilding, Treatment strategies for obesity, Obesity Reviews. 8 (2007) 137-144. http://onlinelibrary.wiley.com/doi/10.1111/j.1467- 789X.2007.00333.x/pdf

5. S. Rubak, A. Sandbaek, T. Lauritzen, K. Borch-Johnsen, B. Christensen, General practitioners trained in motivational interviewing can positively affect the attitude to behavior change in people with type 2 diabetes, Scandinavian Journal of Primary Health Care. 29 (2009) 1–8. https:// www.ncbi.nlm.nih.gov/pmc/articles/PMC3413190/

6. L. Soderlund, M. Madson, S. Rubak, P. Nilson, A systematic Review of motivational interviewing for general health care practitioners, Patient Education and Counseling. 84 (2011) 16-26. http://www.sciencedirect. com/science/article/pii/S0738399110003873

7. S. Rubak, A. Sandboek, T. Lauritzen, B. Christensen, Motivational interviewing: A systematic review and meta-analysis, British Journal of General Practice. 55 (2005) 305-312. https://www.ncbi.nlm.nih.gov/pmc/ articles/PMC1463134/

8. S. Rubak, A. Sandbaek, T. Lauritzen, K. Borch-Johnsen, B. Christensen, An education and training course in motivational influence: GPs’ professional behavior—ADDITION Denmark, British Journal of General Practice. 56 (2006) 429–36. https://www.ncbi.nlm.nih.gov/pmc/articles/ PMC1839017/

9. R. Saitz, L. Sullivan, J. Samet. Training community-based clinicians in screening and brief intervention for substance abuse problems: translating evidence into practice. Substance Abuse. 21 (2000) 21–31. http://link.springer.com/article/10.1023/A:1007755817351

10. J. Daeppen, C. Fortini, N. Bertholet, R. Bonvin, P. Berney, C. Layat, J. Gaume, Training medical students to conduct motivational interviewing: A randomized controlled trial, Patient Education and Counseling. 87 (2012) 313–318. http://www.sciencedirect.com/science/article/pii/ S0738399111006045

11. F. Haesler, A. Fortin, C. Pfeiffer, C. Walters, S. Martino, Assessment of a motivational interviewing curriculum for year 3 medical students using a standardized patient case, Patient Education and Counseling. 84 (2011) 27–30. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3836602/

12. M. Barwick, L. Bennett, S. Johnson, J. McGowan, J. Moore, Training health and mental health professionals in motivational interviewing: A systematic review, Children and Youth Services Review. 34 (2012) 1786–1795. http://www.sciencedirect.com/science/article/pii/S0190740912001983

13. C. Lane, M. Huws-Thomas, K. Hood, S. Rollnick, K. Edwards, M. Robling, Measuring adaptations of motivational interviewing: the development and validation of the behavior change counseling index (BECCI), Patient Education and Counseling. 56 (2005) 166–173. http://www.sciencedirect. com/science/article/pii/S073839910400076X