I have written a good deal in the past about variability in the effectiveness of psychotherapists - see, for example, "What shall we do about the fact that there are supershrinks and pseudoshrinks?", "Psychotherapists & counsellors who don't monitor their outcomes are at risk of being both incompetent & potentially dangerous" and other posts on feedback to therapists.

A couple of more recent research studies further underline this issue of therapist variability.  So Saxon & Barkham - in their 2012 paper "Patterns of therapist variability: therapist effects and the contribution of patient severity and risk" (see below for abstracts & links to all papers mentioned) - set out "To investigate the size of therapist effects using multilevel modeling (MLM), to compare the outcomes of therapists identified as above and below average, and to consider how key variables - in particular patient severity and risk and therapist caseload - contribute to therapist variability and outcomes."  Their study sample was made up of 119 therapists and 10,786 patients.  They reported that "Recovery rates for individual therapists ranged from 23.5% to 95.6% ... The size of therapist effect was similar to those found elsewhere, but the effect was greater for more severe patients. Differences in patient outcomes between those therapists identified as above or below average were large ... ".  In a similar way, Kraus & Castonguay in their 2011 study "Therapist effectiveness: Implications for accountability and patient care" looked at outcomes in a sample of 696 therapists and 6,960 patients.  They reported that "Therapists were defined based on whether their average client reliably improved, worsened, or neither improved nor worsened. Results varied by domain with the widespread pervasiveness of unclassifiable/ineffective and harmful therapists ranging from 33 to 65%. Harmful therapists demonstrated large, negative treatment effect sizes (d = - 0.91 to - 1.49) while effective therapists demonstrated large, positive treatment effect sizes (d = 1.00 to 1.52) ... Public policy and clinical implications of these findings are discussed, including the importance of integrating benchmarked outcome measurement into both routine care and training." 

It would be a dangerous disgrace if we didn't respond to these kinds of findings.  If you read the research papers it quickly becomes apparent that these differences in outcome are not primarily about one or other form of psychotherapy being more or less effective than another.  It is primarily about differences in outcome between different psychotherapists (and this isn't explained by qualifications or years of experience).  And part of the crying shame is that the therapists themselves typically don't actually know whether they are doing really well or doing really badly.  This isn't just a problem for psychotherapy.  It is almost certainly an issue for health professionals in general (and workers in other fields too).  As Saxon & Barkham point out "When the going gets tough, the tough get going".  Well actually, that's not quite what they say ... but they do point out that the differences between more & less effective therapists show more when the clients involved are more severely troubled.  This fits with other findings from the literature, including with the variability found in client premature termination & dropout rates.

In a third recent research paper, "Practice recommendations for reducing premature termination in therapy", Swift et al highlight that a meta-analytic review of "669 studies found that approximately 20% of all clients drop out of treatment prematurely, with higher rates among some types of clients and in some settings."  They go on to "review 6 practice strategies for reducing premature termination in therapy. These strategies include providing education about duration and patterns of change, providing role induction, incorporating client preferences, strengthening early hope, fostering the therapeutic alliance, and assessing and discussing treatment progress."  As a fan of taking client feelings about therapy very seriously & monitoring whether or not therapeutic progress is developing at an effective rate, these recommendations by Swift et al make very good sense.  Thompson & McCabe in a fourth recent paper, "The effect of clinician-patient alliance and communication on treatment adherence in mental health care: a systematic review", highlight the unsurprising importance of therapist-client communication styles in reducing dropout.

As I've mentioned already, Kraus & colleagues emphasise the potential of "integrating benchmarked outcome measurement into both routine care and training."  How else would we really know how we're doing as therapists?  And no, simply relying on experience & intuition really isn't good enough as the research has shown.  This is where the fifth & last of the recent papers, that I'm commenting on in this post, comes in.  Very interestingly, de Jong et al in "Understanding the differential impact of outcome monitoring: Therapist variables that moderate feedback effects in a randomized clinical trial" agree that "Providing outcome monitoring feedback to therapists seems to be a promising approach to improve outcomes in clinical practice."  However they take this a stage further by examining whether the effectiveness of feedback "is moderated by therapist characteristics."  Sure enough, in their study they found "feedback was effective for not-on track cases for therapists who used the feedback. Internal feedback propensity, self-efficacy, and commitment to use the feedback moderated the effects of feedback. The results demonstrate that feedback is not effective under all circumstances and therapist factors are important when implementing feedback in clinical practice."  Do look at my earlier post "Psychotherapists & counsellors who don't monitor their outcomes are at risk of being both incompetent & potentially dangerous" if you're interested in exploring this further.  I know for myself how easy it is to drag one's feet even when the data is so strong.  I certainly have a strong tendency to just continue to practise in much the same ways as I have done for many years.  We can get better at what we do though.  So I've enrolled at the "Feedback informed treatment" website and just booked into the "Achieving clinical excellence" conference in Amsterdam next May.  For more about feedback, see too tomorrow's post "The challenge of receiving (and giving) feedback in both professional & personal environments". 

Saxon, D. and M. Barkham (2012). "Patterns of therapist variability: therapist effects and the contribution of patient severity and risk." J Consult Clin Psychol 80(4): 535-546.  OBJECTIVE: To investigate the size of therapist effects using multilevel modeling (MLM), to compare the outcomes of therapists identified as above and below average, and to consider how key variables--in particular patient severity and risk and therapist caseload--contribute to therapist variability and outcomes. METHOD: We used a large practice-based data set comprising patients referred to the U.K.'s National Health Service primary care counseling and psychological therapy services between 2000 and 2008. Patients were included if they had received >/=2 sessions of 1-to-1 therapy (including an assessment), had a planned ending to treatment, and completed the Clinical Outcomes in Routine Evaluation-Outcome Measure (CORE-OM; Barkham et al., 2001; Barkham, Mellor-Clark, Connell, & Cahill, 2006; Evans et al., 2002) at pre- and post-treatment. The study sample comprised 119 therapists and 10,786 patients, whose mean age was 42.1 years (71.5% were female). MLM, including Markov chain Monte Carlo procedures, was used to derive estimates to produce therapist effects and to analyze therapist variability. RESULTS: The model yielded a therapist effect of 6.6% for average patient severity, but it ranged from 1% to 10% as patient non-risk scores increased. Recovery rates for individual therapists ranged from 23.5% to 95.6%, and greater patient severity and greater levels of aggregated patient risk in a therapist's caseload were associated with poorer outcomes. CONCLUSIONS: The size of therapist effect was similar to those found elsewhere, but the effect was greater for more severe patients. Differences in patient outcomes between those therapists identified as above or below average were large, and greater therapist risk caseload, rather than non-risk caseload, was associated with poorer patient outcomes.

Kraus, D. R., L. Castonguay, et al. (2011). "Therapist effectiveness: Implications for accountability and patient care."  Psychotherapy Research 21(3): 267-276. Significant therapist variability has been demonstrated in both psychotherapy outcomes and process (e.g., the working alliance). In an attempt to provide prevalence estimates of "effective" and "harmful" therapists, the outcomes of 6960 patients seen by 696 therapists in the context of naturalistic treatment were analyzed across multiple symptom and functioning domains. Therapists were defined based on whether their average client reliably improved, worsened, or neither improved nor worsened. Results varied by domain with the widespread pervasiveness of unclassifiable/ineffective and harmful therapists ranging from 33 to 65%. Harmful therapists demonstrated large, negative treatment effect sizes (d = - 0.91 to - 1.49) while effective therapists demonstrated large, positive treatment effect sizes (d = 1.00 to 1.52). Therapist domain-specific effectiveness correlated poorly across domains, suggesting that therapist competencies may be domain or disorder specific, rather than reflecting a core attribute or underlying therapeutic skill construct. Public policy and clinical implications of these findings are discussed, including the importance of integrating benchmarked outcome measurement into both routine care and training.  

Swift, J. K., R. P. Greenberg, et al. (2012). "Practice recommendations for reducing premature termination in therapy." Professional Psychology: Research and Practice 43(4): 379-387.  Premature termination from therapy is a significant problem frequently encountered by practicing clinicians of all types. In fact, a recent meta-analytic review (J. K. Swift & R. P. Greenberg, 2012, "Premature discontinuation in adult psychotherapy: A meta-analysis." Journal of Consulting and Clinical Psychology.) of 669 studies found that approximately 20% of all clients drop out of treatment prematurely, with higher rates among some types of clients and in some settings. Although this dropout rate is lower than previously estimated, a significant number of clients are still prematurely terminating, and thus further research toward a solution is warranted. Here we present a conceptualization of premature termination based on perceived and anticipated costs and benefits and review 6 practice strategies for reducing premature termination in therapy. These strategies include providing education about duration and patterns of change, providing role induction, incorporating client preferences, strengthening early hope, fostering the therapeutic alliance, and assessing and discussing treatment progress.

Thompson, L. and R. McCabe (2012). "The effect of clinician-patient alliance and communication on treatment adherence in mental health care: a systematic review." BMC Psychiatry 12(1): 87. (Free full text available) BACKGROUND: Nonadherence to mental health treatment incurs clinical and economic burdens. The clinician-patient relationship presents a point of intervention. This alliance is negotiated through clinical communication. However, recent medical reviews of communication and adherence exclude studies of psychiatric patients. The following review examines the impact of clinician-patient alliance and communication on adherence in mental health and the specific mechanisms that result in patient engagement. METHODS: In December 2010, a systematic search was conducted in Pubmed, PsychInfo, Web of Science, Cochrane Library, Embase and Cinahl and yielded 6672 titles. A secondary hand search was performed in relevant journals, grey literature and reference. RESULTS: 23 studies met the inclusion criteria for the review. The methodological quality overall was moderate. 17 studies reported positive associations with adherence, only four of which employed intervention designs. 10 studies examined the association between clinician-patient alliance and adherence. Subjective ratings of clinical communication styles and messages were assessed in 12 studies. 1 study examined the association between objectively rated communication and adherence. Meta-analysis was not possible due to heterogeneity of methods. Findings were presented as a narrative synthesis. CONCLUSIONS: Clinician-patient alliance and communication are associated with more favourable patient adherence. Further research of observer rated communication would better facilitate the application of findings in clinical practice. Establishing agreement on the tasks of treatment, utilising collaborative styles of communication and discussion of treatment specifics may be important for clinicians in promoting cooperation with regimens. These findings align with those in health communication. However, the benefits of shared decision making for adherence in mental health are less conclusive than in general medicine.

de Jong, K., P. van Sluis, et al. (2012). "Understanding the differential impact of outcome monitoring: Therapist variables that moderate feedback effects in a randomized clinical trial." Psychotherapy Research 22(4): 464-474.  Providing outcome monitoring feedback to therapists seems to be a promising approach to improve outcomes in clinical practice. This study aims to examine the effect of feedback and investigate whether it is moderated by therapist characteristics.  Patients (n=413) were randomly assigned to either a feedback or a no-feedback control condition. There was no significant effect of feedback in the full sample, but feedback was effective for not-on track cases for therapists who used the feedback. Internal feedback propensity, self-efficacy, and commitment to use the feedback moderated the effects of feedback. The results demonstrate that feedback is not effective under all circumstances and therapist factors are important when implementing feedback in clinical practice.

This whole issue of feedback really interests me at the moment.  Click for three other recent posts I've written on this subject.