Yesterday I wrote about the first morning of this year's BABCP conference and the plenary lecture - " ... Rolls Royce therapy & Anke Ehlers on PTSD (third post)".  Good stuff.  Heart-warming to see such fine, persistent, thoughtful research yielding such encouraging results for PTSD sufferers.  Would that attempts to advance psychotherapy were always this way.  They're not.  Anke Ehlers and David Clark's work - with colleagues - on PTSD, on social anxiety disorder, and on panic disorder are beacons of one way that the psychotherapy field can move forward.  They are the exceptions to the rule.  And the rule is almost certainly that psychotherapy as a field isn't advancing much at all.  As with drug therapy for depression, the results we are getting now are not really much better than the results we used to get ten, twenty, thirty years ago.  I made this point yesterday in the post on Anke Ehlers work.  I'll make it again soon when I post on depression treatment and the Dodo bird hypothesis.  And linked with this - in what for me was the most important talk of this conference, Michael Lambert poured a large bucket of cold water all over his audience in the afternoon plenary "What shall we do about the facts that there are supershrinks and pseudoshrinks?" 

The talk's abstract read: "While cognitive behavior theory and practice place major emphasis on specific interventions to facilitate recovery from psychological disturbance, it is clear that a major contributor to psychotherapy outcome is the specific provider of services, especially in routine care.  A summary of research documenting the size of this effect will be provided, along with evidence about the factors that contribute to variability in patient outcome due to the therapist.  Methods of taking action in this area in order to maximize patient outcome will be discussed.  The roles of training, treatment adherence, and competence will be contrasted with the importance of interpersonal relationships.  Obstacles to identifying outcome as a function of individual therapists will be highlighted and suggestions made for using therapist outcome variability to improve treatment will be made."

So what did Michael, professor of psychology at Brigham Young University in the States, actually say?  Well to put it bluntly, he said that most therapists probably considerably overestimate how effective they are ... not because they're deliberately lying, but because they simply don't know ... they don't have any good personal outcome data to assess how they're doing compared with other therapists.  To make matters worse, therapists' effectiveness at helping clients typically doesn't increase with therapist experience.  Outcome data suggests that, on average, therapists stay about as helpful for their clients after decades of practice as they were early in their careers ... or, more commonly than improving, they actually get rather worse in their results.  Again, because they don't have clear feedback on how they're doing, they don't realise and probably assume that they're getting more effective.  And to make it even tougher, it's hard to know how to change this situation as training and further education doesn't seem to make much difference to average therapist outcomes.  Mm ... large bucket of cold water! 

What does Michael base these very powerful statements on?  Well, as starters, on over a decade of developing and testing methods for assessing and monitoring therapist outcomes on a session by session basis.  So back in 2003, Okiishi, Lambert & colleagues published the paper "Waiting for supershrink: an empirical analysis of therapist effects" with its abstract reading " ... This study examined data collected on 1,841 clients seen by 91 therapists over a 2.5-year period in a University Counseling Center. Clients were given the Outcome Questionnaire-45 (OQ-45) on a weekly basis. After analysing data to see if general therapist traits (i.e. theoretical orientation, type of training) accounted for differences in clients' rate of improvement, data were then analysed again using Hierarchical Linear Modeling (HLM), to compare individual therapists to see if there were significant differences in the overall outcome and speed of client improvement. There was a significant amount of variation among therapists' clients' rates of improvement. The therapists whose clients showed the fastest rate of improvement had an average rate of change 10 times greater than the mean for the sample. The therapists whose clients showed the slowest rate of improvement actually showed an average increase in symptoms among their clients."

Revisiting this territory three years later (this time with a sample of over 5,000 clients) in their paper "An analysis of therapist treatment effects: Toward providing feedback to individual therapists on their clients' psychotherapy outcome" showed a very similar picture.  And Michael said that they now have data on 270 therapists with each therapist having seen between 186 to 1,054 different clients.  This gives one so much more statistical power to assess therapist differences than the tiny patients-per-therapist samples one typically gets in other research studies.  And these aren't rogue findings.  See for example research by other teams such as Lutz & colleagues "Therapist effects in outpatient psychotherapy: a three-level growth curve approach" or Brown & Jones's "Implementation of a feedback system in a managed care environment: What are patients teaching us?" with its data on over 7,000 therapists and its bald conclusion "There are large and stable differences in the effectiveness of clinicians, and outcomes can be improved by referring patients to effective clinicians."  Well that could put a lot of therapists out of work!  These studies don't find significant differences in outcome when analysing results by therapist theoretical orientation.  The differences do seem to be more about variations in interpersonal skills/qualities - see, for example, Anderson et al's paper "Therapist effects: facilitative interpersonal skills as a predictor of therapist success" with its abstract noting "This study examined sources of therapist effects in a sample of 25 therapists who saw 1,141 clients at a university counseling center. Clients completed the Outcome Questionnaire-45 (OQ-45) at each session. Therapists' facilitative interpersonal skills (FIS) were assessed with a performance task that measures therapists' interpersonal skills by rating therapist responses to video simulations of challenging client-therapist interactions. Therapists completed the Social Skills Inventory (SSI) and therapist demographic data (e.g., age, theoretical orientation) were available. To test for the presence of therapist effects and to examine the source(s) of these effects, data were analyzed with multilevel modeling. Of demographic predictor variables, only age accounted for therapist effects. The analysis with age, FIS, and SSI as predictors indicated that only FIS accounted for variance in outcomes suggesting that a portion of the variance in outcome between therapists is due to their ability to handle interpersonally challenging encounters with clients."  It's worth commenting as well that these studies on the differences in results obtained by different therapists show there is variability in how quickly clients tend to improve with different therapists and overlapping but separate variability in how great an improvement clients tend to achieve with different therapists.  In other words, some therapists are very good at helping clients achieve quicker improvements and some therapists are very good at helping clients achieve greater improvements ... and, the ideal, some therapists are very good at doing both (and some therapists ... despite extensive experience in highly evidence-based therapies ... are very bad at helping clients achieve either quick or worthwhile improvement).

Without clear feedback human beings are often very poor at knowing how successful they actually are.  This tendency to misperception applies to nearly all of us.  Michael commented that "90% of therapists interviewed thought they were in the top 25% of effective therapists when compared with their peers ... and no therapist rated themselves as below average in effectiveness."  There have been similar findings with other professions as well.  Many of us will know of the classic US survey where 93% of those assessed thought their driving skills were better than average!  The extensive Wikipedia article on "Illusory superiority" describes just how widespread this bias is.  What can we as therapists do about this?

Michael's suggestions seem to centre around quite demanding organizational therapist assessment systems with clear early warnings elicited when clients aren't responding adequately and well thought out decision trees to follow when outcomes aren't developing as one would hope.  See, for example, his papers "Improving the effects of psychotherapy: the use of early identification of treatment failure and problem-solving strategies in routine practice" and "Outcome measures for practice" and his recent book "Prevention of treatment failure: the use of measuring, monitoring, and feedback in clinical practice".  This is good but not very practical for individual psychotherapists who work on their own or in organizations who won't implement such feedback systems.  Very happily there is at least one good way forward here - using simple session-by-session outcome and alliance assessment measures with clear feedback on whether therapeutic progress is worse than, about average, or better than the expected rate.  The improvement in results using these methods can be stunning.  I'll write more on this soon, but you can begin exploring the territory with the research papers "Using formal client feedback to improve retention and outcomes", "Using client feedback to improve couple therapy outcomes: a randomized clinical trial in a naturalistic setting", "Does a continuous feedback system improve psychotherapy outcome?" & "Effect of client feedback on couple psychotherapy outcomes" and the websites for Scott Miller, Barry Duncan and the Center for Clinical Excellence.

See tomorrow's post for " ... the depression treatment Dodo bird, fathers & child anxiety, and more on couple therapy".