Use of the Outcome Questionnaire45 in a Psychology

Use of the Outcome Questionnaire-45 in a Psychology Training Clinic John Fitzgerald, Ph. D. The Psychology Centre, Hamilton Acknowledging the conscientious support of Philippa Thomas and Jessica Taylor, and all the interns and students who have been on placement with us at the Centre.

Measuring Outcomes and Monitoring Change • Trauer (2010), in Outcome Measurement in Mental Health, suggests we measure outcomes to, • Provide information to managers • Assist in decision making • Incorporate the consumer’s perspective • These are often limited in their client focus, actuarial, and macrooriented. • In training, as in routine clinical practice, we strive to focus more on, • Providing data to therapists and their clients • Supporting collaborative therapy decisions • Facilitating client engagement • This is more about recognising and monitoring change. • Lambert (2010), in Prevention of Treatment Failure, focuses primarily on the collection and use of clinical data aimed at improving the effects of psychotherapy, and the prevention of poor/negative outcomes. This is what we are training students to understand apply.

How Good are we at Measuring Therapeutic Outcomes? • Hatfield & Ogles (2004) Surveyed 2, 000 clinical practitioners in APA 37% used some form of outcome assessment Most common were BDI, GAS, CBCL – these are NOT outcome measures OQ-45 (Lambert, et al. , 1994) only used by 6% of respondents Why? Too much paper work, time consuming, extra burden on the client • Bickman, Rosof-Williams, Salzer, et al. , (2000) Surveyed 539 C&A mental health workers in USA Few rated standardised outcome data as desirable or valued Psychologists no different from other groups • Outcome assessment is an ongoing process, not one that should be relegated to the final session.

More Reasons to be Concerned About Measuring Change. • Code of Ethics for Psychologists Working in Aotearoa/New Zealand (2002) • Principle 2: Responsible Caring • Practice Implication 2. 1. 8: When it is clear that the client is not benefitting, a psychologist initiates a termination of the professional relationship. • How do we judge that “the client is not benefitting”? Measuring outcomes does nothing to address this imperative. • Core Competencies for the Practice of Psychology in New Zealand (2009) • Clinical Scope: Intervention • Knowledge – “Knowledge of how to critically evaluate interventions and modify them when change is required” • Skills – “Critical evaluation of strengths, weaknesses and limitations during interventions” (emphasis added)

Do No Harm! • There is an “assumption … that as psychotherapy is only talking … no possible harm can ensue. ” (Nutt & Sharpe, 2008) • An increasing body of research shows us that about 10% of clients get worse after starting therapy (Jarrett, 2008) • When is a procedure harmful? (Francis, 2009) • when it is significantly detrimental • when it does not do what it purports to do • inaction in the face of imminent harm • Do we have adequate monitoring strategies for recognising each of these eventualities?

Outline of the project • Purposes • Model good practice processes for students/interns (Scientist. Practitioner) • Integrate standardised monitoring procedures into individualised case management • Establish a practice base for future consultation (DHB, PHO, etc. ) • Standard Measures • Outcome Questionnaire-45 introduced when managing a brief therapy evaluation for Waikato PHO, January 2008 • Helping Alliance Questionnaire-II and Patient Health Questionnaire-4 since January 2011. • Took several months for processes to ‘bed-in’ • Measures not routinely collected at commencement • Difficulty integrating with case-specific measures • Limited data management systems and data integration

• Outcome Questionnaire - 45 (OQ 45. 2; Lambert, Hansen, Umphress et al. , 1996) – 45 item self-report questionnaire exploring Symptom Distress (anxiety, affective disorders, adjustment disorders, and stress), Interpersonal Relations (e. g. loneliness, conflict, and family problems), and Social Role (difficulties in role as worker, homemaker, student, etc). • Helping Alliance Questionnaire - II (HAq-II; Luborsky, Barber, Siqueland, et al. , 1996) – the standard client/therapist completed measure against which newer measures are evaluated. The HAq-II has 19 item client and therapist rated forms. • Patient Health Questionnaire for Depression and Anxiety - 4 (PHQ-4; Kroenke, Spitzer, Williams, & Löwe, 2009). 4 item criterion-referenced screening tool used to identify elevated levels of either anxiety (2 questions, GAD-2) or depression (2 questions, PHQ-2). The PHQ-4 is part of a series of abbreviated measures developed by Spitzer and colleagues, and is a public domain instrument. Measures and protocol Session Measure 1 OQ 45, PHQ-4 2 HAq-II 3 OQ-45 4 - 5 OQ 45, HAq-II Every 2 nd session End of therapy OQ 45, HAq-II, PHQ-4 Measures are administered at the start of the session, except for the first session.

Descriptive Results • N=241, clients in our OQ 45 database. This is about 80% of clients seen - excludes age<18 yrs, clients seen for ‘assessment only’, ACC referrals. • Age: mean=36. 4 y, SD=12. 9 y, range 18 -77 y. • Gender: Male=72(30%), Female=169(70%) • 198 (82%), clients completed service. 43 clients still ‘open’. • Overall DNA/Canc rate=20. 6%

All clients ‘Closed’ Cases N=198 Total attended Mean (SD) Range Total DNA Mean (SD) Range Male n=56 (28%) Total attended Mean (SD) Range Total DNA Mean (SD) Range Female n=142 (72%) Total attended Mean (SD) Range Total DNA Mean (SD) Range 1, 852 9. 4 (6. 7) 0 -38 483 2. 4 (3. 0) 0 -17 494 8. 8 (6. 6) 0 -29 118 2. 1 (2. 8) 0 -12 1, 358 9. 6 (6. 8) 0 -38 365 2. 6 (3. 1) 0 -17 AMH clients (n=64, 32%) Total attended Mean (SD) Range Total DNA Mean (SD) Range 703 10. 1 (5. 6) 1 -26 170 2. 7 (3. 0) 0 -9 236 10. 3 (5. 1) 2 -21 50 2. 2 (2. 3) 0 -9 467 11. 4 (6. 0) 1 -26 120 2. 9 (3. 4) 0 -12 Community clients (n=134, 68%) Total attended Mean (SD) Range Total DNA Mean (SD) Range 1, 149 8. 6 (3. 0) 0 -38 313 2. 3 (3. 0) 0 -17 258 7. 8 (7. 4) 0 -29 68 2. 1 (3. 1) 0 -12 891 8. 8 (7. 0) 0 -38 245 2. 4 (2. 9) 0 -17

Cross-Cultural norms • de Jong, Nugter, Polak, et al. (2007) have explored the validity of the OQ-45 in Holland. American samples (Lambert et al. , 2003) Dutch samples (de Jong et al. , 2007) Community Clinical The Psychology Centre Community referred AMH referred All clients SD 25. 4 (12) 49. 4 (15) 22. 2 (10) 48. 9 (16) 46. 6 (12. 9) 48. 6 (13. 2) 47. 3 (13. 0) IR 10. 2 (6) 19. 7 (6) 8. 4 (5) 16. 8 (7) 17. 3 (6. 4) 18. 7 (5. 6) 17. 8 (6. 2) SR 9. 6 (4) 14. 1 (5) 8. 1 (3) 13. 6 (6) 13. 2 (5. 0) 13. 7 (5. 2) 13. 3 (5. 1) Total 45. 2 (19) 83. 1 (22) 38. 7 (16) 81. 0 (20. 5) 78. 4 (20. 4) 79. 5 (25) 77. 0 (20. 3)

OQ-45 scale means and SDs All clients In Out AMH clients In Out Community clients In Out Symptom Distress 47. 3 (13. 0) Interpersonal Relations 17. 8 (6. 2) 13. 6 (6. 4) 18. 7 (5. 6) 15. 5 (6. 3) 17. 3 (6. 4) 12. 4 (6. 2) Social Role 13. 3 (5. 1) 9. 8 (4. 8) 13. 7 (5. 2) 10. 1 (4. 9) 13. 2 (5. 0) 9. 5 (4. 7) Total Score 78. 4 (20. 4) 81. 0 (20. 5) 32. 5 (13. 9) 55. 8 (23. 1) 48. 6 (13. 2) 35. 3 (14. 2) 60. 9 (23. 2) 46. 6 (12. 9) 77. 0 (20. 3) 30. 8 (13. 4) 52. 7 (22. 5) • Clinical threshold scores: SD=36, IR=15, SR=12, Total=63 • Reliable Change Index (RCI) for OQ-45 Total Score is 14 points • All scale In-Out comparisons within each client grouping were statistically significant • Intake SR was the only significant difference between genders (t=-2. 17, p=. 03) • Mean change in the Total Score for the three groups were 21. 9, 18. 7, 23. 9 respectively. Each of these is greater than the RCI threshold.

Progress Monitoring • Available on-line OQ-Analyst assists in the monitoring of client progress, and helps identify when therapy is not progressing … (we do not use this … yet). • However, it has been useful in tracking progress (RH), relapse (GD), and the impact of specific events in treatment (KS and RK). • OQ 45 scores are discussed in supervision, and with the client. • OQ 45 scores are mapped as one way of monitoring and highlighting change.

Helping Alliance Questionnaire-II • Commenced use in 2011 because of clear reported association between therapy outcomes and the quality of therapeutic relationship (see Crits-Christoph & Gallop, 2006; Lambert, 2010), which we had no formal way of monitoring. • Proving difficult to collect. Wording of scale is ‘difficult’ in places (e. g. , “From time to time we both talk about the same important events in my past”) • Intake: n=40, • Client mean=5. 25(0. 4), range 4. 5 -5. 9 • Therapist mean=4. 82(0. 4), range 4. 0 -5. 5 • Outcome: n=24, • Client mean=5. 43(0. 4), range 4. 5 -6. 0 • Therapist mean=5. 17(0. 4), range 4. 3 -5. 8 • Suggested interpretation – • Client means increase during course of therapy 5. 15 -5. 30 • Therapist means increase 4. 63 -4. 92 • We have used a threshold of 1 SD either side of the mean as a ‘flag’ • • Intake – client rating (quality of therapeutic alliance) – 2 low, 5 high Intake – therapist – 9 low, 0 high Outcome – client – 1 low, 7 high Outcome – therapist – 2 low, 1 high

OQ 45 x HAQ-II • We need to be cautious not to assume causality, and be aware that our sample is still small. • At the point of intake there was a negative association between the OQ 45 Symptom Distress subscale (and Total Score) and therapists rating of the alliance (r=-0. 51). • At the point of outcome there was a negative association between the client ratings on the HAq-II and OQ 45 Social Role at intake (r=-0. 47) and outcome (r=-0. 45), and Symptom Distress at outcome (r=-0. 56). • At the point of outcome there was a negative association between therapist ratings on the HAq-II and all OQ 45 subscales at outcome. • All associations were significant at p=<. 05, and all contributed to significant relationships between the HAq-II and the OQ 45 Total Score.

What We Have Learned So Far … Process • Collecting standardised monitoring data is difficult • Of 512 OQ 45 s, 53 (10. 3%) contained errors. • Pro-rating errors - 22 (4. 2%), largely resolved with training. • Arithmetic errors – 31 (6. 1%), ongoing. • Collating and managing progress data in ‘real-time’ is hard, but important. • What helps? • Using the data (this is very IMPORTANT) • Integration of data into supervision • Engaging with the client

What We Have Learned So Far … • Our norms for the OQ 45 are broadly comparable to those found overseas … so we will continue using it • The therapy that our students/interns are able to offer seems to be generally effective (in reducing OQ 45 scores), Consumer Feedback suggests it is also useful to our clients. • There are some interesting associations between the OQ 45 and the HAq-II scores. While not particularly happy with the HAq-II we are keen to continue monitoring Therapeutic Alliance, and working with it as a metric within therapy.