National Dissemination of an EvidenceBased Practice Supported Employment

National Dissemination of an Evidence-Based Practice: Supported Employment in VHA Sandra G. Resnick, Ph. D. VA Northeast Program Evaluation Center, New England MIRECC, and Yale University School of Medicine

Supported Employment n Principles n n n Goal is competitive employment Zero-exclusion Rapid, individualized job search Long-term follow along support Integration with Mental Health Team Considered an EBP n n n Consistent findings in RCTs Well defined model Validated fidelity scale

VACO – NEPEC Partnership n Legislation passed/money allocated in 2003 n Funds provided to VACO n n n Clinical initiative n NOT research NEPEC n CWT monitoring since 1995 n Experience in dissemination of innovative practices Decisions made in partnership: n VACO: all sites will have SE (~130 sites) n NEPEC: centrality of fidelity monitoring n VACO supportive of QUERI grant application

Implementation Plan: Phase 1 n “Mentor-trainer” n n n One per VISN (21) Money for a total of 3 – 4 FTEE 1 Mentor Trainer (MT) n Non-MT n n n sites 1 FTEE Trained by MT National consultants n Train 21 sites (not MTs) n Rich Toscano Mary Sperrazza (local) n

Good Day Assumptions 1. MTs from good sites will provide good training (corollary: MTs will do what Rich does) 2. 3. 4. Fidelity will be used as a QI tool MTs will learn fidelity assessment Programs will benefit from telephone supervision

Bad Day Assumption Impossible, unfeasible plan doomed for failure

What happened?

Activities: 21 MT Sites n Kick-off meetings n n Louisville - September 2004 Palo Alto - March 2006 most between June – November 2005 Fidelity every six months n All sites (no refusals) n Complete data for 2 years

MT Fidelity: November 2008 9 of 21 sites at full implementation Consistent with SE Fair SE Implementation Not SE 52. 2 60. 8 63. 5 64. 6 64. 2 64. 8 n = 20 n = 17

Any Competitive Work at 21 MT sites (last 90 days) number of days worked n = 45 n = 241 n = 456 n = 563 n = 745 M = 39 M = 41 M = 45 M = 40 M = 46 SD = 22 SD = 29 SD = 24 SD = 25 SD = 22

What did we learn?

Not an impossible, unfeasible plan doomed for failure n Some MT sites are implementing SE well n n 9 sites at full implementation Mean fidelity score around 65 Veterans are getting jobs n Some sites hiring extra CWT/SE staff n but… faulty assumptions n

1. Good sites will provide good training n Implementation and training are different skill sets n n Giving feedback Earning respect Assessment Good MTs have good supervisors n n Role models? Hire better people?

2. Fidelity will be used as a QI tool 3. MTs will learn fidelity n Fear factor n n n Implementation, training AND fidelity assessment are all different skill sets n n Teaching to the score Where are the principles? Asking open-ended questions Assessing not teaching Content Unrealistic expectations?

number of calls 4. Sites will use telephone supervision (3 years) site

What now? : Phase 2 n Train the MTs n West Haven n n With Rich n n n New application of MI (Instructional Interviewing? ) How to teach SE principles Focus on positives Fidelity Technical assistance What leads to sustainability?