Independent Evaluation of Victorian Safewards Trial Bridget Hamilton

Independent Evaluation of Victorian Safewards Trial Bridget Hamilton, Justine Fletcher, Brian Mc. Kenna, Catherine Roper, Natisha Sands & Steve Elsom contact: bh@unimelb. edu. au

Safewards Background www. safewards. net

Safewards Model (Bowers 2013)

Safewards identifies six areas or domains which can influence or trigger conflict Domains 1. The Patient Community 2. Patient Characteristics 3. Regulatory Framework Flashpoint 4. Staff Team 5. Physical Environment 6. Outside Hospital Flashpoint Definition Social and psychological situations arising out of features of the originating domains, signaling and preceding imminent conflict behaviours

Safewards interventions Are a set of specific prevention and intervention strategies, developed to correspond to diverse flashpoints identified in the model

Victorian seclusion & restraint data Adult Seclusion and Restraint per 1000 Bed Days 30 Events Per 1000 OBD 25 20 15 Adult Seclusion Adult Restraint 10 5 0 FY 08/09 FY 09/10 FY 10/11 FY 11/12 Financial Year FY 12/13 FY 13/14 FY 14/15 ytd

Safewards Victorian Trial

Safewards Victorian Trial 7 health services Metropolitan 18 wards 4 service types Regional Adolescent/Youth Adult SECU Aged

Safewards Victorian Trial Project objectives • Trial the Safewards model in a range of Victorian mental health services using a partnership approach • Evaluate the trial to: – Determine applicability of the Safewards UK model within Victorian mental health service context – Develop recommendations to the department regarding adaptation of the Safewards model to fit the Victorian mental health service context – Determine the model’s effectiveness in reducing restrictive interventions and enhancing consumer and carer experiences of care and staff safety • Develop resources and recommendations to support a broader implementation of Safewards across the state

Safewards interventions Clear mutual expectations Soft words Talk down Positive words Bad news mitigation Know each other Mutual help meeting Calm down methods Reassurance Discharge messages

Safewards Victorian Trial Training and preimplementation support Implementation Embedding within services Evaluation Evaluate against objectives for: Learning Practice change Experience Effectiveness Sustainability

Safewards Victorian Trial Training and preimplementation support Implementation Embedding within services Evaluation Evaluate against objectives for: Learning Practice change Experience Effectiveness Sustainability

Independent evaluation by Centre for Psychiatric Nursing

Safewards Victorian Trial timeline Training • November 2014 – February 2015 Trial • March 2015 – May 2015 Sustainability • June 2015 – May 2016

Evaluation stages & data sources Training evaluation Pre-training survey Post training survey Train-the-trainer survey Training & Implementation diary Implementation evaluation Safewards readiness checklist Safewards fidelity checklist Focus groups with staff: 1 adult, 1 youth/adolescent, 1 aged persons, 1 SECU Focus group with Consumer consultants Sustainability evaluation Consumer feedback survey Staff feedback survey CMI data Fidelity checklist

Evaluation findings Safewards training

Training outcomes Paired difference Total Knowledge Total Confidence Total Motivation Mean Std. 95% confidence Deviation interval t df Sig. (2 tailed) effect size cohen's d 15. 78 12. 58 13. 12 - 18. 45 11. 77 87 . 000 1. 25 13. 87 13. 84 10. 92 - 16. 82 9. 35 86 . 000 1. 00 7. 44 13. 70 4. 51 – 10. 38 5. 04 85 . 000 0. 54 • Mode of training – 42% whole day & 42% 1 hour in-service • 35% attended one event • 51% attended between 2 -5 events

Enablers & barriers to training Enablers • Previous training • Support processes • Champions • Staff accepting & identifying with Safewards intentions • Cross disciplinary collaboration Barriers • Resistance • Training materials • Language

Evaluation Findings Safewards Implementation

Readiness at the start of the trial Well Prepared Somewhat Prepared Under Prepared Training complete 7 -10 interventions Training complete for most interventions Little /no preparation in two of; -training -champions -implementation planning either/or not both Champions identified 7 -10 interventions Champions identified for most interventions Implementation prepared for 7 -10 interventions Implementation prepared for 3 -6 interventions 3 Wards 5 Wards 10 Wards

Focus groups Nurses connect with the Safewards model & aims: “there's a lot of alignment with the Safewards model, so they kind of they get it which is great, ” “even the people who’ve been most resistant to the interventions, they're not resistant to the whole philosophy and the whole ethos and Safewards approach, they might have a disagreement with a certain intervention, and a certain element of it, you know, be it language, be it posters on the wall”

Focus groups – Wins with implementation Many consumers readily take up the KEO, MHM, DM • Some committed champions are ongoing in each ward • “I’m a bit of a believer you … can’t change things without actually you know sweeping up those with you, they have to be convinced that it’ll work too” Safewards has refreshed existing group programs • “mutual help meetings for example we already had community meetings by adapting those and kind of shifting the focus not on this is what's on today, but what can we do for each other, very easy to implement, didn’t really take much time at all. By building on those things, getting the buy in from patients you know, add on to the inclusiveness” Lots of local resources = extra posters, orientation folders • “We’ve got a small orientation manual for new and temporary staff, which just kind of goes through the basics of what Safewards is and who the champions are and what the roles are …you know just like the article and … a quick easy reference for people to access. ” Changes to ward culture • “Mutual respect… all the different interventions are basically saying the same thing which is treat people as you would want to be treated. ” • “More transparency and information given to patients” “Things that are trending downwards are complaints… far less complaints about staff attitude …and lack of information. “ increase in the number of individual nurse and clinician complements. ”

Focus groups – service specific changes Safewards Victorian Adaptions • Adult - fit with other RRI interventions • Youth - fit with group programs, language in resources • Aged - messages of encouragement, family carer focus to KEO • SECU - surprisingly few changes Question to ask: • Is adaption of an intervention an enhancement or a mutation of the Safewards intervention, does it match the Model?

Evaluation findings Safewards Implementation Fidelity

Fidelity means… …the extent to which an intervention is being done • In the case of Safewards: – Fidelity checklist scores each of the 10 items from 0 10 – Potential total score is 100 – Wards in the UK trial achieved 60 -70 by 8 weeks

Fidelity scores 100 90 Fidelity Score 80 70 60 Overall 50 Metropolitan 40 Regional 30 20 10 0 Time 1 Time 2 Time 3 Time 4 Time 5 Time 6

Fidelity of Safewards implementation 100 90 80 70 60 50 40 30 20 10 0 adult (10) Time 1 adolescent/youth (3) Time 2 Time 3 Time 4 aged (3) Time 5 SECU (2) Time 6

Fidelity scores: Individual youth, aged & SECU services 600 500 400 300 200 100 0 91 100 94 91 95 91 90 90 95 75 55 58 60 62 48 73 17 22 1 2 3 87 87 88 92 90 89 75 100 99 98 86 77 Time 6 90 60 97 80 61 85 68 50 58 4 5 6 7 Time 5 85 65 55 38 35 43 8 Time 4 Time 3 Time 2 Time 1

Fidelity scores: Individual adult services 600 500 97 400 90 300 200 97 90 100 0 98 76 31 1 95 95 94 87 65 89 80 86 95 55 53 43 84 74 84 65 31 1 40 2 3 4 5 65 60 30 26 0 6 89 94 76 77 95 76 74 46 43 7 8 45 48 0 9 70 86 10 Time 6 Time 5 Time 4 Time 3 Time 2 Time 1

Safewards Evaluation Seclusion Rates

Outcomes & measures • Seclusion rate = key measure of containment • Same as UK trial, also fits with RRI • No long range measure of restraint • No routine data re PRN, medication by injection • No measure of conflict (or of flashpoints)

14 12 10 8 non-trial 6 trial 4 Linear(non-trial) 2 Linear(trial) co n p -u w llo fo so lid at io n ria l st -t tri al po e_ tri al 0 pr seclusions per 1000 occupied bed days Seclusions per 1000 occupied bed days comparing pre and post-trial phases with Safewards sites and other sites

Seclusions per 1000 occupied beddays: Adult wards 30 25 20 trial 15 non-trial 10 5 0 pre_trial post-trial consolidation follow-up

Seclusions per 1000 occupied beddays: Youth wards 20 18 16 14 12 trial 10 non-trial 8 6 4 2 0 pre_trial post-trial consolidation follow-up

Safewards Evaluation Consumer experiences & feedback

Consumer survey data “instead of a nurse saying ‘I'm telling you the rules, this is what they are, and then conflict arising, . . . Staff can refer people to what is on the walls, go and check those mutual expectations, this is what we can guarantee you as staff”. “ the other thing that I think is a real culture shift is the baseline anxiety level and fears, and that sense of threat and aggressive behaviour and all the rest of it, has really changed. And mutual help is a big part of that. ”

Consumer perspectives continued. . . There were nurses on my unit who were obviously opposed to Safewards Nurses on my unit were generally positive about Safewards Staff and consumers are on a more even standing now I feel more connected with the nursing staff I feel more positive about being on the ward I feel safer 0% Never Rarely 20% 40% 60% 80% 100% 120% Sometimes Usually Always

Safewards Evaluation Staff experiences & feedback

Staff survey results – view of culture change • “works well to reduce detachment between patients & staff and to build rapport” know each other environment where there is a lot of negativity, utilising positive words (especially during handover & with other staff) created a more professional, supportive & positive workplace” • “This is awesome because it makes it about the patient & empowers them to have a say, & to be part of what goes on around them, during a time when a lot of choice is taken away” positive words mutual help meeting • “In an inpatient

Staff perspectives continued. . . Property damage Absconding Physical Conflict Verbal Conflict 0% Never 20% Rarely 40% Sometimes 60% Usually Always 80% Unsure 100%

Safewards implementation & sustainability

People Knowledge Functional Support Safewards implementation ingredients • Senior buy in initially & actions to address implementation barriers >2 mid-level change agents present in ward: educator/s, NUM or ANUM >1 key other: allied health, medical, consumer consultant active >1 interventions Champions appointed & active on the ward (number not determined) • Strong understanding of the model & interventions in the change agents Grasp of key concepts & model in majority (>50%) of ward based staff Local access to training, a plan to fit with mandatory training Resources & schedule to train (& ideally test) knowledge Dedicated budget for intervention items Local adaptions of interventions to fit interventions with existing ward space, practices, documents, daily & weekly routines, policies and KPIs Timeframe & people to monitor fidelity, measure of conflict & feedback •

so…. ? Contact: bh@unimelb. edu. au