CSUN 2017 02 March 2017 Live Well RERC

CSUN 2017 02 March 2017 Live. Well RERC Emerging ICT Developments for Independent Living Frank De. Ruyter – Duke University Leighanne Davis – Duke University Mike Jones – Shepherd Center John Morris – Shepherd Center National Institute on Disability, Independent Living, and Rehabilitation Research (NIDILRR) Agency for Community Living, US Department of Health and Human Services

Partner Institutions National Institute on Disability, Independent Living, and Rehabilitation Research (NIDILRR), Agency for Community Living, US Department of Health and Human Services 2

The Team… 3

Twofold Mission § Promote ICT access to existing and emerging technologies for all people regardless of ability § Develop and validate ICT applications to improve the capacity for independent living and community participation. 4

Live. Well RERC Projects

R 1 – Discovery of User Needs Purpose § Identify needs & preferences for community living of people with disabilities that may be addressed by ICT apps Activities § Survey of user needs § “Blue Sky” focus groups Deliverables § Published survey results of user needs and preferences § Use-case scenarios and evaluation of impact

https: //www. surveymonkey. com/r/m. Health. Tech

R 2 – Factors Affecting Acceptance of ICT Purpose § Identify facilitators & barriers to acceptance of wearable, sensor and monitoring technology by people with disabilities and their caregivers Activities § Paired-choice preference assessment to weigh tradeoffs and benefits of different ICT solutions Deliverables § Guidelines for balancing benefits and tradeoff of different ICT applications

R 3 – Improving safety & activity independence in the home & community following TBI Purpose § Identify needs that would benefit from new data sources & technology support § Evaluate validity & reliability of sensor-produced data Activities § Semi-structured focus groups with TBI & family § Usability testing in 16 homes Deliverables § Identify most useful/usable technologies to integrate with health coaching platform § Complete field testing of full protocol

Target Population Inclusion Criteria § Sustained moderate to severe TBI § Aged 18 years or older § Living in the home & community § Able to self-manage at least 75% of basic self-care § Caregiver/Family member

Phase 1 Design/Methodology Interviews & Focus Groups To identify: § Technology needs & interests § Perceived benefits § Usability & comfortability § Privacy concerns regarding shared information § Suggested improvements on existing technology solutions

Demographics Study Population (n=11) Person with TBI CG/Family member 6 5 Male 4 - Female 2 5 30. 0 (13. 9) 50 (4. 6) Black or African American 1 - White or Caucasian 4 4 Hispanic 1 1 MVA 4 - GSW 1 - Other 1 - 22. 0 (4. 5) - # of Participants (n) Gender Age, mean years (SD) Race Etiology Time post injury, mean months (SD)

Health, Wellness, and Safety Concerns Top 3 Problems (Participants) 1 Pt 1 Memory Pt 2 Stress Management Pt 3 Physical Exercise Pt 4 Memory Pt 5 Getting physical exercise Pt 6 Memory 2 Anxiety Physical Exercise Socialization Planning Falls Planning 3 Socialization Anxiety Sleep Socialization Attention Top 3 Problems (Caregivers) No CG 1 CG 2 CG 3 CG 4 CG 5 CG 6 1 - Stress Memory Socialization Memory 2 - Anxiety Planning Medication Management Attention Planning 3 - Physical Exercise/Attention/Medication Socialization Management Memory Attention

Health, Wellness, and Safety Concerns Reporting of Troublesome Areas Concern (total # concerned) # of Participants (out of 6) # of Caregivers (out of 5) Memory (7) 3 4 Attention (6) 2 4 Socialization (5) 3 2 Physical Exercise (4) 3 1 Planning (4) 2 2 Anxiety (3) 2 1 Medication Management (2) 0 2 Stress (2) 1 1 Sleep (1) 1 0 Falls (1) 1 0

Technology Solutions § Wearable technology (MS Band, Garmin, etc. ) § 5 out 6 had experienced using a wearable device § Liked information provided (steps, calories burned, % goal achieved, GPS/tracking distant and route traveled) § Exercise technology § Mixed feelings about this type of technology § Majority had positive response to the Kinect camera (real time feedback) and being able to have workout routine readily accessible on a wearable devices § Sleep monitoring technology § Majority of participants didn’t report having a need to monitor or improve sleep and seemed apathetic about using this technology § Although, 1 participant positively responded after interviewer explained how it could be used or be helpful 15

Technology Solutions § Cognitive games § There were mixed feelings about playing cognitive games (4 of the participants really enjoyed playing a variety of games and 2 did not like games at all but expressed they did not like games even prior to injury) § Medication management § Interviewer received the most questions about how this technology worked § Received positive feedback from the majority of participants § Health Coaching Platform § Participants preferred the simpler display 16

Home Sensor Technology Privacy Concerns v Participant 1: (F, 25) § Interested/Willing to share info. v Participant 2: (M, 26) § Interested/Willing to share info. v Participant 3: (M, 58) § Interested/Concerned to share info. v Participant 4: (F, 20) § Interested/Willing to share info. v Participant 5: (M, 27) § Interested/Willing to share info. v Participant 6: (M, 24) § Interested/Willing to share info. 17

D 3 – Use of Behavioral Informatics to support safety & independence at home Purpose § Develop & test behavioral informatics applications included in the virtual coaching suite Activities § Adapt computational models of physical and cognitive functions § Test sensors, interfaces, and system alerts in Smart Home Lab § Evaluate full protocol Deliverables § Transfer technology to ilumivu for commercialization 18

Phase 2 Design/Methodology Technological Development § Technological intervention (virtual coaching app) will be developed based on thematic-analysis of the interview/focus group data and validity testing of data output on current market technologies § Intervention platform will be installed in the homes of persons with TBI and their families

Phase 2: Development Area of focus Existing Under development Planned Physical activity Interactive Video Exercise MS Band 2 Misfit Flash Gym. Central Cognition Cognitive Games (being rebuilt) Sleep management Materials for sleep hygiene, anxiety, circadian Emfit integration Stress management Breathing exercises Physiological stress detection Safety at home Safe at Home On-line assessment EMA/EMI 3 rd party applications Physiologically-triggered messaging Safe at Home integration 3 rd party application integrations Voice EMA 20

Goal – In-home Interventions § Incorporate existing protocols for cognitive monitoring, sleep management, interactive video exercise, & socialization into new coaching platform § Incorporate Safe-at-Home into coaching platform § Explore new wearables § Data collection methods § New inference algorithms § New measures of stress (EDA & HRV) 21

Pilot study: Wear. Tech § Device Validity, Usability, Algorithm Development § Ability to measure stress level in real time § Just-in-time intervention (prediction of unsafe event, recommend relaxation, deep breathing, CBT, etc. ) § Heart rate variability and electrodermal activity § MSBand 2 measures pulse rate and skin conductance § HRV Algorithms based on features related to parasympathetic and sympathetic activity 22

Pilot study: Wear. Tech § Design: § Planned subjects: 11 (completed 8) § Three part study: § 2 hours in lab – stress/relaxation protocol § 48 -60 hours at home § 30 minutes in lab plus participant debriefing § Physiological signals collected: skin conductance (EDA), heart rate variability, accelerometry § Devices: § Wearables: Microsoft Band 2, Empatica e 4, Firstbeat Bodyguard 2 § Other: i 330 12 channel finger tip sensor (in lab only), mobile EMA through Movisens 23

Pilot study: Wear. Tech Objective: § Can we assess stress with wearable sensors? § Compare sensors’ accuracy Current in-progress tasks § Visualize all data from all the different sensors and inspect plots to assure data integrity and potential anomalies. Electrodermal activity from MS Band in Microsiemens vs time in HH: MM Comparing RR intervals from MS 24 Band, e 4, and Firstbeat

Pilot study: Wear. Tech Next steps: § Use the visualizations to help develop algorithms to analyze the time series data and automatically report the presence of mental stress. § Criteria for detecting mental stress: periods in the time-series data where there is relatively increased EDA, lowered heart rate variability, and lack of physical movement. § Use information from interviews to guide design of first iteration of TBI coaching interface 25

Sneak peak!

Sneak peak!

D 1 – Tech/Policy Watch & Emerging ICT Issues Purpose § Identify cutting edge developments in ICT technologies § Recognize mainstream scientific and technology developments that can impact technologies, policies, guidelines and standards related to ICT Activities § Monitor industry ICT innovations § Monitor policies, guidelines, standards on ICT access § Monitor emerging issues related to ICT access Deliverables § Post discoveries on www. Live. Well. RERC. org website

http: //www. livewellrerc. org/

D 2 – App/Tech Factory – Meeting User ICT Needs Purpose § Respond rapidly to user-defined problems & opportunities for new assistive and accessible technology solutions § Leverage expertise inside and outside the Live. Well RERC by engaging broad range of subject matter experts Activities § Use available ICT and AT to develop new hardware solutions § Development of mobile apps to promote ICT access and use Deliverables § New software and hardware solutions § Methodology for engaging third party developers (maker community)

Two Complementary Projects D 2–A: Developing Solutions Using Hardware D 2 -B: Developing Assistive and Accessible Apps for Mainstream ICT Devices Projects Completed or Near Completion

Speak Up: An SPL Meter • Description: User friendly SPL Meter • Google Play Store • https: //play. google. com/store/apps/details? id=my. example. ld 170. speakup • Flurry Analytics to track basic app use • Privacy Policy https: //www. youtube. com/channel/UCn. YR V 42 M_R-Lbbc. NNq. Jhbbw

The Gait Box • Description: Device designed to capture a person’s walking speed • Implementation: Duke geriatric clinic • Next Steps: Test usability and reliability with Duke therapy clinicians

Breathe. Well on Wear • Engineered for Android Wear smartwatches • Designed to assist people with mild traumatic brain injury (m. TBI) and post-traumatic stress disorder (PTSD) • Manage stress through diaphragmatic breathing

Co. Training Platform • Training software platform works with existing portable hardware – Thalmic Labs Myo. Band • The beta version supports the use of an off the shelf armband • Records 8 channels of muscle activity • Can be accessed from any PC at home by patients to increase training time and improve quality of training between visits.

SWOPS – Emotional Regulation and Problem Solving • Interactive problem-solving tool for resolving stressful situations in PTSD and mild TBI. • Adapted the Executive Plus/STEP program used with clients in Shepherd Center’s SHARE Military Initiative • Comprehensive rehabilitation program for assessment and treatment for service men and women who have sustained a mild to moderate traumatic brain injury and Post-Traumatic Stress Disorder (PTSD) from combat in post-9/11 conflicts.

In the works

Alexa Stress Assist • Description: Amazon Echo skill to assist military service members with TBI and PTSD in managing the onset of episodes of post-traumatic stress • Road Blocks: Echo constraints, limited cloud computing knowledge, etc. • Next Steps

Pic. Talker App • Description: A talking photo journal for people with complex communication needs • Constraints: Outdated code • Next Steps

Safe@Night • Description: system designed to reduce the incidence of injury or harm to people with developmental disorders at night • Student working on initial prototype • Next Steps

2017 External App Factory Projects 4 new apps selected for development § Blue. Sky Design § University of North Carolina, Chapel Hill § Arts Access International, LLC § IDEAL Group

App Factory § Annual call for proposals to external developers § Will develop a minimum of 20 apps over the 5 -year funding cycle (average 4 per year) § Next call for proposals – April 2017 § Submission deadline – August 15, 2017 § Awards notifications – October 15, 2017 § Platforms of interest – wearables, sensors, remote monitoring, environmental control and automation § More info – http: //www. livewellrerc. org/2017 appfactoryrfp

Website http: //www. livewellrerc. org/

The Rehabilitation Engineering Research Center for Information and Communication Technology Access (Live. Well RERC) is funded by a 5 -year grant from the National Institute on Disability, Independent Living and Rehabilitation Research in the U. S. Department of Health and Human Services (grant number 90 RE 5023). The opinions are those of the Live. Well RERC and do not necessarily reflect those of the U. S. Department of Health and Human Services or NIDILRR.