Measuring Client Satisfaction Forsyth County Site NC ADRC

Measuring Client Satisfaction Forsyth County Site NC ADRC Project Mary Anne P. Salmon ADRC Teleconference on Evaluation May 25, 2006 Posted 5/23/06

Context The first 2 “stations” in the model w Established in the community w Showed high levels of satisfaction in their own periodic consumer satisfaction surveys Surveys to demonstrate continued high quality during transition to ADRC model

Survey Purposes Gauge maintenance of overall satisfaction w Would you tell a friend or relative to call [Agency name]? Track quality of service (CQI) w Was the person at Helpline who spoke to you friendly and courteous? w Was the person you talked with knowledgeable? w Was the information you received from Helpline clear and understandable? w Were the steps to obtain those services clearly explained? w What could Helpline do differently to make the services they offer more useful? (open-ended)

Survey Purposes continued Examine effects of streamlining w Were you told to go to, or to call, any other places for a service or for more information? w If you telephoned, were you able to talk to a(n) [agency name] representative on the telephone on your first call, or did you need to leave a message? w If you left a message, when did the person return your call? w If you visited the [agency name] office, how long did you wait to see someone? w If you contacted [agency name] for services, are you receiving the service that you were seeking? Call outcome w Did the information you received from [agency name] help you make a decision or find the service you needed?

Additional Data Collected Referral Source Demographics w Ethnicity w Gender Condition/Category w w w Age 60+ Physical disability Developmental disability/metal retardation Mental disorder Alzheimer’s Disease/dementia Head injury/cognitive damage

Methodology Written survey w Mail-out with return envelope to AAA w Online version available for those who contacted the agency by e-mail (not used) Population: Consumers contacting agencies in these timeframes w Senior Services pre-test: May 16– 30, 2005 w Adaptables pre-test: August 21–September 29, 2005 w First post-test: month of February 2006

Methodology continued Sample w 1 in 6 Senior Services consumers in timeframe • Systematic sample from a random start • Pre-test N = 119 • First post-test N = 69 w All Adaptables consumers in timeframe • Pre-test N = 43 • First post-test N = 75 Response Rates w Senior Services pre-test: 23% w Adaptables pre-test: 30% w First post-test: 23% • Adaptables: 29% • Senior Services: 16%

Limitation of the Study Low response rate + small N = Poor ability to detect significance In Fact Few respondent differences are even suggestive of real differences in the population. Unless otherwise stated, differences between agencies and differences between pre- and post-tests were neither suggestive nor statistically significant.

Findings Overall satisfaction w 91% would recommend the service w 9% “unsure” Quality of service w 100% said representative was friendly and courteous w 94% said representative was knowledgeable w 94% said the information they received was clear & understandable w 100% of those referred elsewhere said steps were clearly explained

Findings continued First Contact Streamlining—phone process w Reached a representative • 68% reached a representative • 32% left message w Call returned • • • None within first hour 25% within a day 38% next day 25% same week 12% did not remember w Not improved from pre-test yet Bottom line: Problem with phone coverage “depth” when representatives are out of office

Findings continued Streamlining—walk in visits w 100% were seen within 5 minutes Streamlining—referred to other agency w 50% were referred w Of those referred and seeking services (N = 10) • 40% receiving them • 60% expecting services soon, on waiting list, or other • Increase from pre-test suggestive but not significant* *At pre-test 35% of Helpline and 0% of Adaptables respondents were receiving services.

Findings continued Outcome w 83% said the information they received helped them find a service or make a decision w Another 10% were calling for future information (not seeking a service or decision) Referral Source w 44% from friends or family members w 25% from other agencies (hope to see this ) w 15% from media (hope to see this )

Findings: Condition of Consumer

Findings continued Diversity of Consumers Responding w Ethnicity • • 72% White 19% African American 6% Asian or Asian American 3% Multi-racial w Gender • 66% women

Findings continued Do Ethnicity or Gender Affect Satisfaction? w Ethnicity—Both were largely very positive • African Americans slightly but significantly less likely to say they would recommend to a friend • African Americans give slightly lower (not significant) ratings to “representative was knowledgeable” and “clear and understandable” • No difference in “friendly and courteous” w Gender • No significant differences • Observed differences show men more positive

Questions? ? Comments?