Demonstrating the Quality and Impact of Supportive Housing

Demonstrating the Quality and Impact of Supportive Housing: Key Project-Level Outcome Measures MHSA Promising Practices Call 4/13/11

What is Outcome Measurement? n n n 2 Answers the question: are we doing the right program activities to bring about the outcomes needed by clients Outcomes are benefits to clients from participation in the program. Outcomes are usually measured in terms of enhanced learning (knowledge, perceptions/attitudes or skills) or conditions, e. g. , increased literacy, selfreliance, etc.

Why Outcome Measurement? n n 3 Focuses on short-term, intermediate, or long-term outcomes Determine whether or not program goals/outcome objectives were met To justify the need for further funding To ensure that only effective programs are continued

Logic Models n n n 4 Provide a visual depiction of how a program is supposed to work Describe the inputs, activities and outcomes of a program Visually connect program inputs with short-term and long-term outcomes Specify how the program activities relate to the ultimate outcomes of the program Provide causal links between the operations of the program to short-term and long-term outcomes Clarify the relationship between the program and the problem (and its determinants)

Generic Program Logic Model Inputs People Money Clients Case-load Processes Activities Initiatives Procedures Outputs Services Products Intervening (Contextual) Variables 5 Outcomes Impacts Effects Results

Where to start: Answer Three Key Questions 6 n What did you do? n How well did you do it? n What difference did it make? 6

Evaluating Outcomes: Common Types of Change – New knowledge – Increased skills – Changed attitudes, opinions, or values – Changed motivation or aspirations – Modified behavior – Changed decisions – Changed policies – Changed conditions 7 7

Outcome Measurement: Some Basic Definitions n n 8 Inputs Indicate the amount of resources applied; for example, the amount of funds or number of employees. When related to output or outcome information, the combined information will provide indicators of efficiency/productivity.

Outcome Measurement: Some Basic Definitions n n 9 Outputs Show the quantity of work activity completed. Outputs are expected to lead to desired outcomes, but by themselves do not tell anything about the outcomes.

Outcome Measurement: Some Basic Definitions n n 10 Intermediate Outcomes Events or results that are expected to lead to the end outcomes, but are not themselves “ends. ” Also include characteristics relating to the quality of the service provided to clients, such as accessibility, response time, and overall satisfaction.

Outcome Measurement: Some Basic Definitions n n 11 End Outcomes The consequences/results of what the program did, not what the program itself did. These are likely to be aspects of the client’s condition or behavior that the program seeks to affect.

Outcome Measurement: Some Basic Definitions n n 12 Benchmarks Data that can be used as targets for outcomes or as a comparison with observed outcomes

Essential Outcome Measures for Supportive Housing Programs n 13 Clearly-defined, measurable project-level outcomes are an important tool for evaluating the quality and impact of supportive housing projects

Essential Outcome Measures for Supportive Housing Programs n n 14 Number and Characteristics of Tenants Served Providers can track their progress along the following indicators: Eligible tenants are quickly identified and enrolled into supportive housing (e. g. vacant units are quickly filled); and Tenant characteristics match the eligibility criteria of the supportive housing project or initiative (e. g. length of homelessness, sleeping on the streets, severe mental illness, and/or any other key eligibility criteria. )

Essential Outcome Measures for Supportive Housing Programs n n 15 Housing Outcomes All providers of supportive housing should track: The number/percentage of tenants who remain in supportive housing after one year; and Of those who exit supportive housing, how many or what proportion leave for another stable housing situation

Essential Outcome Measures for Supportive Housing Programs n n 16 Service Utilization All providers of supportive housing should track: The number/percentage of tenants receiving services in supportive housing; and The type of services they receive

Essential Outcome Measures for Supportive Housing Programs n n n 17 Tenant Satisfaction Providers can administer focus groups or a short survey in order to measure tenants’ satisfaction with the following: Safety and cleanliness of housing; Policies, rules, and procedures within the housing Quality and appropriateness of services; and Staffing patterns, staff communication skills, and responsiveness to tenant needs.

Essential Outcome Measures for Supportive Housing Programs n n 18 Additional Project-Level Outcomes: There are many other important outcomes that providers may want to track depending on the goals of the supportive housing project, the target populations, the services being provided, and the capacity to collect data.

Essential Outcome Measures for Supportive Housing Programs n n n 19 Income and Employment Providers may want to track changes in the following outcomes after tenants are housed: Number/percentage of tenants who are employed; Amount of earned income; Number/percentage of tenants with SSI and/or other income supports; and Amount of total income

Essential Outcome Measures for Supportive Housing Programs n n 20 Service Utilization and Cost Effectiveness Research shows that supportive housing reduces tenant incarceration and use of emergency health care, mental health services, and shelter. Tracking these patterns both before and after entry into supportive housing – along with the associated cost savings – can be a powerful tool to increase investment in supportive housing.

Essential Outcome Measures for Supportive Housing Programs n n n n 21 Service Utilization and Cost Effectiveness Agencies/providers should track utilization of other health services and institutions before and after entry into supportive housing, including: Shelters Emergency rooms Hospitals Outpatient health and mental health services Jails and prisons Per diem costs for each of these systems

Essential Outcome Measures for Supportive Housing Programs n n n 22 How will you use the data you collect? Identify where improvements are. This is likely to be the single most important use. Determine how effectively program modifications improved services. This will help identify whether further changes are needed. Motivate staff to continually strive for service improvements. Encourage innovation. Improve fundraising and community relations by including outcome information in communications.

Reported Benefits of Outcome Measurement n n 23 A clear definition of the program's intended outcomes, in itself, provides focus for the program's work. Understanding the current level of outcome achievement provides a barometer to assess progress and direct future activities. Outcome measurement provides invaluable information to improve programs and then see if the improvements make the intended difference. Outcome information is a powerful motivator of staff, who now can observe the progress they are making with participants in a consistent, tangible manner.

Resources and Tools: Point K n n 24 Organizational Assessment Tool: A self-assessment survey and reporting tool that provides a snapshot of organizational strengths and areas for improvement. Logic Model Builder: A step-by-step guide for articulating and connecting organizational or program goals, resources, activities, outputs, and outcomes. Evaluation Plan Builder: Transfers key data from the Logic Model Builder and moves from goal-setting to identification of evaluation questions, indicators, and data collection strategies for evaluating program outcomes and implementation. www. innonet. org

Resources and Tools: Free Management Library n n 25 Basic Guide to Program Evaluation (Including Outcomes Evaluation) in Free Management Library http: //www. managementhelp. org/evaluatn/fnl_eva l. htm

Resources and Tools: Urban Institute n n n n 26 The Urban Institute Series on Outcome Management for Nonprofit Organizations Key Steps in Outcome Management Finding Out What Happened to Former Clients Surveying Clients about Outcomes Analyzing Outcome Information Developing Community-wide Outcome Indicators for Specific Services Using Outcome Information

Resources and Tools: American Evaluation Association n n 27 Website includes materials, links to blogs, extensive information http: //comm. eval. org/EVAL/Home/Default. as px

For more information contact CSH