Evaluation n Assists with allocating resources n what

Evaluation n Assists with allocating resources n what is working n how things can work better

Effective Evaluations n ID strategies/initiatives working best n Provide feedback for decision making n Show cost-effectiveness of different strategies n Increase ability to manage projects/programs

Why Evaluate? n To determine effectiveness n understand or verify impact of service, program, product n To assess efficiency n ensuring optimal use of time & resources n To be accountable n account for what has been accomplished through project funding n To identify ways to improve a program n n n Improving usefulness of the program or service Determining what works & doesn’t (strengths & weaknesses) Determining if program is meeting needs of participants

Types of Evaluation n Formative n Information you collect to help you plan or implement a program/service n n Needs assessment Feasibility assessment Pre-testing program elements Audience analysis

Types of Evaluation n Process n n Assesses aspects of program activities On-going process – feedback for decision making along the way n n n Tracking quantity & type of service Tracking people reached (# and type) Participant satisfaction (e. g. , quality of service) n PIE focus n Perfect, Improve, Eliminate

Types of Evaluation n Impact n Whether & to what extent a program has accomplished stated goals & objectives

Fiscal or Efficiency Evaluation n Compares costs -- economic resources consumed by program n WITH n consequences -- to health effects or outcomes n

Fiscal Evaluation Cost-Benefit Analysis n Compares n cost in $$ n economic benefits/consequences in units of currency n For every $1 spent on preschool program, there are $7 in benefits through reduced societal costs

Fiscal Evaluation – Cost-Effectiveness Analysis n Compares: n costs in monetary units -- $$ n health effects in units of mortality, morbidity, health status measure n Costs $x to prevent one infant death from diarrheal disease by supporting breastfeeding

Fiscal Evaluation Cost-Utility Analysis n Compares n cost in $$ n consequences in health terms as utilities or the preference/desire for a specific outcome n Compare costs and outcomes of interventions n Cost $x to avoid one case of diarrheal disease through oral rehydration or clean water supply

Steps in Evaluation n DESCRIBE/CLARIFY YOUR PROGRAM n Engage stakeholders n Assess resources n DESIGN THE EVALUATION n Determine evaluation methods n Collect the data n Process and analyze the data n Interpret and disseminate the results n TAKE ACTION n Who conducts evaluation?

Evaluation Framework n Provides a plan that links the activities (implementation objectives) and outcomes (outcome objectives) with your evaluation questions, indicators, data sources and data collection methods n Evaluation questions directly reflect program’s implementation and/or outcome objectives.

Evaluation Framework n Indicators n n n What are the specific measures indicating you have achieved your objectives? How will you know if you accomplished your objectives? What would be a success? n Sources n n Who can provide information about this? Where can you get the information? n Tools n What is/are the best way(s) to gather information/find out from people?

Evaluation Framework Implementation or outcome objective: Evaluation Questions Indicators Data Sources Data Methods

Measurement Indicators n Outcome Indicators n measures for your outcomes n Short-term n n Improved access Improved collaboration Policy changes Changes in awareness, knowledge or beliefs n Longer n n n term Service utilization Morbidity/mortality Health status

Measurement Indicators n Process Indicators n Measures for how program delivered n Training sessions held Type of staff Staff activities and time Resources distributed Participation Client Satisfaction n n

Program Logic Model n A diagrammatic representation of a program n n Depicts relationships between program activities, outputs & outcomes Describes a program to stakeholders n n clarifies how the program is structured Shows how different facets of a program are linked Integrates program planning & evaluation Facilitates the assessment of the feasibility of program activities achieving program objectives

Program Logic Model – CAT SOLO n Components – Closely related groups of activities in your program n Activities – describe what you are going to do & provide (implementation strategies) n Target reach – describes who you are trying to n Outcomes – describe impacts on the customers, clients, organizations, systems (short and long-term outcomes)

Program Logic Model – CAT SOLO Components Activities Target Short-term Outcomes Long-term Outcomes

Communicating Evaluation Findings n Communicate info to potential users n Ensure report addresses issues users perceive as important n Ensure timely delivery n Ensure format easily understood by users

Communicating Evaluation Findings n Include copies of survey tools n Summarize computer printouts in table format

Potential Users of Evaluation Results n Immediate Supervisor n Senior Management/Boards of Directors n Policymakers n Funding agencies n Peers in similar programs n Client base/Community n Program personnel

Communicating Evaluation Findings: Figure 4 -4, p. 123