Muskie School of Public Service Institute for Public

Muskie School of Public Service Institute for Public Sector Innovation Where we are trying to go? • Common outcome measures • Quality data collection for outcomes • A clear understanding of individual program process measurement needs • Program evaluation practiced locally

Muskie School of Public Service Institute for Public Sector Innovation Purpose of Trainings • To build capacity to move from reporting inputs/outputs at program level (the current state) to data-driven performance measurement and management linked to organizational goals and priorities (the future state).

Muskie School of Public Service Institute for Public Sector Innovation Training Overview • Four Logic Model sessions starting in November 2008 and running through February 2009. These sessions will be 3 -4 hours each. • Grantees will complete “Evaluation Planning” Worksheets. • Grantees will complete and submit a data collection methodology. • Grantees will submit data to Muskie by summer 2009.

Muskie School of Public Service Institute for Public Sector Innovation Training Objectives - Overall • By early 2009, all JJAG grantees will have developed a project-specific Logic Model(s) and evaluation planning worksheets • By mid 2009, grantees will have developed/modified existing data collection tools, developed an evaluation methodology, and started to collect data.

Muskie School of Public Service Institute for Public Sector Innovation Training Objectives • Differentiate the 3 levels of evaluation • Understand the Steps in the Evaluation Process • Recognize the advantages of Logic Models • Complete a Basic Logic Model

Muskie School of Public Service Institute for Public Sector Innovation Program Evaluation Can Answer: • What have we done? • How well have we done it? • How much have we done? • How effective has our program been? • What could we do better or differently? • Was it worthwhile? • Was it important?

Muskie School of Public Service Institute for Public Sector Innovation Use Three Levels of Evaluation to help answer the evaluation questions • Process: (Interventions, Activities or Strategies) • Impact: (Initial and Intermediate outcomes) • Outcome: (Long-term outcomes)

Steps in the Evaluation Process START HERE 1. Engage Stakeholders 6. Ensure Use and Share Lessons Learned 2. Describe Program 5. Analyze and Interpret Data 3. Determine Evaluation Design 4. Collect Data

Muskie School of Public Service Institute for Public Sector Innovation Program Overview Logic Model Program Components Initial Outcomes Intermediate Outcomes Long-Term Outcomes Interventions Process Impact Outcome

Muskie School of Public Service Institute for Public Sector Innovation HIV Program Logic Model Template (for providers) Program Components (priority populations) Unsafe Sex, Both Partners Male Intervention Group Level Six session HIV prevention education group in Correctional settings. Education will include condom use negotiation with sex partners. Initial Outcomes (KANS) Persons attending the groups will report an increase in the intent to negotiate condom use with sex partners. *remember YOUTH, RACIAL and ETHNIC MINORITIES and PEOPLE with HIV throughout *KANS=Knowledge, Attitudes, Norms, Skills; **B=Behavior Intermediate Outcomes (B) Persons attending the groups will negotiate condom use with sex partners. Persons attending the group will use condoms with their sex partners. Long-Term Outcome (Health) Decreased HIV transmission Improved well-being of HIV infected people

Muskie School of Public Service Institute for Public Sector Innovation Data Sources Exercise Long-Term Outcome Intermediate Outcome Initial Outcome Interventions Evaluation Questions Performance Indicators Measures Data Sources Timing

Muskie School of Public Service Institute for Public Sector Innovation Expectations: 1. Grantees will be responsible for completing multiple worksheets between sessions. 2. We will monitor your progress and report any delays in submitting assignments. 3. Muskie will provide ample TA. You can call on us if you have any questions.