SILCNET Presents Developing Evaluating an Outcomes Focused SPIL

  • Slides: 27
Download presentation
SILC-NET Presents… Developing & Evaluating an Outcomes. Focused SPIL PART 1: Goals & Objectives

SILC-NET Presents… Developing & Evaluating an Outcomes. Focused SPIL PART 1: Goals & Objectives April 21, 2010; 3: 00 -4: 30 PM EST PART 2: Evaluation April 28, 2010; 3: 00 -4: 30 PM EST Presented by: Mike Hendricks, Ph. D. 1

Our 3 -Step Approach n Step 1: Decide what you want to achieve –

Our 3 -Step Approach n Step 1: Decide what you want to achieve – Mission, goals, objectives, partial logic model n Step 2: Turn generalizations into specifics – Measurable indicators, geographic scope, target performance level in 2013, timetable for interim progress n Step 3: Develop your action strategy to achieve each Objective – Activities, lead organization, key partners, resources 2

Step 1: Decide what you want to achieve A. State your Mission - But

Step 1: Decide what you want to achieve A. State your Mission - But whose mission? Your total IL program’s! - “PWDs in our state live independently and participate fully in their community” - Your Mission will be different, but… - Wording is important: – Start with your target group – Follow with a present-tense verb phrase – End however makes sense - But what about a Vision statement? 3

Step 1, cont’d. B. Develop your Goals - Desired outcomes one level below your

Step 1, cont’d. B. Develop your Goals - Desired outcomes one level below your Mission - Objectives explain “How will we achieve our Mission? ” - What outcomes do you need to achieve first, in order to achieve your Mission? - Outcome “stepping stones” on the way to your Mission - Put each Goal in a box, right below the Mission 4

Partial Logic Model with Goals n IF we achieve these three Goals, THEN we

Partial Logic Model with Goals n IF we achieve these three Goals, THEN we can achieve our Mission n Your Goals will be different n You may not even agree with these Goals n Try this format, using the suggested wording 5

Step 1, cont’d. C. - Define your Objectives Typically can’t achieve Goals in 3

Step 1, cont’d. C. - Define your Objectives Typically can’t achieve Goals in 3 years Make progress, yes; fully achieve, probably no Need another level of outcomes below Goals…. Objectives! - “How will we achieve our Goals? ” - What outcomes do you need to achieve first, in order to achieve your Goals? 6

Partial Logic Model with Objectives n IF we achieve these four Objectives, THEN we

Partial Logic Model with Objectives n IF we achieve these four Objectives, THEN we can achieve this Goal n Your Objectives will be different – try not to have too many (37 in one SPIL) n [Occasionally, may need to include sub-Objectives] 7

Step 2: Turn Generalities into Specifics n Objectives are the core, but they’re too

Step 2: Turn Generalities into Specifics n Objectives are the core, but they’re too general n “IL providers are well-trained” – What does this mean – exactly? n Use columns 1 -5 of our SPIL Planning Table (see next slide) 8

SPIL Planning Table Specific Objective Measurable Indicators Geographic Scope Three Year Performance Targets Timeframes

SPIL Planning Table Specific Objective Measurable Indicators Geographic Scope Three Year Performance Targets Timeframes for Interim Progress Objective 1 Objective 2 Objective 3 Objective 4 Objective … 9

Step 2, cont’d. A. Develop measurable indicators for each objective - What exactly should

Step 2, cont’d. A. Develop measurable indicators for each objective - What exactly should change in 3 years? - How exactly will we know if we’re being successful on this Objective? - How exactly would two parallel universes be different? - “# and % of front-line staff who have an up-todate Prime Provider certificate” 10

Step 2, cont’d. B. Specify the geographic scope for each Objective – Statewide? Only

Step 2, cont’d. B. Specify the geographic scope for each Objective – Statewide? Only certain areas? C. Set performance targets for each Objective – What an indicator should “read” at the end of the 3 -year SPIL period – Art + science: other States, national standards, expert recommendations, extrapolation of your own performance, etc. 11

Step 2, cont’d. D. Set timeframes for interim progress – Please don’t wait 3

Step 2, cont’d. D. Set timeframes for interim progress – Please don’t wait 3 years to see how you’re doing – Measure an indicator as often as makes sense (annual, 6 months, 3 months) – Shoot for a trend that makes sense (steady, gathers steam, levels off) 12

Step 3: Develop your action strategy to achieve each Objective n Now you know

Step 3: Develop your action strategy to achieve each Objective n Now you know exactly what you want to achieve – very specifically n But how are you going to achieve each Objective? n Use columns 6 -10 of our SPIL Planning Table (see next slide) 13

Example Full SPIL Planning Table 14

Example Full SPIL Planning Table 14

Step 3, cont’d. A. Determine the Activities you need to conduct in order to

Step 3, cont’d. A. Determine the Activities you need to conduct in order to achieve each Objective - What you do -- “-ing” words such as training, surveying, disseminating, etc. - One or more activities for each Objective - But this will vary for each Objective - Activities depend almost entirely on the decisions you’ve made in Steps 1 and 2 - Put activities right below Objectives on your logic model (IF-THEN) 15

Partial Logic Model with Activities 16

Partial Logic Model with Activities 16

B. Agree on one lead organization for achieving each Objective n Important for the

B. Agree on one lead organization for achieving each Objective n Important for the buck to stop some ONE place n SILC, DSU(s), or CILs might usually be the lead n But maybe not always? 17

Step 3, cont’d. C. Agree on key partners for each Objective - Other organizations

Step 3, cont’d. C. Agree on key partners for each Objective - Other organizations may need to be involved, too – Important to plan from the beginning D. Document the resources needed to achieve each Objective – Generally in $$ terms – As closely as possible, but may not be perfect 18

Step 3, cont’d. E. Clarify funding sources – Part B? Part C? Chapter 2

Step 3, cont’d. E. Clarify funding sources – Part B? Part C? Chapter 2 – Older Blind? 19

Evaluating Your SPIL n Required by RSA n SPIL evaluation plans not the strongest

Evaluating Your SPIL n Required by RSA n SPIL evaluation plans not the strongest n Guiding principles: – Evaluation is important – Focus on improvement, not reporting – Evaluations should vary among states – Certain aspects are probably essential – Evaluation is a collaborative activity 20

Use Evaluation to Answer Three Key Questions 1. How well are we achieving our

Use Evaluation to Answer Three Key Questions 1. How well are we achieving our Objectives? - You’ve already done lots of this work - Create a SPIL Evaluation Plan – Objectives, measurable indicators, sources, methods, responsible organizations, frequency of gathering outcome information 21

Example SPIL Evaluation Plan SPIL Objectives to be Evaluated Systematically Measurable Indicators for Each

Example SPIL Evaluation Plan SPIL Objectives to be Evaluated Systematically Measurable Indicators for Each Objective to be Evaluated Sources of Information Needed for Each Indicator Methods to Gather Each Item of Information Organizations Responsible for Gathering Each Item of Information How Often Each Item of Information is Needed Objective 1 Objective 2 Objective 3 Objective …. 22

2. How satisfied are our consumers? n RSA specifically requires this question n Think

2. How satisfied are our consumers? n RSA specifically requires this question n Think of this as one of your Objectives (e. g. , “Consumers are satisfied with our services”) n Then add to SPIL Evaluation Plan 23

3. How can we improve our State’s total IL program? n This is a

3. How can we improve our State’s total IL program? n This is a different kind of question n Don’t assume this happens automatically n Use the “Three C’s” n COMPILE your evaluation information – describe what is – compare as appropriate – interpret the meaning – recommend improvements 24

Evaluating Your SPIL, cont’d. n CONVENE key stakeholders – At least quarterly – Include

Evaluating Your SPIL, cont’d. n CONVENE key stakeholders – At least quarterly – Include all key stakeholders n CONSIDER the implications – What findings say is happening? – Do we believe it’s true? – What should we do about it? n Improve things, keep evaluating 25

n Click the link below now to complete an evaluation of today’s program found

n Click the link below now to complete an evaluation of today’s program found https: //vovici. com/wsb. dll/s/12291 g 4408 a 26

SILC-NET n Support for development of this Webinar/teleconference was provided by the U. S.

SILC-NET n Support for development of this Webinar/teleconference was provided by the U. S. Department of Education, Rehabilitation Services Administration under grant number H 132 B 070003 -09. No official endorsement of the Department of Education should be inferred. Permission is granted for duplication of any portion of this Power. Point presentation, providing that the following credit is given to the project: Developed as part of the SILC-NET, a program of the IL NET, an ILRU/NCIL/APRIL National Training and Technical Assistance Project. 27