RBA TM ResultsBased Accountability The Fiscal Policy Studies
RBA TM Results-Based Accountability The Fiscal Policy Studies Institute www. raguide. org www. resultsaccountability. com Book - DVD Orders amazon. com resultsleadership. org
SIMPLE COMMON SENSE PLAIN LANGUAGE MINIMUM PAPER USEFUL
Results-Based Accountability is made up of two parts: Population Accountability is about the well-being of WHOLE POPULATIONS for Communities – Counties – States Nations Performance Accountability is about the well-being of CUSTOMER POPULATIONS for Programs – Agencies – Service Systems
Results-Based Accountability COMMON LANGUAGE COMMON SENSE COMMON GROUND
COMMON LANGUAGE
THE LANGUAGE TRAP Too many terms. Too few definitions. Too little discipline Benchmark Outcome Result Modifiers Indicator Measurable Core Urgent Qualitative Priority Programmatic Targeted Performance Incremental Strategic Systemic Measure Goal Objective Target Lewis Carroll Center for Language Disorders Core qualitative strategic Make up your own jargon. Measurable urgent systemic
DEFINITIONS RESULT or OUTCOME 1. A condition of well-being for Population children, adults, families or communities. Children born healthy, Children ready for school Safe communities, City, Prosperous Economy communities A Healthy Green City INDICATOR or BENCHMARK 2. A measure which helps quantify the achievement of a result. Performance Rate of low-birthweight babies, Percent ready at K entry, rate air quality index, index unemployment rate crime rate, PERFORMANCE MEASURE 3. A measure of how well a program, agency or service system is working. 1. How much did we do? Three types: 2. How well did we do it? 3. Is anyone better off? = Customer Results
Population From Ends to Means From Talk to From to Action RESULT ENDS Performance INDICATOR PERFORMANCE MEASURE Customer result = Ends Service delivery = Means MEAN S
IS IT A RESULT, INDICATOR OR PERFORMANCE MEASURE? RESULT 1. INDICATO 2. R PERF. 3. MEASURE RESULT 4. Safe Communities Crime rate Average police response time People live in safe stable affordable housing INDICATO R 5. % RESULT spending >30% of income on housing | INDICATO R 6. People have living wage jobs and PERF. income MEASURE 7. % of people with living wage jobs and
Translation Guide/Rosetta Stone Not the Language Police Ideas 1. A condition of well-being for children, adults, families & communities 2. 3. etc. Group 1 Group 2 Group 3 e RESULT OUTCOME TRANSLATION Back to the Idea tc. GOAL
Results – Indicators – Performance Measures in Amharic, Cambodian, Laotian, Somali, Spanish, Tigrigna, Vietnamese
The Three L’s of Success in any serious change effort 1. Leadership 2. Language Searching for the 3 rd L Leverage Latitude Levity Lithium Luggage
POPULATION ACCOUNTABILITY
Community Outcomes for Christchurch, NZ 1. A Safe City 2. A City of Inclusive and Diverse Communities 3. A City of People who Value and Protect the Natural Environment 4. A Well-Governed City 5. A Prosperous City 6. A Healthy City 7. A City for Recreation, Fun and Creativity 8. City of Lifelong Learning 9. An Attractive and Well-Designed City
CARDIFF, WALES Community Outcomes 1. People in Cardiff are healthy 2. Cardiff has a clean attractive and sustainable environment 3. People in Cardiff are safe and feel safe 4. Cardiff has a thriving and prosperous economy 5. People in Cardiff achieve their full potential. 6. Cardiff is a great place to live, work and play. 7. Cardiff has a fair, just and inclusive society.
Calgary Council Priorities / Results • A prosperous city • A city of inspiring neighborhoods • A city that moves • A healthy and green city
Getting from TALK TO ACTION
Leaking Roof (Results thinking in everyday life) Experience Cm of Water BASELINE Measure Not OK ? Fixed Turning the Curve Story behind the baseline (causes) Partners What Works Action Plan # 2
Results / Outcomes Experience Indicators Baseline & Story
Criteria for Choosing Indicators as Primary vs. Secondary Measures Communication Power Does the indicator communicate to a broad range of audiences? Proxy Power Does the indicator say something of central importance about the result? Does the indicator bring along the data HERD? Data Power Quality data available on a timely basis.
Choosing Indicators Worksheet Safe Outcome or Result____________ Community Candidate Indicators Measure 1 Measure 2 Measure 3 Measure 4 Measure 5 Measure 6 Communicat ion Power H M L H H Proxy Power H M L H H Data Power H M L H L Measure 7 Data Developme Measure 8 nt Agenda
Three Part Indicator List for each Result Part 1: Primary Indicators ● 3 to 5 “Headline” Indicators ● What this result “means” to the community ● Meets the Public Square Test Part 2: Secondary Indicators ● Everything else that’s any good (Nothing is wasted. ) ● Used later in the Story behind the Curve Part 3: Data Development ● New data Agenda ● Data in need of repair (quality, timeliness etc. )
The Matter of Baselines H M OK? L Point to Point Histor y Forecas t Turning the Curve Baselines have two parts: history and forecast
75 people per day 45 people per day MADD 28 people per day 2010
Reboun d
Newcastle, UK Nov 08 – Jan 09 8. 5
Christchurch, New Zealand Number of Graffiti Sites FY 2002 to FY 2010
Hull, UK "Woundings"
“If I include you, you will be my partner. If I exclude you, you will be my judge. ” - Rosell
PERFORMANCE ACCOUNTABILITY
“All performance measures that have ever existed for any program in the history of the universe involve answering two sets of interlocking questions. ”
Performance Measures Quantity Quality How Muc h How Well did we do? (#) did we do it? (%)
Performance Measures Effort How hard did we try? Effect Is anyone better off?
Performance Measures Effort How Well Muc h Effect
Output Effect Input Effort Performance Measures Quantity Quality How much service did we deliver? How well did we deliver it? How much change / effect did we produce? What quality of change / effect did we produce?
Effect Effort Performance Measures Quantity Quality How much did we do? How well did we do it? Is anyone better off? # %
Effort Education Quantity Quality How much did we do? How well did we do it? Number of students Studentteacher ratio Effect Is anyone better off? Number of high school graduates Percent of high school graduates
Effort Drug/Alcohol Treatment Program Quantity Quality How much did we do? How well did we do it? Number of persons treated Percent of staff with training/ certification Effect Is anyone better off? Number of clients off of alcohol & drugs - at exit - 12 months after exit Percent of clients off of alcohol & drugs - at exit - 12 months after exit
Effort Waste Management Services Quantity Quality How much did we do? How well did we do it? # tonnes of residential waste Unit cost per tonne collected Effect Is anyone better off? #/amt to land fill % to land fill #/amt diverted from landfill % diverted from landfill
Transportation / Bridge Inspections Quality Quantity Effort How much did we do? Number of inspections How well did we do it? Percent on schedule Effect Is anyone better off? # of bridges rated good (7 -9) % of bridges rated good (7 -9) # of bridge closings for non- scheduled maintenance Rate of bridge closings for non- scheduled maintenance
Effort Fire Department Quantity Quality How much did we do? How well did we do it? Number of responses Response Time Effect Is anyone better off? # of fires % of fires kept to room of origin
Effort General Motors Quantity Quality How much did we do? How well did we do it? # of production hrs Employees per vehicle produced # tons of steel Is anyone better off? Effect # of cars sold $ Amount of Profit $ Car value after 2 years Source: USA Today % Market share Profit per share % Car value after
Effort Not All Performance Measures Are Created Equal Quality Quantity How much did we do? How well did we do it? Least Also Very Important Effect Is anyone better off? Most Important
RBA Categories Account for All Performance Measures (in the history of the universe) Total Quality Mgmt (TQM) Cost (quality service delivery & customer benefit) Effect Proces s Inpu t Product Output Imp act Quality Effort Quantit y Benefit value Efficienc Admin Efficiency, Admin overhead, Unit cost y cost Staffing ratios, Staff turnover Staff morale, Access, Waiting time, Waiting lists, Worker safety Customer Satisfaction Cost / Benefit ratio Return on investment Client results or client outcomes Effectivene ss Value added Productivit
RBA Categories Account for All Performance Measures (in the history of the universe) Total Quality Mgmt (TQM) Cost (quality service delivery & customer benefit) Effect Proces s Inpu t Product Output Imp act Quality Effort Quantit y Benefit value Efficiency, Admin overhead, Unit cost Staffing ratios, Staff turnover Staff morale, Access, Waiting time, Waiting lists, Worker safety Customer Satisfaction Cost / Benefit ratio Return on investment Client results or client outcomes Effectivene ss Value added Productivit
RBA Categories Account for All Performance Measures (in the history of the universe) Total Quality Mgmt (TQM) Cost (quality service delivery & customer benefit) Effect Proces s Inpu t Product Output Imp act Quality Effort Quantit y Benefit value Efficiency, Admin overhead, Unit cost Staffing ratios, Staff turnover Staff morale, Access, Waiting time, Waiting lists, Worker safety 1. Did we Customer treat you Satisfaction * well? Cost / Benefit ratio 2. Did we help you Return on investmentwith your Client results or clientproblems? outcomes Effectivene ss Value added Productivit * World’s simplest complete customer satisfaction survey
OBA Categories Account for All Performance Measures (in the history of the universe) Total Quality Mgmt (TQM) Cost (quality service delivery & customer benefit) Effect Proces s Inpu t Product Output Imp act Quality Effort Quantit y Benefit value Efficiency, Admin overhead, Unit cost Staffing ratios, Staff turnover Staff morale, Access, Waiting time, Waiting lists, Worker safety Customer Satisfaction Cost / Benefit ratio Return on investment Client results or client outcomes Effectivene ss Value added Productivit
Effort The Matter of Control Quantity Quality How much did we do? How well did we do it? Most Control Effect Is anyone better off? Least Control PARTNERSH
The Matter of Use 1. The first purpose of performance measurement is to IMPROVE PERFORMANCE. 2. Avoid the performance measurement equals punishment trap. ● Create a healthy organizational environment. ● Start small.
CHARTS on the WALL
Advanced Baseline Display Create targets only when they are: FAIR & USEFUL Aspiration NOT Punishment Goal (line) Target or Standard Avoid publicly declaring targets by year if possible. ● x Your Baseline Comparison Baseline Instead: Count anything better than baseline as progress.
Performance Accountability Types of Measures found in each Quadrant How much did we do? # Clients/custome rs # Activities (by served type of activity) # # % Common e. g. client staff ratio, workload ratio, staff measures turnover rate, staff morale, % staff fully trained, % clients seen in their own language, worker safety, unit cost % Activity-specific measures e. g. % timely, % clients completing activity, % correct and complete, % meeting standard Is anyone better off? % Skills / # # How well did we do it? Point in Time vs. 2 Point Comparison # Knowledge % Attitude / (e. g. parenting skills) Opinion % Behavior (e. g. toward drugs) % (e. g. school attendance) % Circumstance (e. g. working, in stable housing)
Service: School Hospital Job Training Fire Department _________________ How much did we do? Primary persons patients students # customers trainedactivity Primary hours diagnostic alarms courses of # job instruction to tests responded How well did we do it? Unit cost Workload ratio % of ___x___ that happen on time Is anyone better If your serviceoff? works really well, how are your customer's better off? # students who persons patients keptwho to %fires graduate get jobs fully recover room of origin
Effect Effort Choosing Headline Measures and the Data Development Agenda Quantity Quality How much did we do? How well did we do it? # Measure 1 -------------- % Measure 8 -------------- # Measure 2 -------------- % Measure 9 -------------- # Measure 3 -------------- % Measure 10 ------------- # Measure 4 -------------- % Measure 11 ------------- # Measure 5 -------------- % Measure 12 ------------- Is anyone # Measure 6 -------------# Measure 15 -----------# ------Measure 7 -------------# Measure 16 -------------- #2 Headline better off? % Measure 13 -------------% Measure 15 ----------% -------Measure 14 -------------% Measure 16 -------------- # Measure 17 -------------- % Measure 17 -------------- # Measure 18 -------------- % Measure 18 -------------- # Measure 19 -------------- % Measure 19 -------------- # % Measure 20 ----------- #3 DDA Measure 20 ----------- #2 DDA #3 Headline #1 DDA
Primary v. Secondary Direct v. Indirect Internal v. External LR UR Baseline & Story
Watermain Breaks per year London Ontario
Select 3 to 5 Performance Measures ACROSS THE BOTTOM OF THE ORG CHART 3 -5 3 -5 3 -5 20 – 60 – 20 Rule 3 -5
HOW Population and Performance FIT TOGETHER
THE LINKAGE Between POPULATION and PERFORMANCE POPULATION ACCOUNTABILITY POPULATION RESULTS Healthy Births Rate of low birth-weight babies Children Ready for School Percent fully ready per K-entry assessment Self-sufficient Families Percent of parents earning a living wage Alignment of measures PERFORMANCE ACCOUNTABILITY Job Training Program # persons receiving training Unit cost person trained # who get living wage jobs % who get living wage jobs Contribution relationship Appropriate responsibility CUSTOMER RESULTS
Performance measure Population Indicator
Population Accountability Result: to which you contribute to most directly. Indicators: Every time you present your program, Use a two-part approach. Story: Partners: What would it take? : Role: as part of a larger strategy. Your Role Performance Accountability Program: Performance measures: Story: Partners: Action plan to get better:
Population Accountability Result: to which you contribute to most directly. Indicators: Every time you present your program, Use a two-part approach. Story: Partners: What would it take? : Role: within the larger strategy. Your Role Performance Accountability Program: Performance measures: Story: Partners: Action plan to get better:
Population Accountability Result: to which you contribute to most directly. Indicators: Every time you present your program, Use a two-part approach. Shortcu Story: t Partners: What would it take? : Role: as part of a larger strategy. Your Role Performance Accountability Program: Performance measures: Story: Partners: Action plan to get better:
Different Kinds of Progress 1. Data a. Population indicators Actual turned curves: the better away from the b. movement Program for performance measures: baseline. customer progress and better service: How much did we do? How well did we do it? Is anyone better off? 2. Accomplishments: Positive activities, not included above. 3. Stories behind the statistics that show individuals are better off.
Board of Directors Meeting AGENDA 1. New data 2. New story behind the curves 3. New partners 4. New information on what works. 5. New information on financing 6. Changes to action plan and budget 7. Adjourn
SUMMARY
RBA in a Nutshell 2– 3 -7 2 - kinds of accountability plus language discipline Results & Indicators Population Performance accountability measures Performance kinds of performance measures. accountability How much did we do? 3 - How well did we do it? Is anyone better off? 7 - questions from ends to means in less than Baselines & Turning the Curve an hour.
“If you do what you always did, you will get what you always got. ” Kenneth W. Jenkins President, Yonkers NY NAACP
THANK YOU ! Book - DVD Orders amazon. com resultsleadership. org
“Data is the unblinking eye of reform. ” Barbary Curley, Area Director, Hyde Park Office, Boston
EXERCISES
Creating a Working Baseline from Group Knowledge Indicator or Performance Measure 65% ng Forecasti ng Backcasti Now Not OK?
Turn the Curve Exercise: Program Performance 5 min: Starting Points - timekeeper and reporter - identify a program to work on - two hats (yours plus partner’s) 10 min: Performance measure baseline - choose 1 measure to work on – from the lower right quadrant (to 2017) – OK not OK? 15 min: - forecast Story behind theorbaseline - causes/forces at work - information & research agenda part 1 - causes Two 15 min: pointers What works? (What would it take? ) works 10 min: Report convert notes to one page to action - what could work to do better? - each partners contribution - no-cost / low-cost ideas - information & research agenda part 2 – what
4. ----- Off the Wall ONE PAGE Turn the Curve Report: Performance Program: ________ Performance Measure (Lay definition) Performan ce Measure Baseline Story behind the baseline --------------------------- (List as many as needed) Partners --------------------------- (List as many as needed) Three Best Ideas – What Works 1. -------------2. -------------- Sharp Edges 3. -----No-cost / low-cost 4. ----- Off the Wall
Turn the Curve Exercise – Lessons Talk to Action in an hour 1. How was this different from other processes? What worked and what didn’t work? 2. Why did we ask for: a. Results before indicators? b. Forecast? c. Story? d. No cost / low cost? e. Two hats? f. Crazy idea? g. Only 3 best ideas? 3. Do you think a lay audience could understand the reports? 4. How many think you could lead this exercise with a small group? (2+ curves at the same time)
Next Steps: 1. What’s one thing I can personally do with what I learned today? 2. What’s one thing I would like to ask someone else to do?
Resources www. raguide. org www. resultsaccountability. com RBA Facebook Group Book - DVD Orders amazon. com resultsleadership. org
THANK YOU ! Book - DVD Orders amazon. com resultsleadership. org
Turn the Curve Exercise: Population Well-being 5 min: Starting Points - timekeeper and reporter - geographic area - two hats (yours plus partner’s) 10 min: Baseline - pick a result and a curve to turn - forecast (to 2017) – OK or not OK? 15 min: Story behind the baseline - causes/forces at work - information & research agenda part 1 - causes Two 15 min: pointers What works? (What would it take? ) - what could work to do better? - each partners contribution - no-cost / low-cost ideas - information & research agenda part 2 – what works 10 min: Report convert notes to one page to action
4. ----- Off the Wall ONE PAGE Turn the Curve Report: Population Result: ________ Indicator Baseline Indicator (Lay Definition) Story behind the baseline --------------------------- (List as many as needed) Partners --------------------------- (List as many as needed) Three Best Ideas – What Works 1. -------------2. -------------- Sharp Edges 3. -----No-cost / low-cost 4. ----- Off the Wall
Next Generation Contracting Contract Provisions Provision 1. Specify the 3 to 5 most important performance measures (from the How well did we do it? and Is anyone better off? categories). Provision 2. Specify that the contractor will use a continuous improvement process (the RBA 7 Questions). Provision 3. Specify how the funder and contractor will work in partnership to maximize LR customer results (quarterly meetings using the 7 questions as the agenda). Provision 4. Specify that the funder will work with the funding community to simplify and standardize contracting and performance reporting.
Next Generation Contracting Contract Provisions Provision 5: : Clear articulation of role in population/community well-being using the language of contribution not attribution. Provision 6: 10% for quality management and administration. Provision 7: Multi-year funding using 3 year rolling contracts Provision 8: Use of targets that are fair and useful. Provision 9: Fund flexibility and virtual funding pool: transfer of up to 10% across line items and program lines. Provision 10: Request for Results: Getting past the sometimes negative effects of competitive RFP contracting or tendering.
- Slides: 94