Performance Management Presentation Provide quality and organizational improvement

Performance Management Presentation Provide quality and organizational improvement services Team Members: Antonio Rodriguez, Amy Culbertson, Adele Egwu, Carmen Kaplan, Gay Presbury, Josh Rose, Joe Wolski ORS National Institutes of Health 15 January, 2004 1

Table of Contents Main Presentation PM Template ………………………………. Customer Perspective………………. Internal Business Process Perspective…………………… Learning and Growth Perspective……………………… Financial Perspective…………………………… Conclusions……………………. . 2

3

Relationship Among Performance Objectives 4

Customer Perspective 5

Customer Perspective (C 1) 6

C 1 a: Percent of Teams with PMPs reviewed by Division Management (as of Quarter 1 FY 04) • ORS • • Total Number of Teams PMP’s Reviewed by Division Management 13 9 69% Overall • • • 33 24 73% ORF • • Total Number of Teams PMP’s Reviewed by Division Management FY 03 Target 46 33 72% 95% Target not met 7

C 1 b: Percent of Program Areas with PMPs • OQM will work with the Program Area leadership to develop Program Area PMPs • • • Program and Employee Services (Shirl Eller) Security Services (Arturo Giron) Scientific Resources (Bob Ostrowski) Management Services (TBD) Real Estate and Facilities (Leonard Taylor) 8

Customer Perspective (C 2) 9

C 2 a: Number of Improvements by Type • • OQM developed systematic method to gather and quantify improvements during FY 03 24 quantifiable improvements achieved to date • • • Provide Print and Digital Media Services reduced their space by several 1, 000 sq ft for estimated cost avoidance of $200 k Provide Library Services has maintained the unit cost of an “information unit” while the number of units rose by 35% Conduct Collaborative Research had a 16% increase in the number of professional presentations delivered Provide Basic Animal Life support had a 113% increase in animal holding capacity through bldg renovations and purchasing new cages Provide Animal Research Services had an 18% increase in ICU utilization through improved record keeping Provide Administrative Services reduced FOIA cycle time by 35 days through enhancing staff competencies 10

C 2 b: ORS/ORF Overhead Rate • Work with OBSF to establish methodology to define overhead in ORS and ORF • Quantify by organizations (Offices) and job series within line operations (e. g. , AOs, Secretaries) • Calculate budgeted cost for overhead by FY 11

C 2 c: Number of Published Service Group Success Stories 12

C 2 c: Number of Published Service Group Success Stories 13

Customer Perspective (C 3) 14

C 3 a: Customer Scorecard Methodology • Administered ORS Scorecard after the PM conference in November 03 • • Used data to tailor methodology to meet customer needs in FY 03 cycle Plan to administer ORS Scorecard after PM conference in January 04 • • Gauge effectiveness of support provided during FY 03 and 1 st Qtr FY 04 Solicit ideas for improvement Use data formulate methodology/support for FY 04 Repeat some questions from prior survey 15

C 3 a: ORS Customer Scorecard Ratings In planning for FY 03 PM, how helpful would each of the following be to the teams? N = 85 Not at all helpful Note: Data gathered in FY 03 (Nov 02) regarding experience with the FY 02 ASAs. Extremely helpful 16

C 3 a: ORS Customer Scorecard Ratings (cont. ) Division/Office Head Involvement with PMPs N = 62 To no Extent To a Great Extent N = 63 Not at all Helpful Note: Data gathered in FY 03 (Nov 02) regarding experience with the FY 02 ASAs. Extremely Helpful 17

C 3 a: ORS Customer Scorecard Ratings (cont. ) Product/Service Satisfaction Ratings Note: The rating scale ranges from 1 - 10 where “ 1” represents Unsatisfactory and “ 10” represents Outstanding. Refer to the Data Analysis and Graphing training for advice on interpreting these results. Note: Data gathered in FY 03 (Nov 02) regarding experience with the FY 02 ASAs. 18

Customer Perspective What does the data tell you? • • While the majority of PM teams had a complete PMP at the close of FY 03, some teams were still working toward this goal There was inconsistent review by Management of teams’ PMPs at the close of FY 03 There were 24 quantifiable improvements during FY 03 Customer scorecard data offered guidance on how best to support teams during FY 03 19

Customer Perspective What actions are planned? • • Work with Associate Directors to develop Program Area PMPs Ensure that Associate Directors have approved their area’s PMPs Administer the ORS Customer Scorecard after the Performance Management conference Work with OBSF to develop methdology to calculate overhead rate in ORS and ORF 20

Internal Business Process Perspective 21

Internal Business Process Perspective Objective IB 1: Tailor the implementation of the performance management process to accommodate needs of ORS/ORF service groups Measure FY 03 Target FY 04 Target IB 1 a: ORS Customer Scorecard ratings NA Baseline FY 05 Target Initiative > FY 04 OQM Web survey Owner Amy 22

I 1 a: ORS Customer Scorecard Ratings • Data will be obtained with FY 04 survey after conference 23

Internal Business Process Perspective Objective Measure FY 03 Target FY 04 Target FY 05 Target >FY 03 PMP >FY 04 Implementation Report Baseline >FY 03 >FY 04 Baseline >FY 03 Organizational >FY 04 Improvements Form IB 2 a: Percent of Service Groups Baseline with ongoing data collection IB 2: Assist service groups with IB 2 b: Number of OQM data gathering and analyzing data to analyses provided improve their performance IB 2 c: Number of improvements by type Initiative OQM Data Analysis Report Owner Joe Amy 24

I 2 a: Percent of Service Groups with ongoing data collection (as of Quarter 1 FY 04) • ORS • • Total Number of Teams with ongoing data collection 13 6 46% Overall • • • 33 26 79% ORF • • Total Number of Teams with ongoing data collection FY 03 Target 46 32 70% Baseline 25

IB 2 b: Number of OQM data analyses provided (as of Quarter 1 FY 04) • • 33 Total Types of Data Analyses Provided: • • • With Division of Facilities Planning (DFP), developed large scale project scheduling system and project data tracking software Assisted DFP in developing benchmark study Analyzed cycle time data for Department of Veterinary Resources (DVR) nutrition program to determine past process capability and predictability Analyzed cycle time data for DVR pharmacy program to determine past process capability and predictability (FY 02) Analyzed Process Data for DVR histopathologies Analyzed process data for DVR Comprehensive full panels Analyzed process data for DVR routine and rush PCR’s Analyzed data on design & submittal reviews for Department of the Fire Marshall (DFM) to determine past process capability & predictability for new permit process Analyzed five years of emergency response data for Fire Department to determine past predictability of calls and response time. Documented existing process to maintain JCAHO accreditations at the Clinical Center and maintain AALAC accreditations in NIH animal facilities. Developed and administered customer Surveys for 22 ORS/ORF Service Providers 26

IB 2 b: Number of OQM data analyses provided (as of Quarter 1 FY 04) con’t. • Customer Surveys Developed and Administered in FY 03: • Perform master and facilities planning • Maintain roads, parking areas, and landscaping • Manage and administer worksite enrichment programs – vendor survey • Manage and administer worksite enrichment programs – climate assessment • Provide security guard services • Manage solid waste streams • Maintain safe working environment – radiation safety • Maintain safe working environment – biological safety cabinets • Maintain safe working environment – occupational medical services • Maintain safe working environment – pest management • Procure and deliver animal product – animal procurement • Procure and deliver animal product – animal transportation • Provide animal research services – phenotyping • Provide library services – translations 27

IB 2 b: Number of OQM data analyses provided (as of Quarter 1 FY 04) con’t. • Provide scientific equipment and instrumentation services • Manage information technology • Manage ORS budget and finance • Provide administrative support • Maintain accreditations - JCAHO • Maintain accreditations – AAALAC • Support foreign staff exchange program – visa document preparation • Support foreign staff exchange program – foreign scientist 1 on 1 orientation 28

IB 2 c: Number of Improvements by Type • • OQM developed systematic method to gather and quantify improvements during FY 03 24 quantifiable improvements achieved to date • • • Provide Print and Digital Media Services reduced their space by several 1, 000 sq ft for estimated cost avoidance of $200 k Provide Library Services has maintained the unit cost of an “information unit” while the number of units rose by 35% Conduct Collaborative Research had a 16% increase in the number of professional presentations delivered Provide Basic Animal Life support had a 113% increase in animal holding capacity through bldg renovations and purchasing new cages Provide Animal Research Services had an 18% increase in ICU utilization through improved record keeping Provide Administrative Services reduced FOIA cycle time by 35 days through enhancing staff competencies 29

Internal Business Process Perspective Objective IB 3: Improve consistency of implementation of the PM with our collaborative methodology Measure FY 03 Target FY 04 Target FY 05 Target IB 3 a: Percent of Service Groups with completed PMPs 75% 80% 85% PMP Implementation Report >FY 02 TBA OQM Assistance Logs 30 hours per team 20 hours per team IB 3 b: Number of OQM hours: By service group By type of assistance By program area BI 3 c: Number of consultant hours by Service Group Initiative 10 hours Consultant Monthly per team Reports Owner Joe All/Josh Adele 30

I 3 a: Percent of Service Groups with completed PMPs (as of Quarter 1 FY 04) 31

I 3 b and I 3 c: Number of External Consultant and OQM Hours by Service Group – All Service Groups FY 03 NOTE: External consultant hours reflect period from Jan. 2003 – Sept. 2003. OQM hours reflect time period from June 2003 – Sept. 2003, when assistance log was implemented. 32

I 3 b and I 3 c: Number of External Consultant and OQM Hours by Service Group – All Service Groups FY 04 - Quarter 1 33

I 3 b and I 3 c: Number of External Consultant and OQM Hours by Program and Employee Services Service Groups FY 03 34

I 3 b and I 3 c: Number of External Consultant and OQM Hours by Program and Employee Services Service Groups FY 04 - Quarter 1 35

I 3 b and I 3 c: Number of External Consultant and OQM Hours by Scientific Resources Service Groups FY 03 36

I 3 b and I 3 c: Number of External Consultant and OQM Hours by Scientific Resources Service Groups FY 04 - Quarter 1 37

I 3 b and I 3 c: Number of External Consultant and OQM Hours by Security Services Service Groups FY 03 38

I 3 b and I 3 c: Number of External Consultant and OQM Hours by Security Services Service Groups FY 04 - Quarter 1 39

I 3 b and I 3 c: Number of External Consultant and OQM Hours by Management Services Service Groups FY 03 40

I 3 b and I 3 c: Number of External Consultant and OQM Hours by Management Services Service Groups FY 04 - Quarter 1 41

I 3 b and I 3 c: Number of External Consultant and OQM Hours by Real Estate & Facilities Service Groups FY 03 42

I 3 b and I 3 c: Number of External Consultant and OQM Hours by Real Estate & Facilities Service Groups FY 04 - Quarter 1 43

I 3 b and I 3 c: Number of External Consultant and OQM Hours by Service Group – All Program Areas NOTE: External consultant hours reflect period from Jan. 2003 – Sept. 2003. OQM hours reflect time period from June 2003 – Sept. 2003, when assistance log was implemented. 44

I 3 b and I 3 c: Number of External Consultant and OQM Hours by Service Group – All Program Areas FY 04 - Quarter 1 45

I 3 a. Relationship Between Total Hours Spent with Service Groups and PMP Implementation Progress (as of September 30, 2003) Note 1: Total hours include both hours spent by OQM and Consultants for all effort codes. Team progress is defined as percent completion of 14 key elements of the 2003 PMP process Note 2: Total Hours do not include hours spent on OQM’s own PMP (Provide quality, performance, and organizational improvement services 46

Internal Business Process Perspective What does the data tell you? • • There is a correlation between hours of consultant assistance (both OQM and external consultants) and progress in implementing performance management Measures of time spent correlate the results of progress on PM implementation 47

Internal Business Process Perspective What actions are planned? • • Continue to hold meetings, as appropriate, with teams to maintain forward movement Assist Associate Directors in developing and using Program Area Performance Measurement Plans Encourage utilization of performance management data in managing day-to-day operations Develop new method to measure team progress in FY 04 • • More clearly define how measure activities related to data collection and analysis Include improvements form in progress assessment 48

Learning and Growth Perspective 49

Learning and Growth Perspective Objective LG 1: Invest in OQM's staff professional development Measure FY 03 Target FY 04 Target FY 05 Target LG 1 a: Number of hours of professional development (conferences, training, reading, research) 10% of available hours 15% of available hours 20% of available hours Gay LG 1 b: Percent of OQM budget spent on professional development Baseline >FY 02 >FY 03 Gay Initiative Owner 50

L 1 a: % of available time spent on professional development (conferences, training, reading, research) Based on data collected from June 03 to Sept 03 Slide 42 51

L 1 b: Percent of OQM budget spent on professional development support • • FY 02 Expenditures on Professional Development = $23, 502 (2. 7 %) FY 03 Expenditures on Professional Development = $10, 980 (. 83%) 52

Learning and Growth Perspective Objective Measure LG 2 a: Percent of Service Providers attending training LG 2: Enhance the staff skills of LG 2 b: Training Evaluation Scores our ORS/ORF service groups and leadership LG 2 c: Number of consultant hours by Service Group FY 03 Target FY 04 Target FY 05 Target >FY 02 >FY 03 >FY 04 Training Statistics Joe 30 hours per team 20 hours per team 10 hours Consultant Monthly per team Reports Adele >FY 03 >FY 04 OQM Web Survey Amy LG 2 d: ORS Scorecard rating(s) Baseline Initiative Owner 53

L 2 a: Percent of service providers attending training 54

L 2 b: Training Evaluation Scores (as of 30 Sep 2003) Note: Ratings for each question were on a scale from 1 (low) to 10 (high) 55

L 2 c: Number of external consultant hours per Service Group FY 03 Target = 30 hours Service Group NOTE: 28% of teams met or exceeded the target. The assumption is that groups attending at least 30 hours of consultation will enhance their data collection and analysis skills. 56

L 2 d: ORS Scorecard rating(s) • To be gathered during FY 04 with OQM survey 57

Learning and Growth Perspective What does the data tell you? • Substantially less formal training was both offered and attended by staff during FY 03 compared to FY 02 • OQM spent 8% of its time on professional development activities • Average overall mandatory training evaluation scores decreased slightly from FY 02 to FY 03 58

Learning and Growth Perspective What actions are planned? • Continue to offer training to ORS/ORF staff • Continue to develop OQM staff • Better identify training needs of individual staff 59

Financial Perspective 60

Financial Perspective 61

F 1 -F 3: % of OQM staff time by Discrete Service From hours logged: June 03 to September 03 62

F 1 -F 3: Number of OQM staff consultation hours by Discrete Service, FY 03 Hours are estimated from 4 months of assistance logs: June 03 to September 03 63

Calculations for Unit Cost Internal OQM Units: Consultation Hours by OQM staff Internal Unit Cost: (Total OQM budget – contract $)/hours of consultation • External Consultation Units: Consultation Hours • Unit Cost: Sum (consultation hours of external consultant x hourly rate)/ total hours of consultation • Overall Unit Cost: Sum (OQM staff consultation hours + external consultant hours) / Total OQM budget 64

Unit Costs DS 1: Internal Unit Cost: $146. 48/consultation hour DS 1: External consultant Unit Cost: $263. 28/consultation hour DS 1: Overall Unit Cost: $168. 48/consultation hour DS 2: $146. 48/consultation hour DS 3: $146. 48/consultation hour 65

Financial Perspective What does the data tell you? • • The Office of Quality Management is well funded Continue to increase the use of internal consultants 66

Financial Perspective What actions are planned? • • Continue to update and use OQM Assistance Logs Continue to increase hours spent by OQM staff on Performance Measurement and Improvement discrete service and decrease reliance on external consultants 67

Conclusions 68

Conclusions from PMP • • • Assess needs of teams using feedback from ORS Customer Scorecard and tailor process accordingly Increase consulting services of OQM staff and decrease reliance on external consultants Work with associate directors to develop Program Area PMPs 69

Appendix • List what you have included in your Appendix: Page 2 of template Customer Perspective C 2 a: List Name of Measure……………………… C 2 b: List Name of Measure………………………. C 2 c: List Name of Measure………………………. C 4 a: List Name of Measure………………………. Internal Business Process Perspective IB 1 a: List Name of Measure………………………. IB 1 b: List Name of Measure………………………. IB 3 a: List Name of Measure………………………. Learning and Growth Perspective LG 1 b: List Name of Measure………………………. LG 2 a: List Name of Measure………………………. Financial Perspective F 1 a: List Name of Measure………………………. F 2 a: List Name of Measure………………………. F 3 a: List Name of Measure………………………. 70

Page 2 of OQM Template 71

Page 3 of OQM Template 72

Customer Perspective 73

Customer Perspective (C 2 a, b and c) 74

Internal Business Process Perspective 75

Internal Business Perspective (IB 1 a) 76

Learning and Growth Perspective 77

Learning and Growth Perspective (LG 1 a and b) 78

Learning and Growth Perspective (LG 2 a, b and c) 79

80

81

82

83

84