Quality Management Office of Developmental Programs ODP QM

Quality Management Office of Developmental Programs ODP QM Certification QM Module 105 Part 2

Learning & Performance Objectives Learning Objectives Performance Objectives • How QM activities support Vision, Mission and Values • Understand structure/system, process and people outcomes • Distinguish QI Teams from other groups • Understand improvement methodology • Understand QM Tools and improvement models such as DDRI and FOCUS-PDCA • Use QM to fulfill the organizational Mission • Identify improvement opportunities and • Establish improvement goals and outcomes • Fulfill a QI Team role • Use a wide range of QM tools and concepts • Apply improvement models • Access and use information resources

QM Module 105 Learning & Performance Objectives (cont’d) Learning Objectives • Understand how data supports QM – Roles/Responsibilities – Data collection design – Data dictionary – Informs decision making Performance Objectives • • • Use graphs and other statistical tools – Organize and analyze data – Select the appropriate chart type to answer specific questions Develop and implement a QM Plan & Action Plan (DDRI and PDCA) – Operationally define measures and key data elements Promote and manage improvement projects Manage continuous and sustainable improvement Write QM reports, monitor progress and make recommendations for improvement priorities.

PDCA OVERVIEW • Applying PDCA • Integrate data into the PDCA model • Questions to ask during PDCA steps • Report Content • Writing the Report

APPLYING PDCA

PDCA OVERVIEW PLAN: – Questions to be answered – Data - what data will be collected, when and how • Operationally defined as needed • Data dictionary as needed – Evidence-based strategies to be implemented or piloted DO: – Implement the evidence-based strategies – Collect and document the data monthly CHECK: – Analyze the data and interpret the findings within the context of the success of the strategies implemented and questions to be answered ACT: – Based on understanding of the data… – Either to sustain the progress or revise the strategies and implement them

PDCA OVERVIEW Continue the PDCA cycle of implementation and ongoing assessment of progress

P (PDCA): P - PLAN • The Action Plan – Based on the QM Plan – Identifies strategies most likely to achieve the target objectives – Data is collected on the measures in the QM Plan – Data is documented ongoing routinely – The Action Plan is a working document that is updated ongoing according to progress/lack of progress

D (PDCA): D - DO • Implement Action Plan • Ensure data collection is occurring • Closely monitor implementation process & data collection with initial weekly reviews • Are there any concerns re implementation? DO NOT A L L OW f l a w ed implem • Resolve early on ent ation to become embedded in the proc ess

C (PDCA): C - CHECK QUESTIONS PRIMARY CONSIDERATION is given to: • Is the QI activity on track to meet the target objective within the timeframe? – Use data to answer this question • If YES – what is working well? – Potential best practices or policy • If NO – determine why – What are the barriers? – What are the obstacles? – What are the challenges? – Are there any confounding factors? • Identify recommendations/potential solutions

C (PDCA): C - CHECK • Ensure implementation problems have been worked out • Monitor data monthly • Compare current performance to projected target objective!! • Use appropriate graphical display: – Adequate # of data points – Appropriate type of graph – Do not rely solely on a table of data

A (PDCA): A - ACT • Are the desired results being achieved? – YES: maintain Action Plan as designed/revised • Continue to monitor for sustainability – NO: based on current analysis, make revisions to plan • Implement plan • Continue to collect and analyze data • Continue to monitor outcomes – Continue cycle • Continue to CHECK

APPLYING THE PDCA MODEL Data Analysis And Keys to Understanding

Applying the PDCA Cycle After the 1 st Q After the 2 nd Q After the 3 rd Q S S E N I L E M I T Apply when development of next year’s plan begins. Apply, minimally, 3 times during an annual cycle Apply at the end of the cycle when beginning a new cycle.

Applying PDCA PLAN IMPLEMENT THE PLAN DO ANALYZE THE DATA CHECK MAKE A DECISION ACT

Applying PDCA PLAN ANALYZE THE DATA DO IMPLEMENT THE PLAN CHECK 1. Retrieve raw data; and aggregate 2. Select appropriate graph type and create • Be sure all graph elements are appropriately and completely labeled • Annotate as needed • Include data for prior timeframe as needed 4. Compare results to target objective or baseline (benchmark)

Analyze the Data: CHECK ANALYZE THE DATA CHECK Do the data indicate preliminary achievement of the target objective? Y T I UN O T R O PP 17

Data Analysis – Keys to Understanding Keys to accuracy in analysis and interpretation include: Operationally defined measures Consistency in data collection Interpretation – specific to dataset findings Appropriate organization and display; i. e. using the right type of graph Supports decision when to ACT Complete a drill down as needed To answer additional questions to clarify findings ANALYZE THE DATA CHECK 18

Data Analysis - Keys to Understanding (cont’d) CRITICAL THINKING • Identify cause (aka root cause) – Most basic and underlying cause – Not always possible to identify; or always evident – Every situation presents learning opportunity • Contributing or Confounding factors – Variables impacting the issue under consideration – May not be direct/linear relationship (i. e. cause) – In some way, influences event/outcome • Both: what are they and what role do they play? – Need to examine the cause and effect relationship • Without this information – limited in ability to remediate/improve/prevent. ANALYZE THE DATA CHECK 19

Data Analysis - Keys to Understanding (cont’d) Basic Critical Thinking Questions*: • • Who What When Where How much Why x 5 ANALYZE THE DATA CHECK 20

Data Analysis - Keys to Understanding (cont’d) PRELIMINARY to analysis: • Are there enough data points to identify – – – • • Trends Shifts Patterns Cycles Decisions to ACT Is data moving in the right direction? Is there a baseline? …an established benchmark? …a specific measurable target objective calculated? ANALYZE THE DATA CHECK 21

Data Analysis - Keys to Understanding (cont’d) When is a drill down necessary? • Analysis indicates there is an outlier or special cause • Both are unknown • Analysis of next lower level continues until source of the outlier or special cause is identified – Examples: • Aggregate analysis indicates a spike/drop in frequency of variable • Shift in direction ANALYZE THE DATA CHECK 22

май-09 мар-09 янв-09 ноя-08 сен-08 июл-08 май-08 мар-08 янв-08 ноя-07 сен-07 июл-07 май-07 мар-07 янв-07 ноя-06 сен-06 июл-06 май-06 мар-06 янв-06 ноя-05 сен-05 июл-05 Data Analysis – Tools ANALYZE THE DATA CHECK 50 40 30 20 10 0

Data Analysis – CHECK ANALYZE THE DATA CHECK • Analyze and interpret within context – Apply the PDCA questions* • Determine: – – Is remediation warranted? Is improvement indicated? Is it too early to tell? Is a drill down necessary? 24

Data Analysis - CHECK WHAT HAVE WE LEARNED? What conclusions can be drawn… • About the desired outcome? • About the corrective actions or improvement activities already implemented? • About the process? • About the system overall? • About the appropriateness of the performance indicators (measures)? • About the accuracy, reliability and validity of the data? – How does this… • …impact the findings? • …the desired outcomes? 25

WRITING THE REPORT

Report Content: KEY Elements Consistent/Standardized (Template) contains: • QM Plan: e. g. , Focus Area • Performance Measures • Findings/Analysis Interpretation: – Barriers/Challenges/Obstacles – Lessons Learned – Performance • Overall • Continuity from prior report related to Actions Taken in response to what was learned • Recommendations • Future Actions/Next Steps (PDCA section) 27

QM Report Template 1

QM Report Template 2

Writing the Report: Report Content Consistent & Standardized • • • Timeframe: FY or CY, Quarter, YTD and Annual Baseline Focus of analysis; e. g. population; system partners Initiative Performance Measures – Exactly as written in QM Plan • Measure being analyzed – Operational Definition as needed • Data source • Reported by 30

Completing the Findings and Analysis Narrative CHECK in PDCA informs the Findings and Analysis section Data is analyzed during CHECK • What do the data indicate about the QM/QI plan and Action plan? – What are the expectations for improvement? – Is the Target Objective on track to be achieved? • Not on track to achieve, then why? If Yes, what works well? – – – Is a baseline being established? Is there a benchmark being aimed for? Is remediation warranted? This usually is process oriented. Is improvement occurring? Generally, outcomes oriented Compliance evident? • After review of the data, what did we conclude? • Can the data be misinterpreted? HOW? ? 31

Writing the Findings and Analysis Data Informs the Findings Begin with stating the Findings = clear and concise • Put findings into context of time – For the current time period under review – Always discuss YTD (year to date) status – Project the target objective results to the end of the timeframe – Iterate the findings exactly as documented – Speak to the graphical display • Always begin with total n • Add details • Drill down when needed Each report stands on its own 32

Writing the Findings and Analysis (cont’d) Data Informs the Analysis Ø Begin to put the findings into context in the Analysis Ø Incorporate other data or sub-measures to support the analysis Interpret the analysis to determine: – Impact of actions already taken – Discuss performance: • What is working well? – Potential for best/promising practices – Policy development • What is not working well? – – Is remediation warranted? Staff training? Policy revision? System or process redesign warranted? 33

Writing the Findings and Analysis (cont’d) Data Informs the Analysis Include these areas of discussion as relevant: • Barriers and challenges • Lessons Learned • Recommendations 34

Writing the Findings and Analysis (cont’d) Barriers/Challenges/Obstacles • State what they are OR • How you will try to identify • Response to BARRIERS is part of PDCA; i. e. , actions that will be taken • Include any contributing factors that impacted: – – Data Collection/Analysis Implementing the Action Plan Desired outcome Discovery, remediation, improvement 35

Writing the Findings and Analysis (cont’d). Barriers/Challenges/Obstacles Examples: • • • Were initially unknown Beyond scope of authority/responsibility or plan Resources allocation-related Timeliness of actions taken Incomplete, inaccurate or untimely data entry Technology breakdown 36

Writing the Findings and Analysis (cont’d) Lessons Learned • Usually something that needs to be integrated into future planning • Can impact timeliness, efficiency, quality of… • Wish it were known in the beginning • Modifies system/process design or remediation or improvement actions Performance • • Critical component of report Determines future actions Based on desired outcome/results Impacts achieving Target Objective within specified timeframe 37

Writing the Findings and Analysis (cont’d) Recommendations • Usually made to leadership with decision-making authority • May be beyond scope of current activity • Usually include: – – – Policy/Procedure development/revision Information management needs (data collection) Staff training Resources needed Discovered as a result of QM activity but not specifically related 38

Writing the Report - PDCA Follow-up Section MAKE A DECISION ACT informs PDCA section Future Actions/Next Steps • Documented in the PDCA section • Based on lessons learned, analysis, performance status Ø Intended to increase likelihood of achieving the desired outcome, target objective, within specified timeframe. • When performance results are: – As intended – Not as intended Continue with Action Plan Steps taken to assure improvement 39

Writing the Report - PDCA Follow-up (cont’d. ) If barriers were identified… Or initial strategies were not effective/successful… Document the PDCA section of the template with: • What you are doing or will do to overcome the barriers. • If you’re not on track to reach your objective, identify what activities will be taken to get on track • For both of the above, also update the Action Plan so it is a reflection of the steps taken to achieve the TO. MAKE A DECISION ACT 40

Writing the Report - PDCA Follow-up (cont’d. ) The ACTION PLAN MUST update the Action Plan when new steps/strategies are being implemented based on the analysis report. • • Development and updating is often overlooked Document the steps to be taken By whom When Consider chronological order Not dependent upon external action/entity More detailed record of what has been implemented MAKE A DECISION ACT 41

RECAP Ø Continue the PDCA cycle to learn what works Ø Use data to: OPP ORT Y UN – Assess progress – Determine what works – Make evidence-based decisions Ø Early on assure implementation is as designed – Make modifications as determined by CHECK Ø Ø Ø Identify impediments to success and address them Pay attention to timeliness Be prepared to ACT Establish the window of opportunity Assure there is continuity from the prior report IT

RECAP (cont’d. ) Ø Document findings and analysis clearly and concisely Ø Document which actions resulted in improvement!! ü CHECK Ø Inform leadership of progress Ø Take steps to develop the next QM/QI plan early enough to implement on time

Expectations for Certified Staff Excerpted from QM Module 102, part 2

Expectations for Certified Staff (cont’d. ) • Stay updated about ODP priorities and requirements for business partners • Create and maintain a culture of continuous quality improvement by being a role model for positive change. • Foster communication, collaboration, and teamwork. • Use data to inform decision making. • Be proactive.

Expectations for Certified Staff (cont’d. ) • Organize data based on QM concepts; use QM tools. • Strategize actions to achieve improvement at the person-centered outcomes, process, and system levels. • Monitor data to evaluate effectiveness of actions. • Recommend convening, facilitate and participate in QI teams to achieve improvements. • Make recommendations to leadership based on findings and analysis.

Expectations for Certified Staff (cont’d. ) Finally, but most importantly… • Review the learning objectives and the performance objectives and strive to acquire the knowledge and skills outlined.

Final Thoughts Quality management staff support business operations staff to fulfill core functions and essential activities in order to achieve the vision and mission of the organization in a manner that is efficient, cost-effective, PERSON-CENTERED and results in quality outcomes and customer satisfaction.

For Further Training or Technical Assistance Dolores Frantz ODP QM Director dofrantz@pa. gov 570 -372 -5800 Ann Ligi ODP QM Lead QM Certification Lead c-aligi@pa. gov 570 -443 -4218 Amy Henasey ODP QM Coordinator ahenasey@pa. gov 570 -443 -4003 Diana Ramirez ODP QM Coordinator dramirez@pa. gov 570 -372 -5993 49