Performance of Routine Information System Management PRISM Assessment

  • Slides: 27
Download presentation
Performance of Routine Information System Management (PRISM) Assessment Training Session 9: Assessment Analysis MEASURE

Performance of Routine Information System Management (PRISM) Assessment Training Session 9: Assessment Analysis MEASURE Evaluation Date:

Session objectives • Explain how to analyze PRISM assessment data • Become familiar with

Session objectives • Explain how to analyze PRISM assessment data • Become familiar with the types of analyses performed in several countries • Describe examples of strengthening interventions based on PRISM assessment findings

Some examples of analyses Performance Diagnostic Tool MAT OBAT Facility/Office Checklist • Percentage of

Some examples of analyses Performance Diagnostic Tool MAT OBAT Facility/Office Checklist • Percentage of districts, health facilities meeting data accuracy, completeness, and timeliness standards • Average scores for use of information (scale of 0− 100) • Level of RHIS data use; percentage of districts, health facilities having active RHIS data use mechanism(s) • Site- level average scores for critical HIS functions • Knowledge, confidence, and competence levels • Personal staff feelings about motivation and reward systems • Staff views on promoting the culture of information • Proportion of sites having RHIS materials available and/or experiencing stockouts • Mean numbers of specific health staff • Proportion of staff who benefited from RHIS training

Analysis RHIS Performance Diagnostic Tool

Analysis RHIS Performance Diagnostic Tool

RHIS performance summary table Improving Weak Use of information Data quality Weak Improving RHIS

RHIS performance summary table Improving Weak Use of information Data quality Weak Improving RHIS does not produce quality data and managers make little use of the information RHIS produces data of an acceptable quality but managers make little use of the information RHIS does not produce quality data but information is used by managers RHIS produces data of an acceptable quality and information is used by managers

Data quality analysis Average percentage of different dimensions of data quality 90% 80% 70%

Data quality analysis Average percentage of different dimensions of data quality 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% Completeness of data Completeness of Reporting timeliness at elements reporting by facilities district Data accuracy at district

Use of information analysis • Describe information use at the site visited: o Display

Use of information analysis • Describe information use at the site visited: o Display of updated information o Feedback mechanism(s) o Discussion based on RHIS data o What kinds of decisions are made? • Which steps in the process of using information are performing well and which ones have weaknesses? o Compilation of data, display, analysis, interpretation, discussion, decision making

Use of information analysis Percentage distribution of facilities showing use of information 45% 40%

Use of information analysis Percentage distribution of facilities showing use of information 45% 40% 35% 30% 25% 20% 15% 10% 5% 0% Use of information Reports showing for monitoring decisions by type of analysis Meetings with discussion on RHIS data Decisions made Promoting use of based on RHIS information data

Level of use of information at the province level (Mali) Level of information use

Level of use of information at the province level (Mali) Level of information use 0% 50% 100% 150% Overall use of information Decisions referred Decisions taken Discussion at district Availability of official meeting records Available feedback reports RHIS data use in feedback reports Adjusted to global % of respondents

RHIS data displays and update status at the province level (Mali) RHIS data display

RHIS data displays and update status at the province level (Mali) RHIS data display and update status 80% 70% 67% 67% 60% 50% 50% 50% Displays 40% 33% 30% 20% 10% 0% Mother health Child health Facility utlization Disease surveillance Updated displays

Diagnostic tool results (Ivory Coast) RHIS performance Health facilities 2008 2012 Data quality 43%

Diagnostic tool results (Ivory Coast) RHIS performance Health facilities 2008 2012 Data quality 43% Data use 38% Status Districts 2008 2012 60% 40% 81% 38% 44% 70% Status

Analysis Management Assessment Tool (MAT)

Analysis Management Assessment Tool (MAT)

RHIS Management Assessment Identify which elements of RHIS management are performing well and which

RHIS Management Assessment Identify which elements of RHIS management are performing well and which are exhibiting problems 1. Governance 2. RHIS operations 3. Information infrastructure 4. Human resources 5. Financial resources 6. Other resources

Data management processes Describe the strengths and weaknesses of data management processes at the

Data management processes Describe the strengths and weaknesses of data management processes at the site you visited: • Data recording • Data transmission • Data processing and analysis • Data presentation

Mean scores for the RHIS management functions (Mali) RHIS management functions mean score 100

Mean scores for the RHIS management functions (Mali) RHIS management functions mean score 100 Governance 80 46 60 Finance 83 Planning 40 25 20 0 50 56 Supervision 42 Training Quality

Analysis Organizational and Behavioral Assessment Tool (OBAT)

Analysis Organizational and Behavioral Assessment Tool (OBAT)

Perceived confidence levels and observed HIS task competence (Mali) RHIS task self-efficacy and competency

Perceived confidence levels and observed HIS task competence (Mali) RHIS task self-efficacy and competency 0% 20% 40% 60% 80% 100% Use of Information Interpretation Self-efficacy Plotting chart Calculus Data verification Observed competence

Perceived confidence and observed HIS task competence, by level (Ethiopia) RHIS task self-efficacy (confidence)

Perceived confidence and observed HIS task competence, by level (Ethiopia) RHIS task self-efficacy (confidence) and competence levels 80% 76% 73% 70% 66% 60% 50% 47% Self-efficacy 40% Confidence 30% 20% 10% 0% Health post Health center Woreda Health Center

Analysis Facility/Office Checklist

Analysis Facility/Office Checklist

Availability of resources at health facilities and woreda (Ethiopia) Percentage distribution for the cluster

Availability of resources at health facilities and woreda (Ethiopia) Percentage distribution for the cluster Resources Availability Health posts Health centers Hospitals Woreda health offices 0 83 56 0 6 1 0 17 0 28 ≥ 2 0 11 100 66 0 86 58 0 11 1 0 17 0 32 ≥ 2 0 11 67 52 Internet Yes 20 33 2 32 Electricity Yes 3 56 67 86 Computer Printer

Analysis Overview of PRISM results and interventions

Analysis Overview of PRISM results and interventions

Overview of the PRISM results HMIS overview and major findings (N= 6 provinces) INPUTS

Overview of the PRISM results HMIS overview and major findings (N= 6 provinces) INPUTS PROCESSES OUTPUTS Technical Factors Data collection forms, procedures 1 Information system integration 83 Computer software 100 IT complexity 67 Planning Training Supervision Quality assurance Finance Information distribution Culture of information promotion RHIS processes HIS forms content knowledge Data transmission Data quality checking skill HIS problem solving skill Organizational factors Governance Behavioral Factors 50 HIS task competence 42 HIS task confidence 25 58 Motivation 52 Data completeness 84 Data accuracy 27 Data processing 64 Data analysis 81 Data presentation 71 Feedback n/a n/a 83 100 83 RHIS Performance Data quality M 1 M 2 M 3 Complete-ness 83 95 100 Timeliness 17 17 17 Accuracy 63 75 85 Information use 26 100 56 1 83 83 0 -59 60 -89 90 -100

Comparison of results - Example Comparison of results 85% 90% 80% 70% 60% 50%

Comparison of results - Example Comparison of results 85% 90% 80% 70% 60% 50% 40% 30% 20% 11% 0% Demonstrated calculation skills (OBAT) Facility error rate (Diagnostic Tool)

RHIS strengthening interventions (Examples from Ivory Coast) • Integrating HIV/AIDS indicators in the overall

RHIS strengthening interventions (Examples from Ivory Coast) • Integrating HIV/AIDS indicators in the overall RHIS (Technical) • Organizing trainings in the use of information, including problem-solving techniques, at district and health facility levels (Behavioral) • Developing feedback bulletins for health offices at all levels (Organizational)

RHIS strengthening interventions (Examples from Liberia) • Building staff capacity in the use of

RHIS strengthening interventions (Examples from Liberia) • Building staff capacity in the use of the DHIS 2 health information software and in data verification, data analysis, and data use (Technical, Behavioral) • On-site mentoring of monitoring and evaluation (M&E) staff (Behavioral) • Building organizational capacity by establishing, organizing, and implementing routine performance review forums (Organizational) • Setting up data quality assurance mechanisms, such as desk review of data and data quality assessments (Technical)

How to access the PRISM Series This slide deck is one of nine in

How to access the PRISM Series This slide deck is one of nine in the PRISM Series Training Kit, which also includes a Participant’s Manual and a Facilitator’s Manual. Also in the PRISM Series is a Toolkit (the centerpiece of the series) and a User’s Kit. The PRISM Series is available in its entirety on MEASURE Evaluation’s website, here: https: //www. measureevaluation. org/prism

MEASURE Evaluation is funded by the United States Agency for International Development (USAID) under

MEASURE Evaluation is funded by the United States Agency for International Development (USAID) under the terms of Cooperative Agreement AID-OAA-L-14 -00004. It is implemented by the Carolina Population Center, University of North Carolina at Chapel Hill, in partnership with ICF International; John Snow, Inc. ; Management Sciences for Health; Palladium; and Tulane University. The views expressed in this presentation do not necessarily reflect the views of USAID or the United States government. www. measureevaluation. org