Performance of Routine Information System Management PRISM Assessment

Performance of Routine Information System Management (PRISM) Assessment Training Session 7: Overview of the PRISM Tools MEASURE Evaluation Date:

Session objectives • Describe how the PRISM Tools are organized • Explain how each PRISM tool relates to the PRISM Conceptual Model • Explain the purpose of each tool

The PRISM Tools Six tools, used for the following purposes: • Establish a baseline for RHIS performance and monitor progress • Identify technical, behavioral, and organizational factors that affect RHIS performance • Aid in designing and prioritizing interventions to improve RHIS performance • Advocate technical and financial support to improve RHIS performance

The PRISM Tools Electronic RHIS Assessment Tool FUNCTIONALITY USABILITY RHIS Performance Diagnostic Tool DATA QUALITY RHIS Overview Tool Facility/ Office Checklist USE OF INFORMATION RHIS Management Assessment Tool (MAT) Organizational & Behavioral Assessment Tool (OBAT)

Linking the PRISM Tools to the PRISM Conceptual Model INPUTS PROCESSES OUTPUTS OUTCOMES IMPACT RHIS determinants Technical factors: • Complexity of reporting forms, procedures • HIS design • Computer software • Information technology complexity Organizational factors: Critical management functions & information needs • Governance • Planning • Training • Supervision • Quality • Finance • Promotion of a culture of information • Availability of resources Performance Diagnostic and Electronic RHIS Assessment Tools Behavioral factors: • Level of knowledge of content of HIS forms • Data quality checking skills • Problem-solving for HIS tasks • Competence in HIS tasks • Confidence levels for HIS tasks • Motivation MAT RHIS processes: • Data collection • Data transmission • Data processing • Data analysis • Data quality check • Feedback RHIS Performance Diagnostic Tool Improved RHIS performance • Data quality Improved health system performance Improve d health status • Information use Overview Tool OBAT Facility/Office Checklist 5

Application of the PRISM Tools Levels of application 1 RHIS Overview Tool Mainly at national level, but can be used at subnational levels 2 RHIS Performance Diagnostic Tool (Data Quality & Use of Information) Health facility and district or higher levels 3 Electronic RHIS Assessment Tool (Functionality and Usability) • • 4 Management Assessment Tool (MAT) District or higher levels 5 Facility/Office Checklist Health facility and district or higher levels 6 Organizational and Behavioral Assessment Tool (OBAT) • • Functionality at national level Usability at each level using the electronic RHIS Part 1 at all levels Part 2 at district and higher levels

1. RHIS Overview Tool Use this tool to understand the structure of: • Existing information systems, type of information collected, and duplication of information • Information flows among different levels of the organization • Interactions among the different HIS The tool consists of: • Data collection and transmission sheet • Information system mapping sheet • Information flow sheet

2. RHIS Performance Diagnostic Too Use this tool to: • Determine the current level of RHIS performance at health facility and district or higher levels • Regularly monitor RHIS performance The tool measures the status of: 1. RHIS performance (data quality and information use) 2. RHIS data management

2. RHIS Performance Diagnostic Tool Measuring RHIS performance Data Quality • Accuracy • Do data reflect what is actually happening at the facility? • Completeness • Are all facilities reporting? • Are all required data elements reported? • Timeliness • Are facilities reporting by or before deadline? Information Use • Analytic report production • Health service performance review and supportive supervision • Revising strategies, developing plans, and setting targets • Mobilizing and shifting resources • Advocacy for more resources • Dissemination of information

2. RHIS Performance Diagnostic Tool Measuring RHIS data management • • • Data collection and transmission o Data collection and reporting guidelines o Designated staff for data compilation and reporting o Clear reporting timeframe Data quality check o Presence of data quality assurance mechanisms Data processing and analysis o Data processing procedures o Comparisons of data Data presentation Feedback mechanism(s)

3. Electronic RHIS Assessment Tool Use this tool to assess how well: • The electronic RHIS is designed with functions that the system is expected to perform at the national level (functionality) • Staff are able to use the system to carry out their functions at any level of the system (usability) The tool measures functions as part of the RHIS process and technical factors, such as: 1. Unique identifiers for the health facility (master facility list) 2. Aggregate report generation 3. Coverage calculation using population estimates 4. Data integration (interoperability) 5. Data disaggregation by sex and age 6. Data analysis and visualization

4. Management Assessment Tool (MAT) Use this tool to measure the effectiveness of key RHIS management functions at the district or higher levels, such as: • RHIS governance • RHIS planning • Training • Supervision • Finance

5. Facility/Office Checklist Use this tool to: • Understand the availability of resources and their condition at health facilities, district offices, and at higher levels • Identify systemic resource issues at any level of the system The checklist measures the availability of: 1. Equipment (computers, printers, backup units, etc. ) 2. Utilities (electricity, backup power, Internet, etc. ) 3. Registers and forms (RHIS supplies) 4. Human resources (adequately trained personnel and the type of RHIS training received)

6. Organizational and Behavioral Assessment Tool (OBAT) Use this self-assessment tool to: • Assess whether organizational mechanisms are in place to produce desired results in RHIS performance • Identify the support of upper management for enhancing the RHIS • Quantify staff performance in the RHIS The tool measures: 1. Promotion of a culture of information 2. Individual behaviors

6. Organizational and Behavioral Assessment Tool (OBAT) OBAT Who should fill out the tool? Who? What? (sections) Part 1 Staff and management at all levels • • Respondent background Promotion of a culture of information RHIS knowledge Perception of competence to perform RHIS tasks Part 2 Staff and management at district and higher levels • Competence to perform RHIS tasks Part 3 Health facility in-charge • Competence to perform RHIS tasks Part 4 Data management staff in • the health facility Competence to perform RHIS tasks

6. Organizational and Behavioral Assessment Tool (OBAT) Promotion of a culture of information Does the organization promote a culture of information by: • Emphasizing data quality • Valuing the use of information • Promoting evidence-informed decision making • Encouraging problem-solving • Promoting feedback • Encouraging a sense of responsibility • Promoting empowerment and accountability

6. Organizational and Behavioral Assessment Tool (OBAT) Individual behavior Behavioral assessment: • Confidence level for carrying out RHIS tasks • Knowledge of the rationale for the RHIS and methods for data quality checks • Skills in data quality assurance, data analysis, interpretation, problem-solving, and use • Motivation to perform RHIS tasks

How to access the PRISM Series This slide deck is one of nine in the PRISM Series Training Kit, which also includes a Participant’s Manual and a Facilitator’s Manual. Also in the PRISM Series is a Toolkit (the centerpiece of the series) and a User’s Kit. The PRISM Series is available in its entirety on MEASURE Evaluation’s website, here: https: //www. measureevaluation. org/prism

MEASURE Evaluation is funded by the United States Agency for International Development (USAID) under the terms of Cooperative Agreement AID-OAA-L-14 -00004. It is implemented by the Carolina Population Center, University of North Carolina at Chapel Hill, in partnership with ICF International; John Snow, Inc. ; Management Sciences for Health; Palladium; and Tulane University. The views expressed in this presentation do not necessarily reflect the views of USAID or the United States government. www. measureevaluation. org