Clinical Audit Saudi Diploma in Family Medicine Center

Clinical Audit Saudi Diploma in Family Medicine Center of Post Graduate Studies in Family Medicine Presented by: Dr. Zekeriya Aktürk zekeriya. akturk@gmail. com www. aile. net 1 / 29

Clinical Audit and the Audit Cycle 2 / 29

What we are going to cover today • The audit cycle 3 / 29

What we are going to cover today • The audit cycle • Criteria and standards 4 / 29

What we are going to cover today • The audit cycle • Criteria and standards • Structure, process and outcome 5 / 29

What we are going to cover today • • The audit cycle Criteria and standards Structure, process and outcome An audit example 6 / 29

What we are going to cover today • • • The audit cycle Criteria and standards Structure, process and outcome An audit example Problems with audit 7 / 29

What we are going to cover today • • • The audit cycle Criteria and standards Structure, process and outcome An audit example Problems with audit How audit fits into ‘Clinical Governance’ 8 / 29

A definition of audit “The systematic critical analysis of the quality of medical care, including the procedures used for diagnosis and treatment, the use of resources and the resulting outcome and quality of life for the patient” Working for Patients 1989 9 / 29

Reactions to audit • • • Enthusiasm Cynicism Obligation Doubt Exhaustion 10 / 29

Types of audit • Managerial 11 / 29

Types of audit • Managerial • Financial 12 / 29

Types of audit • Managerial • Financial • Clinical 13 / 29

Types of audit • Managerial • Financial • Clinical Multi-disciplinary 14 / 29

Types of audit • Managerial • Financial • Clinical Multi-disciplinary Cross boundary 15 / 29

Types of audit • Managerial • Financial • Clinical Multi-disciplinary Cross boundary Cohort based 16 / 29

Types of audit • Managerial • Financial • Clinical Multi-disciplinary Cross boundary Cohort based Comparative 17 / 29

Types of audit • Managerial • Financial • Clinical Multi-disciplinary Cross boundary Cohort based Comparative Significant event based 18 / 29

Effective audit • An educational activity 19 / 29

Effective audit • An educational activity • Promotes understanding 20 / 29

Effective audit • An educational activity • Promotes understanding • Resource effective 21 / 29

Effective audit • • An educational activity Promotes understanding Resource effective Raises standards 22 / 29

Effective audit • • • An educational activity Promotes understanding Resource effective Raises standards Promotes change 23 / 29

Effective audit • • • An educational activity Promotes understanding Resource effective Raises standards Promotes change Source of information 24 / 29

Effective audit • • An educational activity Promotes understanding Resource effective Raises standards Promotes change Source of information Peer led 25 / 29

Effective audit • • An educational activity Promotes understanding Resource effective Raises standards Promotes change Source of information Peer led Involves patients 26 / 29

The audit cycle Problem or objective identified 27 / 29

The audit cycle Problem or objective identified Criteria agreed and standards set 28 / 29

The audit cycle Problem or objective identified Criteria agreed and standards set Audit (Data collected) 29 / 29

The audit cycle Problem or objective identified Criteria agreed and standards set Audit (Data collected) Identify areas for improvement 30 / 29

The audit cycle Problem or objective identified Criteria agreed and standards set Make necessary changes Audit (Data collected) Identify areas for improvement 31 / 29

The audit cycle Problem or objective identified Re-audit Make necessary changes Criteria agreed and standards set Audit (Data collected) Identify areas for improvement 32 / 29

The audit cycle Problem or objective identified Re-audit Make necessary changes Criteria agreed and standards set Audit (Data collected) Identify areas for improvement 33 / 29

Another way of expressing the audit cycle 34 / 29

Another way of expressing the audit cycle • Determine which aspects of current work are to be considered 35 / 29

Another way of expressing the audit cycle • Determine which aspects of current work are to be considered • Describe and measure present performance 36 / 29

Another way of expressing the audit cycle • Determine which aspects of current work are to be considered • Describe and measure present performance • Develop explicit standards 37 / 29

Another way of expressing the audit cycle • Determine which aspects of current work are to be considered • Describe and measure present performance • Develop explicit standards • Decide what needs to be changed 38 / 29

Another way of expressing the audit cycle • Determine which aspects of current work are to be considered • Describe and measure present performance • Develop explicit standards • Decide what needs to be changed • Negotiate change 39 / 29

Another way of expressing the audit cycle • Determine which aspects of current work are to be considered • Describe and measure present performance • Develop explicit standards • Decide what needs to be changed • Negotiate change • Mobilise resources for change 40 / 29

Another way of expressing the audit cycle • Determine which aspects of current work are to be considered • Describe and measure present performance • Develop explicit standards • Decide what needs to be changed • Negotiate change • Mobilise resources for change • Review and renew the process 41 / 29

Criteria and standards 42 / 29

Criteria and standards • Criteria are those aspects of care that you wish to examine 43 / 29

Criteria and standards • Criteria are those aspects of care that you wish to examine • Standards are the pre-stated or implicit levels of success that you wish to achieve 44 / 29

Structure, process and outcome 45 / 29

Structure, process and outcome • Structure refers to resources you have available (including current knowledge, skills and attitudes) 46 / 29

Structure, process and outcome • Structure refers to resources you have available (including current knowledge, skills and attitudes) • Process refers to what you actually do, e. g. a protocol 47 / 29

Structure, process and outcome • Structure refers to resources you have available (including current knowledge, skills and attitudes) • Process refers to what you actually do, e. g. a protocol • Outcome refers to the health benefits, cost effectiveness or patient satisfaction 48 / 29

The great coffee audit 49 / 29

The great coffee audit Problem The doctors feel that their coffee isn’t hot enough after slogging through morning surgery 50 / 29

The great coffee audit Problem Criteria The doctors feel that their coffee isn’t hot enough after slogging through morning surgery The coffee shall be hot and satisfying to the hard pressed docs 51 / 29

The great coffee audit Problem Criteria Standards The doctors feel that their coffee isn’t hot enough after slogging through morning surgery The coffee shall be hot and satisfying to the hard pressed docs The coffee shall be served at a temperature of 85 -90 C on 80% of occasions and there will be 90% satisfaction level expressed by the docs 52 / 29

The great coffee audit Methods The junior receptionist shall check the temperature of the coffee daily for two weeks and circulate a questionnaire to the partners asking them to score a coffee satisfaction level between 1 and 10. The practice manager shall visit Tesco’s and interview the manager about the availability, costs, quality and sell-by dates of the coffee brands available 53 / 29

The great coffee audit Review After a rather tense audit team meeting it was found that the coffee temperature fell below 37 C on at least 33% of occasions and reached the standard on only 10% of occasions. The doctors scored the coffee at an average 3/10 and two expressed it undrinkable. The practice manager reported the results of her Tesco’s visit. 54 / 29

The great coffee audit Change It was agreed to replace the aged coffee maker ( after agreeing suitable redundancy terms for the senior receptionist) with a shiny new machine from Argos. As an additional ‘quality initiative’, cream cakes would be served after surgery. The coffee contract would be switched from the corner shop to Tesco PLC Trust 55 / 29

The great coffee audit Re-audit For a further two weeks it was agreed to measure the coffee temperature and recirculate the questionnaire. It was gratifying to find 100% correlation with agreed standards with the exception of one partner who didn’t like coffee anyway. 56 / 29

The great coffee audit Re-audit Future audit For a further two weeks it was agreed to measure the coffee temperature and recirculate the questionnaire. It was gratifying to find 100% correlation with agreed standards with the exception of one partner who didn’t like coffee anyway. • Cost implications of standard maintenance • Cholesterol assays for partners 57 / 29

Contents of an audit 58 / 29

Contents of an audit • Background 59 / 29

Contents of an audit • Background • Literature review 60 / 29

Contents of an audit • Background • Literature review • Criteria and standards 61 / 29

Contents of an audit • • Background Literature review Criteria and standards Methods or protocol 62 / 29

Contents of an audit • • • Background Literature review Criteria and standards Methods or protocol Results 63 / 29

Contents of an audit • • • Background Literature review Criteria and standards Methods or protocol Results Recommendations for change 64 / 29

Contents of an audit • • Background Literature review Criteria and standards Methods or protocol Results Recommendations for change Recommendations for further audit 65 / 29

Why audit 66 / 29

Why audit • Useful clinically 67 / 29

Why audit • Useful clinically • Encourages teamwork 68 / 29

Why audit • Useful clinically • Encourages teamwork • Improves patient care 69 / 29

Why audit • • Useful clinically Encourages teamwork Improves patient care Financial benefits (sometimes!) 70 / 29

Why audit • • • Useful clinically Encourages teamwork Improves patient care Financial benefits (sometimes!) Becoming contracturally an obligation with the arrival of PCG’s 71 / 29

Problems with audit 72 / 29

Problems with audit • Audit and research 73 / 29

Problems with audit • Audit and research • Statistical verification 74 / 29

Problems with audit • Audit and research • Statistical verification • Outcome measures and proxies 75 / 29

Problems with audit • • Audit and research Statistical verification Outcome measures and proxies Protocols 76 / 29

Problems with audit • • • Audit and research Statistical verification Outcome measures and proxies Protocols Closing the loop - introducing change and re -audit 77 / 29

A word about clinical governance 78 / 29

A word about clinical governance “A framework through which NHS organisations are accountable for continuously improving the quality of their services and safeguarding high standards by creating an environment in which excellence in clinical care will flourish” A First Class Service 1998 79 / 29

‘Clinical Governance’ 80 / 29

‘Clinical Governance’ 81 / 29

‘Clinical Governance’ Research 82 / 29

‘Clinical Governance’ Research Audit 83 / 29

‘Clinical Governance’ Research Audit Evidence based medicine 84 / 29

‘Clinical Governance’ Research Audit Evidence based medicine Dissemination of guidelines 85 / 29

‘Clinical Governance’ Research Practice development plans Audit Evidence based medicine Dissemination of guidelines 86 / 29

‘Clinical Governance’ Research Practice development plans Audit Dissemination of guidelines Evidence based medicine Personal development plans 87 / 29

‘Clinical Governance’ Research Practice development plans Audit Dissemination of guidelines Postgraduate medical education Evidence based medicine Personal development plans 88 / 29

‘Clinical Governance’ Practice accreditation Research Practice development plans Audit Dissemination of guidelines Postgraduate medical education Evidence based medicine Personal development plans 89 / 29

‘Clinical Governance’ Practice accreditation Research Special interest groups Practice development plans Audit Dissemination of guidelines Postgraduate medical education Evidence based medicine Personal development plans 90 / 29

‘Clinical Governance’ Accountability 91 / 29

Clinical Audit: Tools and Techniques Helen Betts Head of School Chair of CHIRAD 92 / 29

What is Audit? A systematic and critical appraisal of the planning, delivery and evaluation of service/s in terms of efficiency, effectiveness and quality, within given resources. 93 / 29

Research is concerned with discovering the right thing to do; audit with ensuring that it is done right. 94 / 29

Research or Audit into Nutrition? • Determination of the population’s consumption of fatty acids • identification of actions to reduce fatty acid levels in local population • investigation of the interaction between the effects of fatty acid and obesity • implementation of actions to reduce coronary heart disease • quantification of the level of fatty acid in prepared foods • communication exercise to inform “at risk” 95 / 29 patients of beneficial lifestyle changes

“Clinical audit involves systematically looking at the procedures used for diagnosis, care and treatment, examining how associated resources are used and investigating the effect care has on the outcomes and quality of life for the patient”. Department of Health Clinical Audit: Meeting and Improving Standards in 96 / 29 Healthcare (1993).

Care is audited against defined standards derived from research findings, professional expertise and information about patient needs and expectations. 97 / 29

In concurrent audit, care is evaluated at the time it is taking place. In retrospective audit, care is evaluated after it has been completed. 98 / 29

Reliability refers to the ability of an instrument to measure the area of interest consistently, in the same way across time and with different assessors. Validity refers to the ability of an instrument to measure what it is intended to measure. 99 / 29

Audits of the quality of care normally undertaken through a process of peer review: the review of a professional’s practice by someone of the same profession, against professionally defined standards. 100 / 29

The main methods used in audit of the quality of care are: • • • Direct observation Checklists Documentation audit Questionnaires Interviews Case review 101 / 29

You are a general practitioner organising an audit of the home care for cardiac rehabilitation patients. List all the professions who contribute to this care, including those from other organisations who input to the holistic programme of home support. How could you receive their observations? 102 / 29

Items that would indicate clinical audit is developing successfully: • It is undertaken by multi-professional healthcare teams • it is focused on the patient • it develops a culture of continuing evaluation and improvement of clinical effectiveness focusing on patient outcomes 103 / 29

Benefits for professionals from a commitment to quality assurance: • • uphold professional/service standards increased job satisfaction opportunity for continual improvement fewer dissatisfied patients recognition/valuing of achievements productive use of time/effort acquisition of new skills/experience 104 / 29

Clinical Audit Julie Bone Clinical Governance & Audit Facilitator Telford & Wrekin Primary Care Trust 105 / 29

Introduction • What is clinical audit? • Audit versus research • The audit cycle • Five stages of clinical audit • Summary 106 / 29

What is clinical audit? • A quality improvement process that seeks to improve patient care and outcomes through systematic review of care against explicit criteria and the implementation of change. 107 / 29

What is clinical audit? • Aspects of the structure, processes, and outcomes of care selected and systematically evaluated against explicit criteria. • Where indicated, changes are implemented at an individual, team, or service level and further monitoring is used to confirm improvement in healthcare delivery. NICE. Principles for Best Practice in Clinical Audit. Oxford, Radcliffe Medical Press, 2002. 108 / 29

What is clinical audit? Clinical audit is at the heart of clinical governance and clinical effectiveness 109 / 29

Audit: - are we doing the best thing in the best way? • Measures current practice against specific standards • Never experimental • Uses data in existence by virtue of practice • May require ethical approval • Aims to improve delivery of patient care 110 / 29

Research: - What is the best thing to do/the best way to do it? • Provides sound basis for clinical audit • Involves experimental trials • Uses detailed and often sophisticated data collection • Needs ethical approval and registration • Aims to add to body of scientific knowledge 111 / 29

The audit cycle 112 / 29

Five stages of clinical audit 113 / 29

Stage 1: Preparing for audit • Involving users • Selecting a topic • Defining the purpose • Planning 114 / 29

Involving users • The focus of any audit project must be those receiving care. Users can be genuine collaborators, rather than merely sources of data (Balogh et al. , 1995). 115 / 29

Selecting a topic • There seems little point in trying to audit a rare condition, with a cheap intervention with a fairly superficial outcome. • Which topic? 116 / 29

Selecting a topic • • • High cost/volume/risk Area of local clinical concern Potential for improvement National priority Organisational priority Evidence based 117 / 29

Defining the purpose • • • The following series of verbs may be useful in defining the aims of an audit (Buttery, 1998): to improve to enhance to increase to change to ensure 118 / 29

Planning • • Involve ALL the right people Time and resources Access the evidence Methodology Pilot Report and Action Re-audit 119 / 29

Stage 2: Selecting criteria • Defining criteria • Sources of evidence • Appraising the evidence 120 / 29

Defining criteria Criteria: • are explicit statements that define what is being measured • represent elements of care that can be measured objectively. 121 / 29

Sources of evidence • • Ne. LH /Cochrane/WISH NICE Official Websites NSFs Local or regional guidelines/policy Royal College or Professional Body Recognised journals 122 / 29

Appraising the evidence • Aim /objectives • Methodology • Results /conclusions • Applicable to your patient group • Bias/ causes for concern 123 / 29

Stage 3: Measuring level of performance • Planning data collection • Methods of data collection • Handling data 124 / 29

Planning data collection Need to establish • the user group to be included, with any exceptions noted • the healthcare professionals involved in the users’ care • the time period over which the criteria apply. 125 / 29

Methods of data collection • • Computer stored data Case notes/Medical Records Surveys Questionnaires Interviews Focus Groups Prospective recording of specific data 126 / 29

Handling data • Health service professionals must be aware of the ethical implications of and their responsibilities under the Data Protection Act (1998) when collecting data and presenting results. 127 / 29

Stage 4: Making improvements • Identifying barriers to change • Implementing change • External relationships 128 / 29

Identifying barriers to change • • Fear of change Lack of understanding Low morale Poor communication Culture Pushing too hard Consensus not gained 129 / 29

Implementing Change • Discuss the results with those likely to be affected • Agree an Action Plan • Clearly define – who is doing what • Check progress • Produce report and disseminate • Share findings and changes in practice 130 / 29

Stage 5: Sustaining improvement • Monitoring and evaluation • Re-audit • Maintaining and reinforcing improvement 131 / 29

Monitoring and evaluation Although improving performance is the primary goal of audit, sustaining that improvement is also essential. Indeed, any systematic approach to changing professional practice should include plans to: • monitor and evaluate the change • maintain and reinforce the change (NHS Centre for Review and Dissemination, 1999). 132 / 29

Re-audit • • Close the loop Review evidence Measure effectiveness Decide how often to re-audit 133 / 29

Maintaining and reinforcing improvement Common factors: • reinforcing or motivating factors built in by the management to support the continual cycle of quality improvement • integration of audit into the organisation’s wider quality improvement systems • strong leadership. 134 / 29

Summary • Defined clinical audit • Compared audit and research • The audit cycle. • Five stages of clinical audit 135 / 29

Celebrate • Share learning • Publicise results • Give credit where credit is due! 136 / 29