WA Health workplacebased assessment for international medical graduates

WA Health workplace-based assessment for international medical graduates <hospital name> <year>

2

Australian Medical Council § Three pathways to registration: § Competent Authority § Specialist Pathway § Standard Pathway 1. AMC examinations § AMC MCQ CAT and the AMC clinical 2. Workplace-based Assessment (WBA) § AMC MCQ CAT § A program of WBA of clinical skills and knowledge delivered by an AMC accredited authority 3

Australian Medical Council The AMC results panel meeting for candidates commencing in early 2017 is scheduled for Tuesday 1 December 2017 For consideration of your results at this meeting you will be required to have all assessments completed and loaded to the AMC assessment portal by: 30 October 2017 Requests for inclusion in earlier AMC results meetings will not be considered. The length of assessment time has been mandated by the AMC. 4

Workplace based Assessment § Introduced under the 2007 COAG Nationally consistent assessment of IMGs initiative. § WBA has been adopted at 15 hospitals nationwide across 7 health providers. § Duration of the WA program is 12 months. The AHPRA supervised practice requirement is fulfilled during the assessment program to support eligibility for general registration. § WBA programs are accredited in accordance with AMC Accreditation of Workplace Based Assessment Providers: Standards and Procedures 5

What does WBA mean to you? § Program of clinical assessment in everyday practice settings with immediate feedback § Assessment is set at end-of-PGY 1 (intern) level measured across six clinical areas: § § § Child Health Women’s Health Adult Health (Internal Medicine) Adult Health (Surgery) Mental Health Emergency Medicine Doctors from the 2013 Bunbury Hospital WBA cohort 6

What is expected of you? § WBA is based on adult (self-directed) learning principles § It is your responsibility to ensure you meet the program requirements: § Sufficient clinical experience across all areas § A minimum of 10 hours in each discipline before you can be assessed in that area § Completion of all assessments § It is your responsibility for: § Pre-reading and sourcing clinical opportunities § Acknowledgement of own limitations and asking for help when needed 7

What support is available? § <first and last name> – WBA program director <phone number and/or email> § Primary and secondary supervisors § <first and last name> – WBA administration officer <phone number and/or email> § Mentors or local support § Employee Assistance Program § Katrina Lynn – senior project officer (Perth) (08) 9222 4288 8

Role of support people WBA Program Director § Participates in the selection of candidates § Organises assessor training workshops § Orientates candidates to the hospital and the WBA program § Convenes and conduct the calibration workshops § Coordinates program delivery § Undertakes program evaluation at the site and coordinates candidate assessment schedules in collaboration with the senior project officer 9

Role of support people Primary Supervisor § Assigned to a candidate for the full 12 -month program and completes requirements of Medical Board of Australia § Completes Self- Assessment and Personal Learning Plan with candidate § Monitors progress and provides support, advice and assistance where required § Prepares formative and summative reports and provides face to face feedback § Provides review of multi-source feedback results § Ensures all documentation is provided to WBA administration officer for secure storage in files 10

Role of support people Secondary Supervisor § Assigned for duration of a rotation in a department § Monitors progress and provides support, advice and assistance where required § May assist in selection and preparation of cases for assessment § May act as an assessor formative assessments and provide relevant feedback § Ensures all documentation is provided to the WBA administration officer for secure storage in the files 11

Role of support people Assessor § Conducts a direct or indirect assessment in a specific clinical area / skill / dimension § Completes the required assessment documentation and provides immediate feedback § May select case and/or patient to be assessed § Ensures patient consent has been obtained § Ensures clinical areas, skills and dimensions to be assessed are known and sufficient preparation time has been allocated 12

Role of support people Administrative officer § Provides administrative support to the Director of Clinical Training, the program director, the supervisors and candidates § Assists with organising the orientation of candidates to WBA § Arranges and coordinates the booking of rotations and placements for candidates § Assists candidates with booking assessments § Loads assessments to the AMC Portal § Works with the senior project officer to continually improve WBA at site level 13

What is assessed? § The AMC has developed a list of clinical dimensions and clinical skills across which performance should be assessed to ensure safe practice at end of PGY 1 level. § The clinical dimensions identified for assessment are: § Clinical skills § § § § history taking physical examinations investigations and diagnosis Prescribing Management clinical procedures counselling/patient education § Clinical judgement § Communication skills § Ability to work as an effective member of the health care team § Ability to apply aspects of public health relevant to clinical settings § Cultural competence § Professionalism § Attention to patient safety 14

WA Health WBA assessment plan Direct assessment methods § 12 Mini-CEX – 2 in each clinical area (or more as needed) § 6 DOPS – 1 in each clinical area (or more as needed) § 1 external assessors report – direct observation (calibration tool) Indirect assessment methods § § 6 CBD – 1 in each clinical area (or more as needed) 2 Formative supervisor’s reports. Between 18 -20 weeks apart 2 Summative supervisor’s Between 18 -20 weeks apart. MSF – 10 colleague evaluations usually undertaken mid-year. Collated into a report for the primary supervisor to discuss with and provide feedback to the candidate. 15

Direct assessment Mini-CEX The Mini-CEX aims to assess a range of core competencies that are used during day-to-day encounters with patients Competency Medical interviewing and communication skills Physical examination skills Professionalism / humanistic qualities Counselling skills Clinical judgement Organisation/efficiency Overall clinical competence Clinical and patient management skills Facilitates patient’s telling of story; effectively listens and uses questions/directions to obtain accurate, adequate information needed; responds appropriately to affect non-verbal cues. Follows efficient, logical sequence; balances screening/diagnostic steps for problem; informs patient; sensitive to patient’s comfort, modesty. Shows respect, compassion, empathy, establishes trust, attends to patient’s needs of comfort, modesty, confidentiality. Explains rationale for test/treatment, obtains patient’s consent, educates/counsels regarding management. Selectively orders/performs appropriate diagnostic studies, considers risks and benefits, arrives at an accurate diagnosis or differential diagnosis and identifies effective management strategies. Prioritises; is timely and succinct. Demonstrates judgement, synthesis, caring, effectiveness and efficiency. 16

17

Direct assessment DOPS During the DOPS assessment, an assessor will observe you performing a procedure, completes the DOPS assessment form at the end of the procedure and provides you with immediate feedback. Skill Demonstrates understanding Obtains informed consent Demonstrates appropriate preparation pre-procedure Appropriate analgesia or safe sedation Technical ability Aseptic technique Seeks help where appropriate Post procedure management plan Communication skills Professionalism Example at PGY 1 level Identifies clear indication for procedure Prior to procedure explains the procedure in plain language and assesses the patients understanding Familiar and practised with any equipment to be used Briefs assistant/nurse Uses correct analgesia or safe sedation as indicated in correct form and dosage Demonstrates familiarity with equipment and materials Washes hands before and after procedure If unsure of any aspect promptly seeks assistance (i. e. , patient anatomy; equipment; failure to proceed as expected) Explains expected progress and common signs and symptoms to patient Uses clear and unambiguous language and checks patient understanding at regular intervals Demonstrates courtesy and consideration to the patient and any assisting staff 18

19

Direct assessment External assessor report § A measure against which all other assessment reports are calibrated. § Completed in the second half of the year. § Following direct observation of the WBA candidate seeing patients in their workplace the external assessor will provide immediate feedback. Doctors from the 2015 Kalgoorlie Hospital WBA cohort 20

21

22

Indirect assessment Case Based Discussion § Structured, non-judgmental reviews of decision-making and clinical reasoning § WBA candidates select three cases from patients they have managed and prepare neatly typed case notes § The assessor will select one of the three cases for the assessment Doctors from the 2012 Bunbury Hospital WBA cohort 23

24

Indirect assessment Multi Source Feedback § Candidate performance is rated by a number of people, including self-assessment § Forms a collective assessment of how the candidate meets the AMC clinical and personal performance dimensions, including honesty and teamwork § Candidates select at least 10 colleagues with whom they have worked during the WBA program: § supervisors § interns § registrars § nurses § RMOs § allied health staff § consultants § ward and medical administrative staff 25

26

27

Indirect assessment Supervisor assessments – self-assessment § Completed by the candidate prior to the supervisor assessments. § Reviewed by the supervisor at the assessment meeting § The self-assessment form allows supervisors to gauge the candidate’s awareness of their own strengths and limitations § Self-Assessment Review Form 28

Indirect assessment Supervisor assessments - formative § Provides a valuable opportunity for candidates to gain feedback from their supervisor on their progress in the WBA program § Should be seen as a progress report which provides candidates with an opportunity to improve performance in preparation for the summative report § Highlight areas for further development or concerns through use of the Improving Performance Action Plan. § Formative supervisor’s report form 29

Indirect assessment Supervisor assessments - summative § Completed following the supervisor’s observation of the candidate in the workplace, or after the supervisor has discussed the candidate’s performance with other members of the healthcare team § Summative supervisor’s report form Doctors from the 2011 Bunbury Hospital WBA cohort 30

Resources to assist with assessments § Learning resources specific to WBA in WA have been sourced and developed and can be accessed on the WA Health intranet site. § The required textbook for WBA is the AMC Anthology of Medical Conditions § A copy has been purchased for your hospital. You can also purchase your own copy through the AMC § Supervisors and assessors will be able to provide advice on other resources available for specific clinical areas Doctors from the 2011 Bunbury Hospital WBA cohort 31

Passing standards § Candidates must complete all assessments § The passing standards are set by the AMC 32

Passing standards § The passing standard reflects the standard applied to Australian-trained doctors at the end of internship (PGY 1). Assessment Rating scale Application Multisource feedback 5 -point rating scale 0 = unable to assess 1 or 2 = unsatisfactory 3 = marginal 4 or 5 = satisfactory 9 -point rating scale 1, 2 or 3 = unsatisfactory 4, 5 or 6 = satisfactory 7 , 8 or 9 = superior Improving Performance Action Plan Clearly below the level expected Borderline Pass Expected level Clearly above the expected level Must achieve either expected level or clearly above the expected level Mini-CEX, DOPS, CBD Summative supervisor’s report Final supervisor’s report A mean score of 3 or higher is normally required to pass, but any marginal scores will be reviewed in the context of the global scores. Any scores of 1 will lead to a fail. Respondents are requested to provide additional information if they return a score of 1, 2, or 3 for any question, to enable an informed pass/fail decision, and to ensure that constructive feedback can be provided See next page for specific examples All summative supervisors’ reports must achieve a pass score. If one or more supervisor’s reports indicate that the required PGY 1 standard has not been achieved, a review must be held. Prepared by the primary supervisor for the Medical Board of Australia and the AMC. Summarises progress through WBA. 33

Passing standards (Mini-CEX) A rating of: Constitutes a: Mini-CEX Example: 1 for any single clinical skill or dimension Fail (the entire assessment encounter will be counted as a fail, despite all other skills receiving a pass mark) 2 for one clinical skill or dimension Fail (the entire assessment encounter will be counted as a fail, despite all other skills receiving a pass mark) 1. Medical interviewing skills – 5 2. Physical examination skills – 6 3. Professionalism/humanistic skills – 4 4. Counselling skills – 1 5. Clinical judgement – 7 6. Organisation/efficiency – 5 7. Overall clinical competence - 4 1. Medical interviewing skills – 3 2. Physical examination skills – 6 3. Professionalism/humanistic skills – 4 4. Counselling skills – 2 5. Clinical judgement – 7 6. Organisation/efficiency – 5 7. Overall clinical competence - 4 1. Medical interviewing skills – 2 2. Physical examination skills – 6 3. Professionalism/humanistic skills – 4 4. Counselling skills – 2 5. Clinical judgement – 7 6. Organisation/efficiency – 5 7. Overall clinical competence - 4 1. Medical interviewing skills – 5 2. Physical examination skills – 6 3. Professionalism/humanistic skills – 4 4. Counselling skills – 2 5. Clinical judgement – 7 6. Organisation/efficiency – 5 7. Overall clinical competence - 4 1. Medical interviewing skills – 3 2. Physical examination skills – 6 3. Professionalism/humanistic skills – 4 4. Counselling skills – 3 5. Clinical judgement – 7 6. Organisation/efficiency – 5 7. Overall clinical competence - 4 and 3 for another clinical skill or dimension 2 for any 2 clinical skills or dimensions Fail (the entire assessment encounter will be counted as a fail, despite all other skills receiving a pass mark) 2 for any single clinical skill or dimension Pass and 4 or higher for all other clinical skills or dimensions 3 for any 2 clinical skills or dimensions Pass and 4 or higher for all other clinical skills or dimensions 4 or higher for all clinical skills or dimensions Pass 34

Completing assessments on time § The WBA candidate checklist tracks progress through the program § An assessment schedule is developed by the program director and it is recommended that candidates follow this closely to ensure that: § Assessments are completed on time § Eligibility to be reviewed by the AMC in the recommended timeframe. § Please read the Important Information for further details. 35

36

37

38

Process for completed assessments § All forms are to be: § Checked to ensure that all sections are complete including comments § Signed by the supervisor/assessor and the candidate § Forwarded to the WBA program director for review and signature § Forwarded to the WBA administration officer for loading onto the AMC portal § Assessment forms: § Direct assessment § Indirect assessment 39

AMC Feedback § Assessors name, title and role to be included in each assessment form § A variety of assessors to be used – not one assessor to complete all assessments § Different assessments are not to be conducted on one patient § Do not use the same case type or condition for all assessments § Other WBA candidates cannot assess for Multi Source Feedback § When a task or requirement is not observed for any assessment the assessor must provide a reason why. 40

AMC Feedback § Only one case (out of the three selected) is considered in Case Based Discussion § DOPS must be procedural skills and not physical examination skills § DOPS assessors must be registered medical practitioners with four years of experience in the Australian healthcare environment (or equivalent AMC-designated environment) § Registered Nurses may also assess DOPS if they have the appropriate clinical assessment experience – Program Director to advise § DOPS forms must indicate what was being assessed (i. e. , procedure or information), and should include comments from the assessor. 41

General information § The hospital will invoice you directly for WBA fees. § Contact: <first and last name>, Medical Administration § The AMC will invoice you directly for their administration fee through your AMC portfolio § An email/letter will be sent by the AMC § Please update your contact details with the AMC. This is particularly important for ensuring you receive your AMC certificate at the end of the program – instructions for collection are sent via post! 42

Orientation period § If not already done so: § Read the Candidates Guide and familiarise yourself with the requirements of WBA § Complete the modules at http: //wbaonline. amc. org. au/ § Meet with your primary supervisor and complete the Self Assessment and Personal Learning Plan § Once completed, provide to the program director for review and signature § Provide to the WBA administration officer for scanning and filing 43

WBA assessment process § As you rotate through the clinical areas, arrange for someone to assess you for the Mini-CEX, DOPs and CBDs § Forward completed assessments to the program director for review and signature § Arrange for summative and formative reports using your individual timeline as a guide § Talk to the WBA team. They will assist you in arranging any direct or indirect assessments throughout the twelve month period 44

Key messages § Be organised: § Follow your assessment schedule/learning plan. § WBA is self-directed, however if you think you are falling behind talk to the WBA team. They are there to support you! § Take note of the AMC feedback – your assessments will be returned if you do not follow this information! § Make sure the AMC has your correct/new address so you receive your AMC certificate on time! 45

Questions? Doctors from the 2011 Bunbury Hospital WBA cohort 46