Evaluation is for Improvement Not Provement Professional Competence

Evaluation is for Improvement Not Provement

Professional Competence • The habitual and judicious use of communication, knowledge, technical skills, evidence based decision making, emotions, values, and reflection to improve the health of the individual patient and the community. ﻣﻬﺎﺭﺗﻬﺎی ، “ﺑﻪ کﺎﺭگیﺮی ﻋﺎﺩﺕگﻮﻧﻪ ﻭ ﻣﺪﺑﺮﺍﻧﻪ ﺩﺍﻧﺶ ﺍﺭﺯﺷﻬﺎ ﻭ ، ﻋﻮﺍﻃﻒ ، ﺍﺳﺘﺪﻻﻝ ﺑﺎﻟﻴﻨﻲ ، ﻣﻬﺎﺭﺗﻬﺎﻱ ﻋﻤﻠﻲ ، ﺍﺭﺗﺒﺎﻃﺎﺗی ﺩﺭ ﺟﻬﺖ ﺧﺪﻣﺖ ﺑﻪ ﻓﺮﺩ ﻭ ، ﺑﺎﺯﺍﻧﺪﻳﺸﻲ ﺩﺭ ﻓﻌﺎﻟﻴﺖ ﺣﺮﻓﻪﺍﻱ ﺭﻭﺯﻣﺮﻩ " ﺟﺎﻣﻌﻪ Epstein and Hundert, Designing and assessing professional competence. JAMA 287: 226, 2002 4

What is Clinical Competency? The ability to handle a complex professional task by integrating knowledge, skills and attitudes. ﻧگﺮﺵ ﻣﻬﺎﺭﺕ ﻋﻤﻠی ﺷﻨﺎﺧﺖ ( )ﺩﺍﻧﺶ ﻣﻬﺎﺭﺕ ﻋﻤﻠی ﻭ ، ﺗﻠﻔیﻖ ﺩﺍﻧﺶ ﻧگﺮﺵ ﻫﺎی ﺣﺮﻓﻪ ﺍی ﺩﺭ ﺿﻤﻦ ﺍﻧﺠﺎﻡ یک ﻭﻇیﻔﻪ ﺑﺎﻟیﻨی پیچیﺪﻩ Reference Van der Vleuten, C. The assessment of professional competence: developments, research and practical implications Advances in Health Science Education 1996, Vol 1: 41 -67 Rice et al. , Competency-based objectives for clinical training. Can. J. Med. Technol. 57: 136, 1995

What is the best way to evaluate Clinical Competency? Performance assessment: Masked SP Behavior Skills & attitudes Competency assessment: OSCE Clinical based tests: MCQ Knowledge tests: MCQs Miller’s Pyramid of Competenc Miller GE. The assessment of clinical skills, competence, performance Acad Med, 1990; 65(Suppl. ): S 63 -S 67.

Professional authenticity A simple model of competence Does Shows how Knows Miller GE. The assessment of clinical skills/competence/performance. Academic Medicine (Supplement) 1990; 65: S 63 -S 7. Performance or hands on assessment Written, Oral or Computer based assessment

Professional authenticity Model of competence Behaviour~ skills/attitudes Cognition~ knowledge Miller GE. The assessment of clinical skills/competence/performance. Academic Medicine (Supplement) 1990; 65: S 63 -S 67.

climbing the pyramid Does Shows how Knows Performance assessment in vivo: Masked SPs, Video, Portfolio…. . Competence assessment in vitro: OSCE, …. . (Clinical) Context based tests: MCQ, essay type, oral…. . Factual tests: MCQ, essay type, oral…. .

Validity of testing formats Professional practice assessment Performance assessment: OSCEs, long/short cases, etc Problem-solving assessment: PMP�MCQs, . . Knowledge assessment: MCQs. .

Miller’s Pyramid for Assessing Clinical Competence Work Does OSCE Action Performance Shows How PMPs Competence Knows How MCQs Knowledge Knows

. . …Expanding our toolbox Does Shows how Knows Established technology of efficient written or computer-based high fidelity simulations (MCQ, Key Feature, Script Concordance Test, MEQs…. )

. . …Expanding our toolbox Does Shows how Knows Established technology of structured high fidelity in vitro simulations requiring behavioural performance (OSCE, SPbased testing, OSPE…. )

. . …Expanding our toolbox Does Shows how Knows Emerging technology of appraising in vivo performance (Work-based assessment: Clinical worksampling, Mini-CEX, Portfolio, practice visits, case orals…. )

. . …Expanding our toolbox Does Shows how Emerging technology of appraising in vivo performance (self-, peer, co-assessment, portfolio, multisource feedback, learning process evaluations……) Knows how Knows “Domain specific” skills “Domain independent” skills

ACGME Core Competencies (The 6 Competencies) 1. 2. 3. 4. 5. 6. Patient Care Medical Knowledge Professionalism Interpersonal & Communication Skills Practice-Based Learning and Improvement Systems-Based Practice 16

What are assessment tools? Lab-based practical PBA(Procedure – based Assessment) DOPS آﺰﻣﻮﻥﻫﺎﻱ ﻋﻤﻠﻲ Practical

What are assessment tools? (Chart Stimulated Recall ) CSR یﺎ CBD GRF OSCEs Mini-CEX آﺰﻣﻮﻥﻫﺎﻱ ﺑﺎﻟﻴﻨﻲ Clinical

What are assessment tools? Structured ﺷﻔﺎﻫﻲ Unstructured

ﺧﺪﻣﺖ ﻣﺤﻞ ﺩﺭ ﻫﺎﻱ ﺍﺭﺯﻳﺎﺑﻲ ( Workplace Based Assessment) • Mini-CEX (Clinical Evaluation Exercise) • • • CBD یﺎ CSR (Chart Stimulated Recall) DOPS ( Direct Observation Of procedural Skills) PBA (Procedure – based Assessment) 360° ﺍﺭﺯیﺎﺑی (Mini-PAT) OSCE (objective structured clinical examination) GRF(Global Rating Form )

Climbing the Pyramid Does Shows how Knows Performance Assessment : Video, Real patient( Mini-CEX – DOPS-…) Competence Assessment : OSCE (Clinical) Context Based Tests: essay type, oral Knowledge Tests: MCQ

What’s in a name? O S C E bjective tructured linical xamination OSCE

OSCE Objective Structured Clinical Examination ﻋیﻨی ﺳﺎﺧﺘﺎﺭﺩﺍﺭ ﺑﺎﻟیﻨی آﺰﻣﻮﻥ

• . آﺰﻣﻮﻥ ﺗﻮﺍﻧﺎیی ﺍﻧﺠﺎﻡ ﻣﻬﺎﺭﺗﻬﺎی ﺑﺎﻟیﻨی ﺍﺳﺖ OSCE Students have to demonstrate their skills, not just describe theory §

Competencies: The SIX Domains

Objective Structured Clinical Examination q. Medical authorities & faculties started to implement the OSCE exams for clinical and communication skills assessment of medical students, clerks, interns, residents, and graduates since the 1980 s. q. It is a standardized method of clinical evaluation that eliminates bias. www. OSCEhome. com

Objective Structured Clinical Examination ØOSCE is a form of performancebased testing used to measure candidates’ clinical competency. ØDuring an OSCE, candidates are observed and evaluated as they go through a series of stations in which they interview, examine and treat standardized patients (SP) who present with some type of a medical problem. www. OSCEhome. com

Objective Structured Clinical Examination q“The OSCE is an approach to the assessment of clinical competence in which the components of competence are assessed in a planned or structured way with attention being paid to the objectivity of the examination”. Harden, 1988 www. OSCEhome. com

Objective Structured Clinical Examination q The curriculum tells the staff what to teach. . . q The OSCEs tells the students what to learn ! Ø And that is the point…. Ø What OSCE candidates should learn? Ø What candidates can do rather than what they know. www. OSCEhome. com

Objective Structured Clinical Examination q. The application of their knowledge rather than the recall of knowledge. q. The ‘WHAT’ to ask and do. q. The ‘HOW’ they ask and do. q. The ‘WHEN’ they ask and do. q. How do they organize themselves. q. How do they communicate with patients. www. OSCEhome. com

Objective Structured Clinical Examination • The OSCE has offered a strikingly new and exiting way of making valid assessment of clinical performance of medical student , resident, and fellows.

What is the purpose of the ? OSCE • Provide feedback on performance • Evaluate basic clinical skill • Measure minimal competency

What is OSCE? • It is some sort of student evaluation that examinee will go through several stations to be evaluated in clinical fields by special rating scales and checklists. M. Mohammadi MD

OSCE is Useful to Evaluate: • Clinical skill • Practical attitudes • Practical knowledge

OSCE =Performance-Base assessment (Skill Based Assessment) : Actual Performance of clinical skills is observed in order to adjudicate the competence of Health professionals

What is the format of OSCE?

What is the format of OSCE? • It consists of several Stations • (12 -20 each lasting 5 to 15 minutes not more than 30 seconds rest between each).

10 9 8 7 6 5 4 3 2 1

What is the stations’ contents?

What is the stations’ contents? • • • Case history Physical exam Real patient Standardize pt. Simulated pt. • • • History taking Clinical procedure Radiographs Slide presentation …

How do you evaluate the examinee?

How do you evaluate the examinee? • Checklist • Rating scale • Short answered questions • …

Who are the Raters?

Who are the Raters? • • • Simulated pt. Standardized pt. Clinical staff Non clinical staff Non expert raters

OSCE is suggested the first best method for evaluation of: • Interviewing • • Counseling & educating pt’s & families Performing procedures Physical examination Preventive health services Creation of therapeutic relationship with patients Listening skills Respectful & altruistic behaviors Sensitive to cultural, age, gender & disability issues

12/7/2020 79

GRF, 360 -degree DOPS, Mini-CEX OSCE MCQ