OSCE reliability repeatability reproducibility the ability to perform
OSCE 評分 信度 (reliability, repeatability, reproducibility) the ability to perform its functions consistently 效度 (validity) the tool measures what it claims to measure
OSCE 評分 信度 (reliability, repeatability, reproducibility) 效度 (validity) ‧ ‧ ‧‧‧‧‧
OSCE 評分 標準化病人 validity reliability 個案 validity reliability 評估表 validity reliability 考官 validity reliability
考官訓練 寬鬆效應 (leniency effect, Kneeland 1929): rating the positive more positive 嚴苛效應 (severity effect, Ford 1931): rating the negative more negative 月暈效應 (halo effect): rating of a trait influences the rating of another trait 隨機效應 (randomness effect): 評分者沒有背景知識或訓練不足,以致於無法對量尺作出適當 的區別,因而採用一種隨意的方式來對受試者進行評估。 http: //www. bctest. ntnu. edu. tw/flying 41 -50/flying 43 -5. htm
考官訓練 • 說明會 • 影音演練 • Discussion group • Modified Delphi-Method
提升客觀性 Direct Observation of Competence Training (Muller, 1998; Holmboe, 2004; Angkaw, 2006) 1. Performance dimension training 2. Frame-of-reference training 3. Behavioral observation training
Behavioral observation training Role-play : Best VS. Worst
提升客觀性 Differential rater function over time (DRIFT) (Mclaughlin, 2009; Wolfe, 2001; Klein, 2003; Hopkins, 1998. ) 1. Rater fatigue 題目難與易? 2. 15 -20分鐘熱身 (warming-up) 可提升sensorimotor & cognitive performance?
評分標準建議 • 評分表評分項目逐項說明 Criterion-based design(同2011) • Modified Delphi-method rater training Rating practice by Video + Real-time discussion forum on internet 30
及格標準設定 • 實作評量之標準設定方法,大致可以歸納成三大類(Berk, 1984, 1986; Hambleton, 1990; Hambleton & Zaal, 1991): 1. judgmental methods Nedelsky method Ebel method Angoff method 2. Empirical methods Livingston method Linden & Mellenbergh method 3. Combination methods Borderline-group method Contrasting-group method 31
Example from Boursicot(2003) y=ax+b 譚克平教授 33
Angoff方法 �� 1971 �� popular �� 很多變種 �� Modified Angoff �� Extended Angoff �� Yes/No 譚克平教授提供
- Slides: 50