Evaluation of a Simulator Based Novice Driver Risk

Evaluation of a Simulator Based, Novice Driver Risk Awareness Training Program A Master Thesis Presented by Frank Diete, M. S. IEOR 27 th September, 2007 Committee: Prof. Donald Fisher, Prof. John Collura, Prof. Sundar Krishnamurty

Outline § Review of Motivation § Experimental Design • Training Experimental Group • Training Control Group • Simulator Evaluation § Scoring and Results • Training Results • HPL Simulator Results § Conclusion Master Thesis Defense, presented by Frank Diete, 09. 27. 2007 2

Background and Motivation Young, novice drivers are overrepresented in car crashes (Insurance Institute for Highway Safety (2006). Fatality Facts 2005: Older people. Retrieved July 6, 2007, from http: //www. iihs. org/research/fatality_facts/olderpeople. html#sec 0) Master Thesis Defense, presented by Frank Diete, 09. 27. 2007 3

The reasons are failures in search Field Studies Pradhan et al. (2005)* Police Accidents Reports Eye Tracking Data - Scanning Patterns (Mc Knight & Mc Knight, 2003) (Mourant and Rockwell (1972), Crundall & Underwood (1998)) Failures in Selective Attention Simulator Study (Pradhan et al. (2005)*) Novice drivers do not show an appropriate tactical scanning behavior Master Thesis Defense, presented by Frank Diete, 09. 27. 2007 4 *Pradhan, A. K. , Hammel, K. R. , De. Ramus, R. , Pollatsek, A. , Noyce, D. A. , & Fisher, D. L. (2005). The Use of Eye Movements to Evaluate the Effects of Driver Age on Risk Perception in an Advanced Driving Simulator. Human Factors, 47, 840 -852.

Risk Awareness and Perception Training (RAPT) RAPT-1 was developed and evaluated on the HPL driving simulator by Pollatsek et al. (2006)* § RAPT does … • … train novice drivers where to look § RAPT does not … • … train manual skills • … train time sharing skills After training 90% of critical areas in plan views could be identified by participants. Master Thesis Defense, presented by Frank Diete, 09. 27. 2007 5 *Pollatsek, A. , Narayanaan, V. , Pradhan, A. , & Fisher, D. L. (2006 b). The use of eye movements to evaluate the effect of PC-based risk awareness training on an advanced driving simulator. Human Factors, 48, 447– 464.

Simulator Evaluation Results Total Effect of Training (57. 7% vs. 35. 4%) Far Transfer Scenarios (63. 5% vs. 43. 5%) Near Transfer Scenarios (51. 9% vs. 27. 3%) What explains this big difference in performance? 1) Difficulty to generalize from plan views to a 2) perspective view as seen on the driving simulator Pollatsek et al. , 2006* 2) Necessity of practicing hazard anticipation at the same time as Although 90% of risky areas could be identified in the training, only 20. 0% 24. 6% driving Multitasking does not occur automatically! 52% of these areas were fixated in the evaluation! 22. 3% Master Thesis Defense, presented by Frank Diete, 09. 27. 2007 6 *Pollatsek, A. , Narayanaan, V. , Pradhan, A. , & Fisher, D. L. (2006 b). The use of eye movements to evaluate the effect of PC-based risk awareness training on an advanced driving simulator. Human Factors, 48, 447– 464.

Link to proposed study A training program using both RAPT and the Drive Square simulator might result in better training effects. RAPT* SIMRAPT + = Better Hazard Anticipation? Master Thesis Defense, presented by Frank Diete, 09. 27. 2007 7 *Pollatsek, A. , Narayanaan, V. , Pradhan, A. , & Fisher, D. L. (2006 b). The use of eye movements to evaluate the effect of PC-based risk awareness training on an advanced driving simulator. Human Factors, 48, 447– 464.

Experiment Research Hypotheses § Hypothesis 1: A training program using a combination of plan views (RAPT) and a head mounted driving simulator (SIMRAPT) will result in trained participants recognizing risks significantly more often on the HPL driving simulator than untrained novice drivers. The effects of training will generalize from the combined RAPT and SIMRAPT training program to scenarios on the HPL driving simulator that measure both the near and far transfer of the ability to recognize risks. § Hypothesis 2: The difference in performance between trained and untrained novice drivers using the combined RAPT/SIMRAPT training program will be larger than in past studies which used only the RAPT training program where performance in this and previous studies was evaluated on the HPL driving simulator using the same scenarios. Master Thesis Defense, presented by Frank Diete, 09. 27. 2007 8

Outline § Review of Motivation § Experimental Design • Training Experimental Group • Training Control Group • Simulator Evaluation § Scoring and Results • Training Results • HPL Simulator Results § Conclusion Master Thesis Defense, presented by Frank Diete, 09. 27. 2007 9

Experiment Design The effects of the training were evaluated by comparing simulator performance of a trained experimental and an pseudo-trained control group Experimental Group (12 participants) RAPT-1 Participants § 16 or 17 years old § Less than 6 month driving experience SIMRAPT HPL Simulator Evaluation Control Group (12 participants) Mass RMV Pseudo-SIMRAPT Compare Master Thesis Defense, presented by Frank Diete, 09. 27. 2007 HPL Simulator Evaluation 10

Participants Experimental Group Control Group 10 permit drivers 2 license drivers 7 male 5 female Average Age: 16. 5 years STDV: 0. 4 years Master Thesis Defense, presented by Frank Diete, 09. 27. 2007 Average Age: 16. 5 years STDV: 0. 5 years 11

Experiment Design The effects of the training were evaluated by comparing simulator performance of a trained experimental and an pseudo-trained control group Experimental Group (12 participants) RAPT-1 Participants § 16 or 17 years old § Less than 6 month driving experience SIMRAPT HPL Simulator Evaluation Control Group (12 participants) Mass RMV Pseudo-SIMRAPT Compare Master Thesis Defense, presented by Frank Diete, 09. 27. 2007 HPL Simulator Evaluation 12

Training Program: Experimental Group RAPT SIMRAPT Instructions Pre-Test / Training on 8 scenarios Post-Test The slides for the RAPT Power. Point were developed by Anuj Pradhan for prior studies. Master Thesis Defense, presented by Frank Diete, 09. 27. 2007 13

Training Program: Experimental Group RAPT Driver feels comfortable yes SIMRAPT Practice Drive no Drive 1 st scenario Written Instructions Present plan view And feedback Drive next scenario no Correct head-turn done? no Repeat scenario Master Thesis Defense, presented by Frank Diete, 09. 27. 2007 yes All scenarios driven? yes End training 14

Experiment Design The effects of the training were evaluated by comparing simulator performance of a trained experimental and an pseudo-trained control group Experimental Group (12 participants) RAPT-1 Participants § 16 or 17 years old § Less than 6 month driving experience SIMRAPT HPL Simulator Evaluation Control Group (12 participants) Mass RMV Pseudo-SIMRAPT Compare Master Thesis Defense, presented by Frank Diete, 09. 27. 2007 HPL Simulator Evaluation 15

Training Program: Control Group Mass RMV Training Drive Square Simulator Nothing related to hazard anticipation Master Thesis Defense, presented by Frank Diete, 09. 27. 2007 16

Training Program: Control Group Mass RMV Training Driver feels comfortable yes Drive Square Simulator Practice Drive no Drive all 8 scenarios one after another Written Instructions Drive all 8 scenarios again in another order No feedback given End training Master Thesis Defense, presented by Frank Diete, 09. 27. 2007 17

Experiment Design The effects of the training will be evaluated by comparing simulator performance of a trained experimental and an pseudo-trained control group Experimental Group (12 participants) RAPT-1 Participants § 16 or 17 years old § Less than 6 month driving experience SIMRAPT HPL Simulator Evaluation Control Group (12 participants) Mass RMV Pseudo-SIMRAPT Compare Master Thesis Defense, presented by Frank Diete, 09. 27. 2007 HPL Simulator Evaluation 18

HPL simulator evaluation Both the experimental and the control group were evaluated on the HPL driving simulator § Evaluation immediatly after the training § Instructions „obey traffic laws etc. “… § Calibration of the eye-tracker § Participants are asked to follow a lead vehicle § Practice Drive § 16 Scenarios in four blocks 8 near and 8 far transfer scenarios* *The scenarios for the simulator evaluation had been developed by Anuj Pradhan at the Human Performance Lab for prior studies. Master Thesis Defense, presented by Frank Diete, 09. 27. 2007 19

The study in a movie Master Thesis Defense, presented by Frank Diete, 09. 27. 2007 20

Outline § Review of Motivation § Experimental Design • Training Experimental Group • Training Control Group • Simulator Evaluation § Scoring and Results • Training Results • HPL Simulator Results § Conclusion Master Thesis Defense, presented by Frank Diete, 09. 27. 2007 21

Results RAPT Training Master Thesis Defense, presented by Frank Diete, 09. 27. 2007 22

SIMRAPT Results – Experimental Group Sequences of scenarios in SIMRAPT: 1 st participant: 1 – 2 – 3 – 4 – 5 – 6 – 7 – 8 2 nd participant: 2 – 3 – 4 – 5 – 6 – 7 – 8 – 1 3 rd participant: 3 – 4 – 5 – 6 – 7 – 8 – 1 – 2 …. . Scoring in SIMRAPT: • 0, 0. 5 or 1 point for each scenario and drive • Number of trials counted for each scenario and drive Master Thesis Defense, presented by Frank Diete, 09. 27. 2007 23

SIMRAPT Results – Experimental Group 79% 0% Average Score first trial Master Thesis Defense, presented by Frank Diete, 09. 27. 2007 2. 58 1. 33 Average number of trials 24

SIMRAPT Results – Experimental Group Average Score first trial Master Thesis Defense, presented by Frank Diete, 09. 27. 2007 Average number of trials 25

Drive Square Results – Control Group Average Score 1 st and 2 nd run by scenario. Slight improvement from 8. 9% to 14. 1%, but not significant (t=1. 93, p>. 05) Master Thesis Defense, presented by Frank Diete, 09. 27. 2007 26

Conclusion of training results § High and significant training effect for the experimental group from the 1 st to the 8 th drive on SIMRAPT Generalization within training § Tendency of small training effect for control group from 1 st to 2 nd run on Drive Square Master Thesis Defense, presented by Frank Diete, 09. 27. 2007 27

Outline § Review of Motivation § Experimental Design • Training Experimental Group • Training Control Group • Simulator Evaluation § Scoring and Results • Training Results • HPL Simulator Results § Conclusion Master Thesis Defense, presented by Frank Diete, 09. 27. 2007 28

HPL Simulator Evaluation - Scoring Each scenario* was scored 1 or 0 depending on eye fixation behavior! *The scenarios for the simulator evaluation had been developed by Anuj Pradhan at the Human Performance Lab for prior studies. Master Thesis Defense, presented by Frank Diete, 09. 27. 2007 29

Near Transfer – Opposing Truck Left Turn scenario Risk Recognized Master Thesis Defense, presented by Frank Diete, 09. 27. 2007 Risk Not Recognized 30 *The scenarios for the HPL driving simulator that are used for this study and the slides for the Power Point Training had been developed by Anuj Pradhan at the Human Performance Lab within the last years.

Far Transfer – Mullins Center scenario Risk Recognized Master Thesis Defense, presented by Frank Diete, 09. 27. 2007 Risk Not Recognized 31 *The scenarios for the HPL driving simulator that are used for this study and the slides for the Power Point Training had been developed by Anuj Pradhan at the Human Performance Lab within the last years.

Results for simulator evaluation Test of Hypothesis 1: Significant training effect of combined training program? T-Intersection Amity-Lincoln Left Fork Opposing Truck Left Turn Pedestrians on Left Intersection with One-Way Street All 16 scenarios included in evaluation! Mullins Center Bus Left Turn at Triangle St Signal Ahead at hill Right on Red Truck Crosswalk Curved Stop Ahead Adjacent Truck Left Turn Truck Blocking Travel in Lane Blind Drive Intersection with new green Master Thesis Defense, presented by Frank Diete, 09. 27. 2007 32

Results for simulator evaluation Experimental Control Difference Significance Near Transfer 72. 2% 37. 5% 34. 7% t(18)=3. 1, p<. 01 Far Transfer 72. 5% 57. 3% 15. 2% t(20)=2. 1, p=. 052 Total 72. 4% 46. 9% 25. 5% t(20)=3. 2, p<. 01 Master Thesis Defense, presented by Frank Diete, 09. 27. 2007 33

Results for simulator evaluation Pretty consant training effect for all near transfer scenarios Generalization only for particular far transfer scenarios Master Thesis Defense, presented by Frank Diete, 09. 27. 2007 34

Results for simulator evaluation Master Thesis Defense, presented by Frank Diete, 09. 27. 2007 35

Results for simulator evaluation Test of Hypothesis 2: Training effect of combined training program higher than for just PC-based training as in Pollatsek et al. (2006)? T-Intersection with One-Way Street Amity-Lincoln Mullins Center Left Fork 11 scenarios included Opposing Truck Left Turn Pedestrians on Left Bus Left Turn at Triangle St Signal Ahead at hill in evaluation! Right on Red Truck Crosswalk Curved Stop Ahead Adjacent Truck Left Turn Truck Blocking Travel in Lane Blind Drive Intersection with new green Master Thesis Defense, presented by Frank Diete, 09. 27. 2007 36

Results for simulator evaluation The non-published variances and means for the RAPT-1 evaluation are based on former work at the Human Performance Lab and were provided for use in this study by Anuj Pradhan, graduate student at UMASS Amherst. Master Thesis Defense, presented by Frank Diete, 09. 27. 2007 37

Results for simulator evaluation The non-published variances and means for the RAPT-1 evaluation are based on former work at the Human Performance Lab and were provided for use in this study by Anuj Pradhan, graduate student at UMASS Amherst. Master Thesis Defense, presented by Frank Diete, 09. 27. 2007 38

Results for simulator evaluation SIMRAPT Evaluation (Pollatsek et al. (2006) Trained Untrained difference Near Transfer 67. 5 36. 1 31. 4 48. 4 27. 2 21. 2 Far Transfer 81. 3 49. 0 32. 3 57. 8 35. 3 22. 5 Total 72. 2 42. 3 29. 9 52. 0 31. 1 20. 9 The non-published variances and means for the RAPT-1 evaluation are based on former work at the Human Performance Lab and were provided for use in this study by Anuj Pradhan, graduate student at UMASS Amherst. Master Thesis Defense, presented by Frank Diete, 09. 27. 2007 39

Outline § Review of Motivation § Experimental Design • Training Experimental Group • Training Control Group • Simulator Evaluation § Scoring and Results • Training Results • HPL Simulator Results § Conclusion Master Thesis Defense, presented by Frank Diete, 09. 27. 2007 40

Conclusions 1. RAPT Recognize Risks from Plan View 2. SIMRAPT Ability to recognize risks in driving task on Drive Square Simulator 3. RAPT/SIMRAPT Ability to fixate on areas of potential risk on HPL driving simulator Training succeeded for particular scenarios 4. Training effects exceed those of PC Training, though not significantly Master Thesis Defense, presented by Frank Diete, 09. 27. 2007 41

Suggestions for future research § Generalization Deversify training § Duration of the training Redesign Training § Effects of simulator driving for control group More extensive study necessary Master Thesis Defense, presented by Frank Diete, 09. 27. 2007 42

Thank you for your attention Any Questions? Additions? Suggestions? Master Thesis Defense, presented by Frank Diete, 09. 27. 2007 43