DRIVESENSE HIGH COGNITIVE LOAD MONITORING SYSTEM PROJECT UPDATE

DRIVESENSE HIGH COGNITIVE LOAD MONITORING SYSTEM PROJECT UPDATE 8/13/2015 MIRJANA SPASOJEVIC, SR. PRINCIPAL SCIENTIST FUTURE EXPERIENCE/CTG 1

PROJECT GOALS AND MILESTONES RFP: Select biometric sensors, software and hardware vendors Development: Integration with a driving simulator Validation: Testing with human subjects in realistic scenarios 2

EXECUTIVE SUMMARY HIGH COGNITIVE LOAD MONITORING SYSTEM OPPORTUNITY SPACE: HIGH COGNITIVE LOAD DETECTION ENABLING TECHNOLOGIES: PUPILOMETRY + DRIVER-FACING CAMERA DRIVESENSE: • • • First HARMAN Driver Monitoring initiative Introduce high-cognitive load detection to the market Leverage trend of growing driver-facing camera install rate No need for body-worn or seat-embedded sensors Basic Demonstration Target: CES 2016 Future Plans: HMI & ADAS intervention HARMAN International. Confidential. Copyright 2015. 3

DRIVE SENSE APPROACH 1 Monitor driver bio-signals 2 Analyze cognitive load. 3 Intervene by modifying HMI/ ADAS. eye tracker + pupillometry software 4

USE CASES AND INTERVENTIONS monitor driver state HMI activates Do Not Disturb for calls, texts adapt HMI INTERVENE? sensors monitor vehicle speed, traffic, time of day, road conditions. . . vehicle/ environment assist ADAS alert driver* ALERT! High workload state detected. Please focus on the road. (*) via other modalities HMI navigation suggests easier routes provide input for ADAS warnings, collision parameters 5

DRIVER MONITORING ARCHITECTURE sensor processing sensors Driving Context Sensing data stored in head unit or in the cloud visualizations Driver State Driver Activity Driving Simulator Metrics Real time alerts Compute Cognitive Load Adapt HMI Assist ADAS Vehicle Driver alerts Environment Workload monitor Database SYSTEM ARCHITECTURE Interventions components implemented in V 1 6

DRIVESENSE COMPONENTS Vendor Product Features • Tracks pupil size and gaze direction Seeing Machines FOVIO eye tracker (hardware) Eye. Tracking Inc. (lead integrator) Eye. Works and Workload Module (software) • Calculates Index of Cognitive Activity (ICA) based on pupil size NADS Mini. SIM • Driving simulator hardware and software • Driving scenario authoring environment National Advanced Driving Simulator, Univ. of Iowa (harwdare, software) 7

SYSTEM OVERVIEW SEEING MACHINES EYETRACKING INC. DRIVING SIMULATOR NEW COMPONENT DEVELOPED FROM NADS, U OF IOWA BY EYETRACKING INC. 8

HOW DOES PUPILLOMETRY WORK? INDEX OF COGNITIVE ACTIVITY (ICA) Pupil data Approach: Measure pupil diameter via eye tracker to separate light reflex from a cognitive load reflex Light Reflex: less light larger pupils duration 100 ms Cognitive Reflex: high cognitive load larger pupils duration 3 -10 ms ICA: Reflects abrupt changes in pupil size due to cognitive reflex DSP filters ICA 9

HOW TO VALIDATE? ICA metric Simulator metric 1. Set up a realistic driving scenario (e. g. , 20 min highway drive) 2. Create secondary tasks that will increase cognitive load (e. g. , listening to radio, interacting with voice mail, doing hard mental problems) 3. Collect Cognitive Load metrics (ICA) 4. Collect simulator metrics (SM) that reflects quality of driving (e. g. , braking response time, distance to lead car, lane departure) 5. Demonstrate correlation between changes in CL and SM: o Demonstrate that when CL increases the SM shows that the driving gets worse 10

CES DEMO SETUP: HARDWARE operator console (will be hidden) Quarter cabin driving simulator 11

CES DEMO SETUP: HARDWARE Fovio eye tracker from Seeing Machines (research version) Note: vendor is working on a compact version that is more suitable for embedding in the dashboard. 12

TEST DRIVER EXPERIENCE § 5 driving scenarios with secondary tasks § Driving metrics reported distance to lead car brake event brake pedal pressure § Workload reported by cognitive workload module (pupil data) Video link 13

REAL TIME DATA VISUALIZATION (EXAMPLE) lead braking distance to lead car driver braking workload medium Workload Gauge high low Workload vs Simulator Metrics 14

VISUALIZATION DISPLAY Custom visualization § Workload gauge § Simulator metrics (Note: upper middle screen has green dot that shows where the driver is looking) Video link 15

PROJECT MILESTONES AND NEXT STEPS M 1 M 2 M 3 • Issued RFP, selected vendors and signed the contract • Completed May 2015 • • Hardware and software acquisition Component Integration and new data visualization developed Testing with human subjects and system calibration (by Aug 31, 2015) Full report September 2015 • CES 2016 physical demonstrator production • Prepare marketing documents and video • Due: January 2016 16