Transmitted by the expert of Japan Informal Document

Transmitted by the expert of Japan Informal Document VMAD-01 -05 1 st VMAD IWG, January 14 -15, 2019 Agenda item 5. How to consider Innovative Validation Methods for Safety of Automated Driving System January 14 -15, 2019 MLIT, Japan

Contents 1. Presentation at 176 th WP. 29 session in Nov. 2018 (informal document WP. 29 -176 -29) 2. New assessment / Test Method 3. Structure of the Translation Process with an example 4. Scenario Structure 5. Scenario Database 6. Test Scenario Catalog 2

1. Presentation at 176 th WP. 29 session on Nov. ’ 18 Translation Process 3

2. New assessment /Test method Translation Process Classification WP 29 GRVA IWGs Concrete examples based on JP guideline Objective ü Vehicle safety concept ü ü ODD, Applicable systems ü ü Related laws/issues Each region issue Measurable criteria ü Validation methods ü ü To realize society where accidents caused by ADS resulting in injury or death become zero ADS, under their ODD, shall not cause any traffic accidents resulting in injury or death that are rationally foreseeable and preventable • ODD: Highway, No snow/Ice, Day/Night • Applicable system: Highway chauffer • Liability, Driver's responsibility • Traffic rule, etc. Normal case • Cause: ADS is responsible • Foreseeable: no unrealistic scenario • Preventable: No illegal and No extreme conditions Measurable criteria　　Test scenario MRM case Track tests, simulation, on road tests, etc. under scenarios randomly picked up from the test scenario database 4

3. Structure of the translation process with an example Input from WP 29 & GRVA Vehicle safety concept ODD, Applicable system Related laws/issues Page 6, 7 ADS, under their ODD, shall not cause any traffic accidents resulting in injury or death that are rationally foreseeable and preventable • ODD: Highway, No snow/Ice, Day/Night • Applicable system: Highway chauffer • Liability, Driver's responsibility • Traffic rule Traffic flow observation data A Foreseeable scenario B Preventable: No illegal and No extreme conditions Consider validation method Track On road … Audit tests Structured all possible traffic scenario in the ODD Accident Taxonomy Page 8 A Foreseeable: empirically predictable scenario w. r. t observed field data Validation method Scenario Structure Field data Discussion points in IWG Output of IWG Substantiation Iteration Process Scenario DB (no unrealistic scenario) Page 9 Test scenario Selected scenario for certification B Preventable scenario Test Scenario Catalog (No illegal and No extreme conditions) 5

4. Scenario Structure (1/2) AD scenario category can be separated from the view point of the physics of AD system, which is perception, judgment, and maneuver in order to cover holistic root cause. Scenario Structure a Perception limitation sensor malfunction b Traffic Disturbance Traffic participants’ unsafe behavior Road Ego Vehicle c Vehicle Disturbance Cause of vehicle instability Traffic Participants Surrounding Vehicle - position - behavior 6

4. Scenario Structure (2/2) Scenario Structure a Extract the most demanding parameter from real environment data for each road geometry classification based existing laws and regulations. b Define 8+1 position around ego-vehicle and movement which can invade the ego trajectory. a Ego-vehicle behavior Road geometry b Traffic participants’ position surrounding vehicle behavior Road geometry Ego-vehicle behavior Lane keep Lane change Free Driving Following Lane change Overtaking Merge Merging in front Merge Branch --- Branch Ramp Free Driving Following Lane change Overtaking Main roadway Road Structure Ordinance 7

5. Scenario DB Check Scenario Structure by real world data a b The monitored traffic flow data and accident data can be utilized to check if the scenario catalog can cover the foreseeable scenario. Define scenario parameter boundary The monitored traffic flow data is needed not only to check the scenario coverage but also to define the scenario parameter range based on the real world traffic behaviors. Field data Accident Taxonomy e. g) … … … GIDAS Pre-Crash Scenario Typology(NHTSA) classification a … Scenario　Structered w/quantitative parameter range region/country specific Traffic flow observation data Fixed Camera data Trajectory extraction Parameter range Scenario DB b Equipped vehicle data Foreseeable scenario 8

6. Test Scenario catalog Exclude parameter ranges unavoidable for automated driving vehicle Foreseeable Scenario extract Preventable Scenario 9