SBAS and GBAS Integrity for Non Aviation Users

SBAS and GBAS Integrity for Non. Aviation Users: Moving Away from "Specific Risk" Sam Pullen, Todd Walter, and Per Enge Stanford University spullen@stanford. edu ION ITM 2011 San Diego, CA. 25 January 2011 Integrity for Non-Aviation Users: Moving Away from "Specific Risk" 1

Motivation (1): SBAS and GBAS for Non. Aviation Users • Where augmentation signals can be received, SBAS and GBAS benefits are available to all users. • However, integrity algorithms in airborne MOPS are designed to support specific aviation applications. – Resulting integrity protection levels are not wellsuited for other classes of users • Correcting this would increase the attractiveness of SBAS and GBAS to non-aviation transport users (auto, rail, marine) and others. 25 January 2011 Integrity for Non-Aviation Users: Moving Away from "Specific Risk" 2

Motivation (2): Accuracy and Integrity Illustrative example – not to scale or direction HPL (nonaviation application) 95% HPE HPL (per MOPS) • Accuracy bounds (e. g. , 95% vertical position error, or VPE) can be measured and modeled with high precision • Integrity bounds (e. g. , 10 -7 vertical protection level, or VPL) cannot be – – 25 January 2011 Lack of sufficient measurements Flaws in Gaussian extrapolations to low probabilities Dependence on details of failure models and assumptions Too little is known; too much is uncertain… Integrity for Non-Aviation Users: Moving Away from "Specific Risk" 3

WAAS VPE vs. VPL from FAA PAN Data (3 rd Qtr 2010: July – Sept. ) Source: WAAS PAN Report #34, Oct. 2010. VPL (m) http: //www. nstb. tc. faa. gov/ Display. Archive. htm Max. VPE 7 m (at Barrow, AK) 25 January 2011 95% VPE 1. 2 m 99% VPE Integrity 1. 6 m for Non-Aviation. VPE Users: (m) Moving Away from "Specific Risk" 4

WAAS Reference Station Classifications (for this study only) Figure source: FAA GNSS Press Kit http: //preview. tinyurl. com/4 ofdzz 4 18 Remote Stations 13 Outer Stations 7 Inner Stations 25 January 2011 Integrity for Non-Aviation Users: Moving Away from "Specific Risk" 5

Max. VPE and VPL from FAA PAN Data (1 Jan. 2004 – 30 Sept. 2010) Worst Case Between “Inner” and “Outer” WAAS Stations -> “In. Out” Set 50 VPE or VPL (meters) 45 40 Max. VPL 35 30 25 20 15 10 5 0 Max. VPE 95% VPE 10 15 20 25 30 35 Quarterly PAN Report Number (8 – 34) 25 January 2011 Integrity for Non-Aviation Users: Moving Away from "Specific Risk" 6

Max. HPE and HPL from FAA PAN Data (1 Jan. 2004 – 30 Sept. 2010) Worst Case Between “Inner” and “Outer” WAAS Stations -> “In. Out” Set 45 As expected, both HPE and HPL are significantly lower than VPE and VPL. 40 HPE or HPL (meters) 35 One unusual result: 12 m error at Cleveland in Spring 2005 (correct number? ) 25 January 2011 30 Max. HPL 25 20 15 10 Max. HPE 5 95% HPE 0 10 15 20 25 30 Quarterly PAN Users: Report Number – 34)Risk" Integrity for Non-Aviation Moving Away from(8 "Specific 35 7

Ratio of Max. VPL and Max. VPE from FAA PAN Data (“In. Out” Station Set) 9 Less error reduction after PAN #20 (March 2007). VPL / VPE Ratio for Max. Cases 8 7 6 Mean Ratio = 5. 38 5 4 Noticeable improving trend likely due to error reduction at individual WAAS reference stations. 3 2 1 0 10 15 20 25 30 35 Quarterly PAN Report Number (8 – 34) 25 January 2011 Integrity for Non-Aviation Users: Moving Away from "Specific Risk" 8

Ratio of Max. HPL and Max. HPE from FAA PAN Data (“In. Out” Station Set) 10 HPL / HPE Ratio for Max. Cases 9 Unusual error at Cleveland (if correct) just barely exceeded by HPL. 25 January 2011 8 7 Mean Ratio = 5. 21 6 5 4 3 Weaker but visible improving trend – more variability. 2 1 0 10 15 20 25 30 Quarterly PAN Report Number (8 – 34) Integrity for Non-Aviation Users: Moving Away from "Specific Risk" 35 9

How Many Samples Were Collected? All validated PAN data from 1 Jan. 2004 to 30 Sept. 2010 Assume data correlated over 600 sec (10 min) 7. 1 10 -6 independent samples 25 January 2011 Assume data correlated over 150 sec (~ one CAT I approach) 2. 8 10 -7 independent samples Assume data correlated over 30 sec 1. 4 10 -8 independent samples Integrity for Non-Aviation Users: Moving Away from "Specific Risk" 4. 25 10 -9 sec (49, 324. 6 days) (105. 04 years) 10

Average vs. Specific Risk Assessment • Average Risk (my definition): the probability of unsafe conditions based upon the convolved (“averaged”) estimated probabilities of all unknown events. – Probabilistic Risk Analysis (PRA) is based on this procedure – Risk aversion and value of information (VOI) are applied to the outputs of PRA integrity risk requirements, alert limits • Specific Risk (my definition): the probability of unsafe conditions subject to the assumption that all (negative but credible) unknown events that could be known occur with a probability of one. – Evolved from pre-existing FAA and ICAO safety standards – Risk aversion and VOI and buried inside specific risk analysis – Results (risk and protection levels) are inconsistent with PRA 25 January 2011 Integrity for Non-Aviation Users: Moving Away from "Specific Risk" 11

Simplified Example: Ionospheric Spatial Decorrelation (1) Severe Ionospheric Storm Observed over CONUS on 20 November 2003 20: 15 UT 25 January 2011 21: 00 UT Integrity for Non-Aviation Users: Moving Away from "Specific Risk" 12

Simplified Example: Ionospheric Spatial Decorrelation (2) • Using PRA, estimated “prior” probabilities of severe decorrelation are combined with the likelihood of SBAS or GBAS mitigation to derive resulting user risk. – Prior probabilities need not be known precisely – Benefits of improved mitigation (“better information”) appear naturally as lower integrity risk. • Under FAA interpretation of Specific Risk, worst-case iono. delay gradient is “credible” and thus is assigned a probability of one. – Worst-case for GBAS (CAT I): an extremely large gradient that escapes detection by “matching speed” with ground station » This differs in real time for each site and GNSS geometry – Worst-case for SBAS (LPV): a very large gradient that is just small enough to avoid detection by master station 25 January 2011 Integrity for Non-Aviation Users: Moving Away from "Specific Risk" 13

Simplified Example: Ionospheric Spatial Decorrelation (3) Simulated results for Memphis GBAS impacted by severe ionospheric gradient (RTCA 24 -SV GPS, 6 -km, User-to-ground separation, 1 and 2 -SV impacts) 0. 14 Most errors are exactly zero due to ground detection and exclusion, but all zero errors have been removed from the histogram. 0. 12 0. 1 Most plotted (non-zero) errors are below 10 m even under severe conditions. PDF 0. 08 0. 06 Worst-case error, or “MIEV”, is 41 m 0. 04 0. 02 0 25 January 2011 0 5 10 15 20 25 30 35 User Vertical Position Error (meters) Integrity for Non-Aviation Users: Moving Away from "Specific Risk" 40 45 14

Benefits of an “Average Risk” Approach (Potential SBAS PL Reduction) 40 35 95% VPL or HPL (meters) 30 25 95% HPL 20 15 5 0 24 Conservative reduction factors from PAN data: Adjusted VPL 10 From reports since Jan. 2008 Max. 95% PLs among stations in CONUS (“In. Out” set) VPL / 4. 0 Adjusted HPL 25 26 27 28 HPL / 2. 5 29 30 31 32 33 34 PAN Report Number • “Average risk” approach supports large reductions in HPL and VPL implied by WAAS PAN data, pending more complete database analysis. • Use “full-scale” PRA to re-assess “rare-normal” and faulted errors. 25 January 2011 Integrity for Non-Aviation Users: Moving Away from "Specific Risk" 15

A Combined “Average/Specific” Risk Approach • Depending on user and decision maker risk aversion, separate “average risk” and “specific risk” integrity requirements could be issued. – Both apply at all times one or the other will tend to dominate for a particular application. • For example: 10 -7 integrity risk per operation (“average”) plus requirement that a worst-case undetected condition cannot increase the total vehicle loss risk by more than a factor of 10. – For aircraft case, factor of 10 increase in total risk equates to specific risk requirement of 10 -5 per operation for nav. system (more strictly, 9 10 -6) – Specific factors for each vehicle and application would vary. – There is no “correct” degree of risk aversion. 25 January 2011 Integrity for Non-Aviation Users: Moving Away from "Specific Risk" 16

Summary • Existing integrity assurance procedures for SBAS and GBAS are unique to aviation and its history and may not be suitable for other users. • SBAS (and GBAS) data analysis suggests that 10 -7 HPL and VPL can be greatly reduced if “average risk” approach is taken. – Examination of past data is useful, but more thorough PRA analysis should be conducted. • If worst-case elements of risk assessment are still desired, an average/specific risk mixture can be used. – This flexible “mixture” capability should satisfy almost any level of user and decision maker risk aversion. 25 January 2011 Integrity for Non-Aviation Users: Moving Away from "Specific Risk" 17

Backup Slides follow… 25 January 2011 Integrity for Non-Aviation Users: Moving Away from "Specific Risk" 18

WAAS VPE from FAA PAN Data (3 rd Qtr 2010: July – Sept. ) No. of Samples Source: WAAS PAN Report #34, Oct. 2010. http: //www. nstb. tc. faa. gov/ Display. Archive. htm Max. VPE 7 m at Barrow, AK Meas. from 37 WAAS stations 25 January 2011 Integrity for Non-Aviation. VPE Users: (m) Moving Away from "Specific Risk" 19

(from PAN #34) Example Error Table from PAN #34 25 January 2011 Integrity for Non-Aviation Users: Moving Away from "Specific Risk" 20

Max. VPE and VPL from WAAS PAN Data (1 Jan. 2004 – 30 Sept. 2010) (all numbers are in meters) 25 January 2011 Integrity for Non-Aviation Users: Moving Away from "Specific Risk" 21

95% and Max. VPE from FAA PAN Data (1 Jan. 2004 – 30 Sept. 2010) 50 Note: VPL always bounds VPE. Vertical Position Error (meters) 45 Severe iono. scintillation in Alaska in March and May 2007 (user receiver should prevent) 40 35 VAL for LPV Remote Stations 30 25 Max. VPE 20 15 95% VPE Outer Stations 10 Inner Stations 5 0 10 15 20 25 30 35 Quarterly PAN Report Number (8 – 34) 25 January 2011 Integrity for Non-Aviation Users: Moving Away from "Specific Risk" 22

WAAS VPE vs. VPL in CONUS (2003 – 2006) (from Wanner, et al, 2006) Vertical Position Error (meters) Ratios: 6. 9 6. 4 6. 8 6. 5 6. 0 6. 8 99. 99% VPE 99. 9% VPE 95% VPE Mean VPE 1 s VPE 25 January 2011 Integrity for Non-Aviation Users: Moving Away from "Specific Risk" 23

WAAS Max. VPE in CONUS (2003 – 2006) (from Wanner, et al, 2008) 25 January 2011 Integrity for Non-Aviation Users: Moving Away from "Specific Risk" 24

An “Average Risk” Approach to SBAS (and GBAS) – word version • Data imply an “average risk” equivalent VPL for WAAS ~ 4 – 5 times lower than current value. • Re-assess “rare-normal” and faulted error models and data to build a “certifiable” safety case. – Multiple rare-normal (“fault-free”) models built from existing data to incorporate remaining uncertainty – All fault-mode analyses follow the same approach: • Estimate prior fault probabilities and probability uncertainties. • Simulate all significant variations of each fault type rather than “worst case” focus convolve with prior dist. to estimate risk. – Faults whose impact is driven by worst-case scenarios (ionosphere, signal deformation) will become less important. – Multiple-fault scenarios neglected as too improbable may become more important, as probabilistic weighting of risk may show that fault-combination cases are non-negligible. 25 January 2011 Integrity for Non-Aviation Users: Moving Away from "Specific Risk" 25

A Combined “Average/Specific” Risk Approach (1) Derived from FAA “Hazard Risk Model” (1) and Simplified Aircraft Accident Risk Breakdown (2) Major Hazardous Catastrophic (Slight risk of aircraft loss/pilot challenged) (Risk of a/c loss; Severe loss of safety margin) (Likely a/c hull loss) 10 -5 ~ 1% 10 -7 10 -6 10 -9 10 -7 ~ 10% 10 -9 ~ 1% (~ 100 systems) Overall a/c loss prob. Loss prob. due to equipment failure Loss prob. due to GNSS nav. failure (1) FAA System Safety Handbook, 2008. http: //www. faa. gov/library/manuals/aviation/risk_management/ss_handbook/ (2) R. Kelly and J. Davis, “Required Navigation Performance (RNP), ” Navigation, Spring 1994. 25 January 2011 Integrity for Non-Aviation Users: Moving Away from "Specific Risk" 26