AIEnabled Hypersonic Missions Dr Ali K Raz Dr

AI-Enabled Hypersonic Missions Dr. Ali K. Raz Dr. Kris Ezra School of Aeronautics and Astronautics Center for Integrated Systems in Aerospace (CISA) Purdue University April 19 th, 2019 Sandia A 4 H Field Day

Project Goals/Mission Statement Develop System-of-Systems Modeling and Simulation to provide AI-Enabled Hypersonic Mission Capability • We will provide: • An agent-based framework for simulating hypersonic missions with multiple distributed systems • i. e. , ability to simulate varying capabilities of blue & red assets • An assessment of current state-of-the-art in autonomy and its applicability to hypersonic missions • i. e. , what are the relevant ML techniques and how to tailor these techniques for hypersonic missions • An end-to-end simulation of hypersonic mission with integrated autonomy Timeline Preliminary Hypersonic So. S Simulation (Parametric Agents) Autonomy Conceptual Formulation Proof-of. Concept (e. g. , dog-fight) Increased Red and Blue Asset fidelity (5/6 Do. F model, So. Smetrics) Year 1 End-to-End Autonomy Implementation Year 2 Sandia A 4 H Field Day Autonomous Hypersonic Decision Making Fidelity Enhancement and Classified Processing Year 3

Hypersonic Mission Overview In-Flight Path Update Re-plan Mission Optimal Path 2 No fly zone 2 1 HGV In-Flight Path Update Re-plan Mission 3 1 Moving Target 1 Status-Quo: Optimal path to intercept target 2 Real-world scenario: In-Flight update to avoid adversary measures 3 End-Game: Ensure mission success with re-planning Sandia A 4 H Field Day

Hypersonic Mission Requirements • Use AI methods for autonomous planning of Hypersonic missions • Human decision making infeasible • AI ultimate goals is to intercept target • But depends on physics of hypersonic vehicle • Flexible and abstract simulation of Hypersonic missions • Physics based limitations of Hypersonic vehicle maneuverability • Availability of inflight information • Target models and adversary capabilities • Mission Trades: • Nature and timeliness of information availability • AI needs vs. minimum HGV capabilities • Metrics for end-game evaluation/training Sandia A 4 H Field Day Flight Path Velocities v. Altitude from 3 -Do. F Optimization

Context at Purdue: So. S Analysis Enabler: Flexible Abstract Modeling and Simulation DAF integrates Functional and Communication Modeling into one framework • Other models are often rigid (Analysis Model Y) or narrow and not geared to model multi-faceted So. S problem space (Comm Model X) • The best model for So. S analysis is not necessarily the one that fills the entire circle and may change with problem context Sandia A 4 H Field Day

Context at Purdue: Recent Developments: AI Enabled So. S Architecting • • Use Artificial Intelligence (AI) to discover relationships between features and metrics: better understanding of the reasons for “goodness” of So. S architectures Generate optimal architectures, with features that will result in desired metrics: better architecting Status quo Data sources (including MBSE) Ranked architectures A 1 Features of architectures A 2 So. S analysis AN Architectures Research Question How can we leverage AI Enabled So. S Architecting for Hypersonic Mission Analysis? 1) Discover relationships with AI techniques 2) Generate architectures with features that achieve desired metrics Sandia A 4 H Field Day A 1 A 2 A 4 A 7 . . . AN A 1 MM Metrics

Our approach to AI-Enabled Hypersonic Missions So. S Mission Model HGV Autonomy Path Planning Avoid Adv. counter measures Dogfight Agent Models: Hypersonic Vehicle Targets and Adversaries Information Availability AI Inner Loop AI Outer Loop Software and Tools: • Mission Modeling MATLAB • Hypersonic Trajectory Modeling Beluga • AI Implementation Python Sandia A 4 H Field Day Mission Success End-game metrics, maneuvering capability & outcome

Hypersonic Vehicle Model Abstraction (Notional) Inputs HGV Blue Force Sensor Data Outputs Data Interpretation Decision. Making Model (AI/ML) Onboard Sensor Data Equations of Motion Trajectory Planning Vehicle Parameters Physical Modifications Physical Model Selected trajectory and delta v loss estimations Vehicle State Control Capability Sandia A 4 H Field Day

AI Approaches Under Consideration • AI Outer loop • Reinforcement learning (RL) • Deep. Mind’s Alpha. Go • Two player game with perfect information • Deep. Mind’s Capture the Flag - Quake III Arena • Cooperative game • Two teams • Robust Adversarial RL • AI Inner loop • Genetic Fuzzy Tree (GFT) • Psibernetix’s ALPHA - Virtual Dogfight Sandia A 4 H Field Day

Deep. Mind’s Alpha. Go • Two Deep Neural Networks (DNNs) • Policy network • Outputs move probabilities • Initially trained by supervised learning on accurately predict human expert moves • Subsequently refined by policy-gradient reinforcement learning • Value network • Outputs a position evaluation Ref: Mastering the game of Go without human knowledge https: //www. nature. com/articles/nature 24270 Sandia A 4 H Field Day

ALPHA - Virtual Dogfight Training • Random versions of ALPHA fight against a version tuned with human input • Winning versions of AI are "bred" with each other • Best-performing traits carried on to next generation • AI fights against each other to simulate natural selection • Through subsequent generations of AI versions fighting against each other, only one remains • Hence name ALPHA Ref: Genetic Fuzzy based Artificial Intelligence for Unmanned Combat Aerial Vehicle Control in Simulated Air Combat Missions, Journal of Defense Management, DOI: 10. 4172/2167 -0374. 1000144 Sandia A 4 H Field Day

Near-Term: AI Questions • How to map the HGV mission space to an ‘agent state and action’ space for AI training? • Discretize HGV states and actions? • What does the decision tree look like for HGV states and actions? • What End-game metrics are needed for AI training and evaluation? Sandia A 4 H Field Day

Summary: Purdue Capabilities and Project Outcomes • Purdue CISA Capabilities: • Well-documented history of So. S developments in Do. D application areas • Discrete Agent Framework • Analytical tools and quantitative methods for So. S mission analysis • AI-enabled So. S architecting • Project Outcomes • End-to-End Hypersonic mission scenarios & simulation • Abstract/flexible models primed for fidelity enhancements • Autonomous hypersonic agent model and decision making • Tailored AI methods for defense applications • Interplay of So. S features and metrics with AI methods Timeline Preliminary Hypersonic So. S Simulation (Parametric Agents) % Comp Autonomy Conceptual Formulation Proof-of. Concept (e. g. , dog fight) Increased Red and Blue Asset fidelity (5/6 Do. F model, So. Smetrics) Year 1 End-to-End Autonomy Implementation Year 2 Sandia A 4 H Field Day Autonomous Hypersonic Decision Making Fidelity Enhancement and Classified Processing Year 3

Thank you & Questions Ali Raz Visiting Assistant Professor School of Aeronautics and Astronautics akraz@purdue. edu Sandia A 4 H Field Day

DNN Training in Alpha. G 0 • Trained from self-play by RL • In each position Monte Carlo Tree Search (MCTS) is executed • Guided by DNN • Outputs probabilities of each move • Typically stronger than the raw move probabilities of DNN Ref: Mastering the game of Go without human knowledge https: //www. nature. com/articles/nature 24270 Sandia A 4 H Field Day

DNN Training in Alpha. G 0 (contd. ) • Takes raw board position, as an input and passes through convolutional layers with some weights • Outputs • Probability distribution over moves • Probability p of current player winning in position s • Weights are updated to: • Maximize similarity of probability distribution and MCTS probabilities • Minimize error between predicted winner p and game winner z Ref: Mastering the game of Go without human knowledge https: //www. nature. com/articles/nature 24270 Sandia A 4 H Field Day