Simulating Pilots Decision Making by an Influence Diagram

Simulating Pilot’s Decision Making by an Influence Diagram Game Kai Virtanen, Tuomas Raivio and Raimo P. Hämäläinen Systems Analysis Laboratory Helsinki University of Technology S ystems Analysis Laboratory Helsinki University of Technology 1

Outline • • Air combat simulation models Existing modeling approaches Influence diagram (ID) Control decisions in one-on-one air combat ID for a control decision ID game for a control decision Simulation example Conclusions S ystems Analysis Laboratory Helsinki University of Technology 2

Air combat simulation models • Analysis of air combat and pilot training are expensive tasks • Every air combat situation cannot be analyzed in practice • Real time piloted: – Training in a realistic environment • Batch: – Controlled and repeatable environment – Discrete-event approaches Computer generated forces need a model that imitates pilot decision making S ystems Analysis Laboratory Helsinki University of Technology s r e d Or Co ds n a mm 3

Existing modeling approaches • Dynamic optimization and game theory: – Optimal flight paths – Simple performance criteria – Lack of realistic uncertainty models – Non-real-time computation • Models emulating the decision making of a pilot: – Computational techniques of AI: Rule-based, Value-Driven – Capture the preferences of a pilot – Real-time computation – Short planning horizon => Not optimal but myopic control commands How to handle uncertainties? Behavior of the opponent? S ystems Analysis Laboratory Helsinki University of Technology 4

Influence diagram (ID) (Howard et al. 1984) • Directed acyclic graphs • Describes the major factors of a decision problem • Widely used in decision analysis application areas Time precedence Probabilistic or functional dependence Informational arc Decision Alternatives available to DM Conditional arc Chance Random variables Conditional arc Deterministic Conditional arc Utility Deterministic variables A utility function S ystems Analysis Laboratory Helsinki University of Technology 5

Influence diagram (continued) • State of the world is described by attributes • States are associated with – Utility – Probability • • Utility is a commensurable measure for goodness of attributes Results include probability distributions over utility Decisions based on utility distributions Information gathering and updating using Bayesian reasoning S ystems Analysis Laboratory Helsinki University of Technology 6

Control decisions in one-on-one air combat Decision maker (DM) t=0 t=Dt ¼ Find the best maneuvering sequence for the DM with respect to the goals 1. Avoid being captured by the AD 2. Capture the AD by taking into account - Preferences of a pilot - Uncertainties - Dynamic decision environment - Behavior of the AD ¼ t=Dt t=0 Adversary (AD) Influence diagrams representing the control decision of the DM: • Single stage ID (Virtanen et al. 1999), – pilot’s short-term decision making • Multistage ID (Virtanen et al. 2001), − preference optimal flight paths against given trajectories New model: Influence diagram game S ystems Analysis Laboratory Helsinki University of Technology 7

ID for a control decision Adversary's Present State Present Combat State Present State Adversary's Maneuver Adversary’s State Present Measurement Combat State Measurement Maneuver State Situation Evaluation Present Threat Situation Assessment • Evolution of the players’ states described by a set of differential equations • The behavior of the AD? S ystems Analysis Laboratory Helsinki University of Technology 8

ID game for a control decision DM’s belief about AD’s viewpoint The game: - Non-zero-sum - Payoff = Expected utility Solution: Combat state DM's viewpoint - Discrete controls => Matrix game - Continuous controls => Nonlinear programming - Nash or Stackelberg equilibrium The best control of the DM against the worst possible action of the AD S ystems Analysis Laboratory Helsinki University of Technology 9

Simulation example • Initial state advantageous for AD • DM’s aircraft more agile • Solution generated with the ID game • DM wins altitude, km DM AD X-range, km y-range, km S ystems Analysis Laboratory Helsinki University of Technology 10

Conclusions • The influence diagram game: – Models preferences under uncertainty and multiple competing objectives in one-on-one air combat – Takes into account • Rational behavior of the adversary • Dynamics of flight • Utilization: – Air combat simulators, a good computer guided aircraft – Contributions to the existing air combat game formulations • Several computational difficulties are avoided • Roles of the players are varied dynamically • Producing reprisal strategies – Other simulation applications S ystems Analysis Laboratory Helsinki University of Technology 11