Exploration of Hybrid and Intelligent Control Architectures in

Exploration of Hybrid and Intelligent Control Architectures in Conjunction with Probabilistic Verification (ONR N 00014 -97 -1 -0946) Overview of ONR UCAV Project S. Shankar Sastry ONR Project Review, July 21, 1998 Electronics Research Laboratory University of California, Berkeley ONR N 00014 -97 -1 -0946 1

Research Objective and Research Thrusts Research Objective: Design and evaluation of intelligent control architectures for UCAV’s Research Thrusts: · Intelligent control architectures for coordinating UCAV’s · Verification and design tools for intelligent control architectures · Perception and action hierarchies for vision-based control and navigation ONR N 00014 -97 -1 -0946 2

Project Concept · Central control paradigm (optimization of system-wide mission objectives) breaks down when dealing with distributed multiagent systems · Real-world environments are complex, spatially extended, dynamic, stochastic and largely unknown · Autonomous intelligent systems in real-world environments require – sensory and control functions • based on system decomposition based on hierarchical hybrid, and multiagent designs using multiple levels of abstraction – structural and parametric learning methods • to adapt to initially unknown environments – generalized estimation methods, uncertainty management and robust control techniques • to deal with residual uncertainty in stochastic, partially observable environments ONR N 00014 -97 -1 -0946 3

Project Concept · Real-time decision making is achieved by – parallelism – reflexive control – compilation – anytime approximation · Hierarchical perception and control paradigm – architectural fusion of the central control paradigm with autonomous intelligent systems · Distributed intelligent systems – hierarchical or modular to control complexity – globally organized emergent behavior – robust, adaptive and fault tolerant and degraded modes of operation – architectural organization involving the use of compositionality ONR N 00014 -97 -1 -0946 4

Technology Drivers · Intelligent Multi-Agent Systems The need for a theoretical framework for an integrative approach arises from advances in computation, communication, intelligent materials, visualization and other technologies which make it possible to expect more from a multi-agent system, than a centralized control framework. ONR N 00014 -97 -1 -0946 5

Intelligent Multi-Agent Systems · Unmmaned Autonomous Vehicles · Distributed Command Control · Simulated Battlefield Environment · Decision Support Aids for Human Centered Systems · Automatic Target Recognition · Robust and Fault tolerant Systems · Distributed Communication Systems · Distributed Power systems · Intelligent Vehicle Highway systems · Air Traffic Management Systems · Intelligent Telemedical Systems ONR N 00014 -97 -1 -0946 6

Berkeley Team · SASTRY: Specializes in decentralized control of distributed systems and hybrid design and verification techniques, with applications to automated highway systems, air traffic management systems and robotics. · MALIK: A leader in low-level and intermediate vision, with recent work in crucial aspects of image segmentation, association, grouping and attribute evaluation. · SENGUPTA: Experience in observation and control for distributed systems. Extensive background in discrete event and hybrid systems. Application to transportation and communication problems. · GODBOLE: Hybrid control of multi-agent systems. Extensive background in applications to automated highway and air traffic management systems. Inter-agent coordination problems. Design of fault management systems. · LYGEROS. Hybrid control synthesis. Background in automated highway and air traffic management systems. Formal methods for verification of large-scale systems. Fault tolerant control systems. · SHAKERNIA. First-year graduate student in EECS. B. S. (1997) EECS, UCB. ONR N 00014 -97 -1 -0946 7

Theoretical Underpinnings · Architectural design for multi-agent systems – centralization for optimality – decentralization for safety, reliability and speed of response · Perception systems sharing many representations – hierarchical aggregation – wide-area surveillance – low-level perception · · · Frameworks for representing and reasoning with uncertainty Incorporation of learning, adaptation and fault toperance: parametric uncertainty with update and adaptation at the continuous levels, learning of new “logical entities”, reinforcement learning at the logical levels and meta-learning for redefining architecture Soft-computing approaches to intelligence augmentation for humancentered systems: reconciliation of human decision making schemes with machine performance, intelligent agents, keeping the human in the loop, sufficing rather than optimizing ONR N 00014 -97 -1 -0946 8

Research Thrust 1: Intelligent Control Architectures · An architecture design problem is concerned with design of both the observation and control · An architecture design problem for a distributed system begins with specified safety and efficiency objectives and aims to characterize communication, observation and control · Our investigation of the intelligent control architecture design problem is concerned with three formalisms – intrinsic model – supervisory control of discrete event systems – hybrid system formalism ONR N 00014 -97 -1 -0946 9

Research Thrust 2: Verification and Design Tools · Design Mode Verification · Faulted Mode Verification · Probabilistic Verification – A rapproachment between stochastic control, Bayesian decision networks and soft computing – The heart of the approach is to not verify that every run of the hybrid system satisfies certain safety or liveness parameters, rather to check that the properties are satisfied with a certain probabillity, given uncertainties of actuation and sensing ONR N 00014 -97 -1 -0946 10

Research Thrust 3: Perception and Action Hierarchies · Hierarchical Vision · Control Around the Vision Sensor · Surveillance We are designing a perception and action hierarchy centered around the vision sensor to support the observation and control functions of air vehicles ONR N 00014 -97 -1 -0946 11

Exploration of Hybrid and Intelligent Control Architectures in Conjunction with Probabilistic Verification (ONR N 00014 -97 -1 -0946) Architectures for UCAV and Results on Multi-Agent Coordination Raja Sengupta and Datta Godbole ONR Project Review, February 24 -25, 1998 Electronics Research Laboratory University of California, Berkeley ONR N 00014 -97 -1 -0946 12

Outline · Research Thrusts – Hierarchical UCAV Architecture Design – Design of Decentralized Observation, Communication and Control for Discrete Event Systems • Application to Fault Detection and Identification – Distributed hybrid control for multi-agent systems ONR N 00014 -97 -1 -0946 13

UCAV Architecture · A group of UCAV’s are a coordinating multi-agent system with limited centralized control – mobile communication systems support coordination – due to limitations on computing and communication resources the UCAV’s must cooperatively and individually exhibit a high degree of autonomy – low bandwidth, asynchronous, event-driven coordination is preferred to synchronous, time-driven coordination · A single UCAV is a real-time embedded system · The architecture should specify observation and control semantics at each layer – the layers should be consistent and programmable ONR N 00014 -97 -1 -0946 14

UCAV Architecture Strategic Objective Mission Control Ship UCAV Strategic Layer Inter-UCAV Coordination Trajectory Constraints Tactical Layer Sensor Info on Targets, UCAV’s Trajectory Regulation Layer Environmental Sensors Actuator Commands UCAV Dynamics ONR N 00014 -97 -1 -0946 15

Preliminary UCAV Architecture · Regulation – control of UCAV actuators • fully autonomous · Tactical – safe and efficient trajectory generation and mode control • fully autonomous · Strategic – trajectory constraints, UCAV to UCAV/ship coordination, weapons configuration, fault management · Central Mission Control – mission planning, resource allocation, mission emergency response, mission optimization, pilot interface ONR N 00014 -97 -1 -0946 16

PATH AHS Architecture Network Layer Link Layer Regulator (flow control) SC&P* Info Coordination Layer (maneuver protocols) SC&P* Info Regulation Controller Throttle Actuator Brake Actuator Roadside Vehicle Communications with neighbors Physical Layer Steering Actuator Vehicle Dynamics (Plant) *Sensory, Capability & Performance ONR N 00014 -97 -1 -0946 17

Example: AHS Architecture · The layers are consistent and programmable · Coordination – Synchronous, time-driven for platoon stability (regulation) – Asynchronous, event-driven for maneuvers (coordination) ONR N 00014 -97 -1 -0946 18

Decentralized Observation, Communication, Control for Multi-agent Systems · Given a strategic objective and local observation what is the information exchange with the mothership and other UCAV’s required to command tactical control ? – Given a distributed control problem and the local observation at each site, what is the inter-site communication (minimal) or coordination protocols required to solve this problem ? · Given a system mission what is the strategic objective (possibly dynamic) of each UCAV ? – How to distribute among the available agents a specified centralized control problem ? ONR N 00014 -97 -1 -0946 19

Literature: Distributed Control with Decentralized Information · Decentralized control of large-scale systems – linear systems, time-driven, design for stability · Stochastic scheduling – queuing networks, time-driven, design for performance · Distributed control of discrete event systems – event driven, design for correctness, safety · Distributed control of hybrid systems – time and event-driven, design for correctness, safety ONR N 00014 -97 -1 -0946 20

Communication and Control Synthesis for DES · Symbolic representation of system actions (events) · Behavior is a causal ordering of symbols (event trace) · Objective: – Given a DES plant model, specification of the control objective, local observation and control capability, synthesize a minimal inter-agent communication and the control law of each agent. ONR N 00014 -97 -1 -0946 21

Communication and Control Synthesis for DES · Advantages: – Will synthesize symbolic, event-driven, inter-agent communication over a finite message set – Very simple models permitting logical or combinatorial analysis and insights – AHS Example: Worked for most coordinating maneuvers other than stability properties for vehicle following. · Limitation: No formal way to capture continuous dynamics – The semantics of an event is generally some alignment or safety conditions in velocity, position, and euler angles with respect to targets or other agents – Distributed control of hybrid systems ONR N 00014 -97 -1 -0946 22

DES Problem 1: Observation and Communication Agent Communication Channels A 1 A 2 A 3 Plant (Lp) • The agents have partial observation, but can exchange me • The plant has a set of distinguished events (failures) • OBJECTIVE: Design the inter-agent communication schem to detect and isolate the distinguished events ONR N 00014 -97 -1 -0946 23

DES Problem 1: Observation and Communication Only Theorem 1 (Lp, áåpoi, åfiñiÎI) is decentrally diagnosable if there exists n Î N such that for all sf Î åf, usfv Î Lp, |v| £ n, implies (w Î Lp) Ù ( i, Påpoi (w) = Påpoi (usfv)) Þ (sf Î w). If any two sufficiently long plant traces look the same to all the diagnosers, then either they have no failures or have all the same failures. Synthesis: The communicate all plant observations solution works. General drawback: Redundant information is communicated. -L(f) may not be regular even though Lp is regular. Current focus: Minimal communication, protocol synthesis, trace abstraction ONR N 00014 -97 -1 -0946 24

DES Problem 2: Control, Observation, and Communication · Each agent has a set of controllable events · The controllable events are a subset of the set of observable events · The next event is either an uncontrollable event from the plant, a controllable event enabled by an agent, or a message event scheduled by an agent · Control objective is specified by a language · Researching the existence and synthesis of coordination protocols ONR N 00014 -97 -1 -0946 25

Distributed Control and Communication of Hybrid Systems · Symbolic and flow representation of system actions · Game/Team-theoretic approach to synthesis – Agents play as a team against a non-cooperative target • Characterize the saddle disturbance in the team-target game • Use the saddle disturbance to formulate and solve an optimal control problem to characterize the saddle team strategy • Derive the inter-agent communication and individual agent control from the necessary conditions · Theorem: If for an initial state the worst disturbance is independent of the team control then the target-team game has a saddle solution ONR N 00014 -97 -1 -0946 26

Hybrid Approach: Application to AHS Lane Change · FT generates disturbance in response to downstream traffic and P, RT play as a team · We guess the saddle disturbance · Use the saddle disturbance to formulate and solve an optimal control problem for the saddle team control · The inter-agent coordination requires three messages R T FT P ONR N 00014 -97 -1 -0946 27

Summary · Developed a preliminary UCAV architecture – UCAV to UCAV and UCAV to ship coordination – Hierarchical control and observation · Focused theoretical research efforts on – Sensing, Control, and Communication for Distributed Multi -agent Systems – Failure Detection and Identification – Design of Hybrid and Decentralized Control Systems – Current Results • Existence and synthesis of inter-agent communications for partially observed distributed discrete event systems • Synthesis of safe hybrid control laws for distributed hybrid systems using game theory and optimal control ONR N 00014 -97 -1 -0946 28

Quick Time Movie of Helicopter Flight ONR N 00014 -97 -1 -0946 29