Ethical Mission Definition and Execution for Unmanned Systems

Ethical Mission Definition and Execution for Unmanned Systems: a Practical Approach Duane Davis, Don Brutzman, Bob Mc. Ghee, Curt Blais Computer Science Department Modeling, Simulation, and Virtual Environments (MOVES) Institute Naval Postgraduate School (NPS), Monterey California USA

Autonomy Schools of Thought 1. Military use of autonomy is inherently unethical Robert Sparrow, International Committee for Robot Arms Control 2. It is inherently unethical to NOT use autonomy if it is shown to perform better than humans Ronal Arkin, Regents’ Professor, Georgia Tech 3. Life-and-death decisions must be made by humans, even if autonomous systems could do it better General Paul Selva, Vice Chairman, US Joint Chiefs of Staff None of these provides a realistic solution or way forward

Broad Observations about Military Employment of Autonomy Military ethics is not about religion or morality • Law of Armed Conflict (LOAC) & International law • Rules of Engagement (ROE) • Guidance and Doctrine (CDR Intent, orders, TTPs, etc. ) Autonomous systems are just that—systems! • Employment in accordance with same ethical standard • Robots, like other systems, require trust within their intended operational regime Responsibility resides with the commander • Compliance (orders, doctrine, ROE, etc. ) • Employment of subordinate units and systems • Attainment of mission objectives

Foundations for Ethical Autonomy • Robots perform exactly as programmed—they don’t actually “decide” to do anything • Robots are inanimate objects—they cannot assume responsibility for the consequences of their actions • Responsibility (and liability) must reside with specific, identifiable humans—operators, programmers, manufacturers, etc. • Assumption of responsibility requires a level of predictability—possible outcomes must be reasonably foreseeable

Practical Requirements for Ethical Autonomy 1. Semantic Correctness • Rigorous mission definition fully understood by the CDR • Assurance (mathematical soundness) of mission flow between atomic components 2. Consistent Clarity • No potential for changes prior to execution • I. e. CDR cannot approve a preliminary form for “translation” 3. Executable Composition • Mission composed exclusively of target-vehicle-specific behaviors • Only “trusted” behaviors (certification at the behavior level)

Mission Execution Automaton (MEA) Generalization of a Universal Turing Machine (UTM) • Finite State Machine (FSM) encoding of an arbitrary autonomous vehicle mission • Mission Execution Engine (MEA) to transition over the FSM • External agent (robot) capable of executing behaviors corresponding to individual states of the mission-level FSM Strengths • Missions (FSM) easily read and understood by operators • Determinism allows exhaustive mission flow testing • Potential for “executable” specifications (no coding required!) Upshot (or why any of this matters from an ethical standpoint) • Behaviors can be “certified” individually to provide trust • Mathematical rigor provides operators the ability to fully understand what the vehicle is going to do over the course of the mission

An Exemplar Mission-Level FSM

Addition of Ethical Constraints

Mission Execution Engine Logic Start Success, Failure, Pending Constraint Breach mission logic for each task Enter Constraint Follow-on State Yes Fa il Success Follow-on Constraint Follow-on o No Stop N s d No ee s Fail Followon Ye cc Su Evaluate Response Ye Execution continues until a “Stop” condition is encountered Enter Success Follow-on State Issue Behavior Order Constraint • Might consider a pending ethical breach a “fail” • But this is NOT an MEE requirement Enter FSM Start State Enter Fail Follow-on State

Upshot of MEA Utilization Exclusive use of trusted behaviors and constraints • Behavior: a trusted “atomic” capability of the specific target vehicle • Constraint: a pending condition that the vehicle is trusted to identify • Micro-level assurance that the individual states will be executed according to the applicable guidance Exhaustive testability of the high-level mission • Mission definition abstracted from behavior specifics • Exhaustive testing of possible state sequences (simulation, manual walkthrough, rehearsal, etc. ) • Macro-level assurance that the mission will proceed through the required behaviors (states) as desired

Making the Process “Sailor Proof” Providing assurance that the approved MEA is correct • Are FSM states actually trusted behaviors? • Are states actually executable on the target vehicle? • Is the mission constructed correctly (e. g. , no loops, missing transitions, unreachable states) Description Logics (DL): a mathematically rigorous mechanism for definition of what a valid mission “is” • Define “concepts” and “roles” that can be applied to mission elements • Define rules for concepts and roles to ensure correct mission construction • Applied to vehicle-specific executable orders

A Mission Execution Ontology (MEO)

Summary In a military context, “ethics” equates to an operational commander taking responsibility • Requires formal understanding of the micro- and macrolevel aspects of autonomous vehicle tasking MEA provides assurance that tasking aligns with commander’s intent • High-level mission flow is correct • Individual behaviors and constraints are trusted in the context of mission requirements MEO enforces mission correctness • Correct mission construction • Executability on the intended target vehicle

Questions?