AI Planning Scheduling and Data Analytics for a

AI Planning & Scheduling and Data Analytics for a new generation of Mission Planning tools / 52 S. Fratini & N. Policella

Software Engineering Software Products Consulting Services Solenix: Our Services 2 / 52 § § § Spacecraft Operations Ground Segment Engineering Management Support Research of Advanced Technologies and Concepts Data Processing § § Support to management and operations of complex systems Focus on usability and ergonomics High performance Integration capabilities § § Client solutions: mobile and web clients Distributed systems Data-driven systems: data management & analysis Complex graphical data visualisation 24/11/2017 Consulting Services Software Products 1 st International Round Table on Intelligent Control for Space Missions Software Engineering

Intelligent Control Plan Schedule Monitor 3 / 52 24/11/2017 Execute 1 st International Round Table on Intelligent Control for Space Missions

Intelligent Control Plan Schedule Monitor 4 / 52 24/11/2017 Execute 1 st International Round Table on Intelligent Control for Space Missions

Automated Planning and Scheduling? Wikipedia says… § Automated planning and scheduling, sometimes denoted as simply AI Planning, is a branch of artificial intelligence that concerns the realization of strategies or action sequences, typically for execution by intelligent agents, autonomous robots and unmanned vehicles. § In AI Planning, planners typically input a domain model (a description of a set of possible actions which model the domain) as well as the specific problem to be solved. Such planners are called "Domain Independent" to emphasis the fact that they can solve planning problems from a wide range of domains. • A schedule … consists of a list of times at which possible tasks, events, or actions are intended to take place, or of a sequence of events in the chronological order in which such things are intended to take place. The process of creating a schedule - deciding how to order these tasks and how to commit resources between the variety of possible tasks - is called scheduling. “I keep six honest serving men (they taught me all I knew); their names are What and Why and When and How and Where and Who. ” --- Rudyard Kipling 5 / 52 24/11/2017 1 st International Round Table on Intelligent Control for Space Missions

Planning and Control 6 / 52 24/11/2017 1 st International Round Table on Intelligent Control for Space Missions

Scheduling vs. Planning in Operations Space operations vs. A. I. : different meanings Planning Scheduling A. I. Time Space Operations 7 / 52 24/11/2017 FACD 0013|0|5|2|1|A|0| FACD 0014|0|5|2|1|A|0| FACD 0015|0|5|2|1|A|0| FACD 0016|0|5|2|1|A|0| FACD 0017|0|5|2|1|B|0| FACD 0018|0|5|2|1|B|0| 1 st International Round Table on Intelligent Control for Space Missions

Realistic Planning Problems 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 8 / 52 Time Numerical Reasoning Concurrent actions Context–Dependent Effects Interaction with Users, “man in the loop” Oversubscribed Problems Focus on Optimization rather than on Achievement Robustness, Flexibility Execution Monitoring Re-Planning & Re-Scheduling Scalability Integration of external solving processes Need for Formal Verification and Validation of Plans and Schedules 24/11/2017 1 st International Round Table on Intelligent Control for Space Missions

Knowledge in Automated Planning and Scheduling 1. 2. 3. 4. 5. Knowledge about the domain Ø Timing, causal relationships, resources Knowledge about “good plans” Ø Oversubscription, Optimization, Robustness, Flexibility Explicit search-control knowledge Ø Domain specific know-how, Optimization Knowledge about the user and about user’s preferences Ø “man in the loop” Knowledge about plan repair during execution Ø Execution monitoring, re-planning and re-scheduling Explanation 9 / 52 24/11/2017 1 st International Round Table on Intelligent Control for Space Missions

Integrating Planning and Scheduling It does not matter how many resources you have… …if you don’t know how to use them! 10 / 52 24/11/2017 1 st International Round Table on Intelligent Control for Space Missions

Integrating Planning and Scheduling “Planning & scheduling are rarely separable” 11 / 52 24/11/2017 1 st International Round Table on Intelligent Control for Space Missions

AI Planning in Space D/L (Down. Link) Mission Planning TM (Tele. Metry) nt me TC (Tele. Commands) Mission Planning Team (ESA-ESOC, Germany) Seg L /D UL n Pla S G U/L (Up. Link) Ground Station 12 / 52 Planning ent MPT n Pla ard Ground Station Management Plan Lander Robotics S/C Autonomy On Bo Science Planning Team (ESA-ESAC, Spain) nce und Science Planning Scie Gro SPT Planning Requests Payload Experts PI (Principal Investigators) Seg m S/C Attitude, Modes FCT (Flight Control Team) S/C

Timelines Timeline (n): a graphical representation of a period of time, on which important events are marked. - Oxford English Dictionary A timeline is a way of displaying a list of events in chronological order, sometimes described as a project artifact. It is typically a graphic design showing a long bar labeled with dates alongside itself and (usually) events labeled on points where they would have happened. - Wikipedia 13 / 52 24/11/2017 1 st International Round Table on Intelligent Control for Space Missions

Timeline Based Approach • Modelling – – Modeling – Solving • Focus on key systems Describe their possible consistent temporal behaviours Represent the relevant constraints (domain theory) Solving – Synthesize timelines according to current goals satisfying modelled constraints Problem Solving = Timeline synthesis 14 / 52 24/11/2017 1 st International Round Table on Intelligent Control for Space Missions

Timeline-Based Planning & Scheduling Off() On() max On() Symbolic On t Burned O [lb, ub] On() 24/11/2017 Off() On() t 2 – Describe Temporal Constraints Features 3 – Synchronize Features 15 / 52 Value Constraints Off Transitions Numeric 1 - Represent Features Pulses O 1 st International Round Table on Intelligent Control for Space Missions

Architectures 16 / 52 24/11/2017 1 st International Round Table on Intelligent Control for Space Missions

Timelines in ESA 17 / 52 24/11/2017 1 st International Round Table on Intelligent Control for Space Missions

The ESA APSI Framework 18 / 52 24/11/2017 1 st International Round Table on Intelligent Control for Space Missions

Alphasat TECO system 19 / 52 24/11/2017 1 st International Round Table on Intelligent Control for Space Missions

Alphasat TECO system § The Alphasat spacecraft carries an Inmarsat communications payload and four Technology Demonstration Payloads (TDPs), provided under ESA responsibility § ESA is in charge of coordinating the use of the different TDPs through the TDP ESA Coordination Office (TECO) § TECO’s main objective is to manage the selection of TDP science programs, planning, and data archiving for Alphasat TDP Operations Centres Operations Centres 20 / 52 24/11/2017 Consolidated plan Individual requests Plan and execution feedback Inmarsat ESA TECO Execution feedback 1 st International Round Table on Intelligent Control for Space Missions

Alphasat TECO system § Weekly planning of the execution of the activity requests from the different TDP OCs on the shared Inmarsat platform • Temporal constraints (e. g. do not perform science during chemical manoeuvres) • Resource constraints (e. g. limited power and bandwidth) § The goal is to deliver an automated planning system requiring as little operator involvement as possible: • Nominal robust plan generation and validation is performed automatically by the software system • Operator involvement is required only in case of anomaly §Challenge: • Automatic explanation approach (as the decisions taken are taken by an automatic system) • No black-box • Build trust with the final users 21 / 52 24/11/2017 1 st International Round Table on Intelligent Control for Space Missions

In-flight experience 22 / 52 § TECO systems has proved to be • reliable, • robust, • flexible § TECO System grants • to Inmarsat (mission operator and satellite owner) a safe and transparent operations, and • to the hosted payloads the maximum experimental return § Nominal operations phase since January 1 st, 2014 • More than 35 K activities for the TDPs have been scheduled. • All the abortions were fully explained and understood, with none of them due to the activities planning (i. e. , TECO System). 24/11/2017 1 st International Round Table on Intelligent Control for Space Missions

TIAGO – Tool for Intelligent Allocation of Ground Operations for Cluster-II 23 / 52 24/11/2017 1 st International Round Table on Intelligent Control for Space Missions

Cluster-II Mission Overview The Cluster-II mission is part of ESA’s Horizon 2000 missions programme from 1985 investigating the Earth’s magnetic field with 10 European and 1 American scientific instrument. • 4 identical satellites with magnetic cleanliness • Launch in 2000 (first launch in 1996 failed) • Extended until 2018 • Apogee: ~110. 000 km • Perigee: ~20. 000 km • Orbital period: 54. 3 h 24 / 52 24/11/2017 1 st International Round Table on Intelligent Control for Space Missions

Problem Overview Allocate ground station passes to… • download all scientific data recorded from instruments • avoid overflow of on-board memory • keep on-board memory level < 80% (robustness) Constraints: • Ground station availability & booking procedures • Link Budget: changing downlink bitrates (due to highly elliptic orbit) • Science Modes: Intense commanding at the beginning/ending of the pass • Operational Constraints (duration, separation, frequency) 25 / 52 24/11/2017 1 st International Round Table on Intelligent Control for Space Missions

Original Approach Planning activities can take up to 1. 5 man-days per week 26 / 52 24/11/2017 1 st International Round Table on Intelligent Control for Space Missions

New Workflow 27 / 52 24/11/2017 1 st International Round Table on Intelligent Control for Space Missions

An Integrated P&S Problem Planning/Scheduling 28 / 52 24/11/2017 1 st International Round Table on Intelligent Control for Space Missions

Tiago MMI 29 / 52 24/11/2017 1 st International Round Table on Intelligent Control for Space Missions

Autonomy & Robotics 30 / 52 24/11/2017 1 st International Round Table on Intelligent Control for Space Missions

Intelligent Control Plan Schedule Monitor 31 / 52 24/11/2017 Execute 1 st International Round Table on Intelligent Control for Space Missions

Motivation for Autonomy • Upcoming missions will require a higher degree of remote operations to increase quality and quantity of science return • Remote operations are a challenging scenario, mainly because communication delays and errors • Autonomy can entail opportunistic science, provide contingency recovery procedures and allow fast reaction to tracked events • AI planning-based control layers have demonstrated to be able to entail autonomy, but modeling still constitute a bottleneck for the use of these technologies 32 / 52 24/11/2017 1 st International Round Table on Intelligent Control for Space Missions

OPS-OSA Activities – Autonomy & Robotics Ø APSI - Advanced Planning & Scheduling Initiative (ESOC GSP Study, 2007 -2008) Ø GOAC - Goal Oriented Autonomous Controller (ESTEC TRP Study, 2009 -2011) Ø IRONCAP (2011 -2012) Ø Application on Tele. Robotics (2013) Ø On-Board Autonomy for Net. Sat (2016 -) 33 / 52 24/11/2017 1 st International Round Table on Intelligent Control for Space Missions

The GOAC Approach Ø A goal-oriented system Ø Interleaved sense-plan-act cycles (reactors) Ø Reactors embed a domain independent, model driven, timeline-based planning technology Ø Well defined interaction protocol: Ø Goals (up to down) Ø Observations (down to up) Ø Increased depth allows HTN and planning at different temporal scopes Ø Increased width allows planning with different functional scopes 34 / 52 24/11/2017 1 st International Round Table on Intelligent Control for Space Missions

GOAC Achievements Ø Support for “Long”, “Medium” and “Short” Term planning: Ø Planning for mission goals, complex planning problems, long horizon Ø Planning for platform management goals, medium term horizon Ø Task sequencing, fast planning, very short horizon Ø Interleaved Planning and Execution Ø Plan flexibility: plans are flexible enough to allow a smooth execution Ø Planning and Re-Planning 35 / 52 24/11/2017 1 st International Round Table on Intelligent Control for Space Missions

IRONCAP • Goal: Preparing ESA for future robotics missions operations through the Investigation and Prototyping of Innovative Planning Operations Concepts for Rovers equipped with Autonomy Capabilities • Developing an operational concept for autonomous Rovers and define the processes and tools required for Rover ground control. • Developing a prototype of a Rover planning and scheduling facility supporting the operational concept • Demonstrating and evaluating the prototype in the context of two case studies 36 / 52 24/11/2017 1 st International Round Table on Intelligent Control for Space Missions

Application on Telerobotics The MOCUP rover with LEGO NXT 2. 0 Mindstorms kit. Deliberative Layer - APSI Executive Agent - GOAC Functional Layer – METERON Infrastructure Tele Commands 37 / 52 24/11/2017 1 st International Round Table on Intelligent Control for Space Missions Telemetry

Monitoring, Novelty Detection & Diagnostics, Dr. MUST Courtesy of Jose Martinez Heras – Black Hat S. L. 38 / 52 24/11/2017 1 st International Round Table on Intelligent Control for Space Missions

Intelligent Control Plan Schedule Monitor 39 / 52 24/11/2017 Execute 1 st International Round Table on Intelligent Control for Space Missions

From Monitoring… • Is everything fine? • Classic: Out-Of-Limits (OOL) Hard Limit Soft Limit Hard Limit 40 / 52 24/11/2017 1 st International Round Table on Intelligent Control for Space Missions

…to Anomaly Detection • • Is everything fine? Anticipate Anomalies: Anomaly Detection 20, 000 – 40, 000 TM parameters Unusual behaviour Potential anomaly OOL 41 / 52 24/11/2017 1 st International Round Table on Intelligent Control for Space Missions

Diagnostics Anomaly: deviation from expected behaviour. • Recognized by particular behaviour of one or more parameters over a time period Key questions: – Does this anomaly already occurred in the past and went Dr. MUST unnoticed? When? Pattern Matching – What are the effects on this anomaly? Correlator – What are its causes? – Can we minimize its effects? – Can we prevent this anomaly from happening again? 42 / 52 24/11/2017 1 st International Round Table on Intelligent Control for Space Missions

Anomaly investigation approach Very labour intensive Traditional Traditiona (weeks) Dr. MUST Automatic Runs unattended (minutes / hours) 43 / 52 24/11/2017 1 st International Round Table on Intelligent Control for Space Missions

Operational Assessment • Allows “ Googling” through spacecraft data – Searching for similar occurrences – Or correlated occurrences • 44 / 52 Impressive performance – Queries run very quickly compared to manual searches 24/11/2017 1 st International Round Table on Intelligent Control for Space Missions

Thermal Power Consumption Prediction + Produced Power - Platform Power - Thermal Power Science Power 45 / 52

Data Mining Competition http: //kelvins. esa. int 46 / 52

Amperes Data Mining Competition Real Telemetry Prediction Time 47 / 52 24/11/2017 1 st International Round Table on Intelligent Control for Space Missions

Conclusions 48 / 52 24/11/2017 1 st International Round Table on Intelligent Control for Space Missions

Closed Loop for Intelligent Control Plan Schedule Anomaly Detection Dr. MUST Prediction 49 / 52 24/11/2017 Monitor APSI TIAGO Temporal Planning Resource Allocation and Optimization GS Mission Planning Support Execute 1 st International Round Table on Intelligent Control for Space Missions TECO Timeline-Based Execution Flexibility and Controllability Support for Autonomy

Lessons Learned 50 / 52 § Declarative Model-Based Engine: • Support cost reduction, foster reusability • Ease interoperability and standards design • Verification & Validation • Support for what-If Analysis § Timeline-Based AI Planning: • Model cognitively close to the users’ practice • Algorithm synthesis oriented to the explanability of the solution § Relevance of user interaction services: • Support for a mixed-initiative approach to the solving process • Man-in-the-loop 24/11/2017 1 st International Round Table on Intelligent Control for Space Missions

To Conclude… Move the perspective from Procedural Based Approach To Model Based, Goal-Oriented Reasoning & Monitoring 51 / 52 24/11/2017 1 st International Round Table on Intelligent Control for Space Missions

Thank You Solenix Deutschland Gmb. H Spreestr. 3 64295 Darmstadt 52 / 52 Germany simone. fratini@solenix. ch nicola. policella@solenix. ch www. solenix. de