Big Data and Control Theory Anil Aswani Pat

Big Data and Control Theory Anil Aswani, Pat Bouffard, Young-Hwan Chang, Jeremy Gillula, Haomiao Huang, Soulaiman Itani, Mike Vitus, and Claire Tomlin February 23 2012 1

Control Theory 2

Control Theory 3

Control Theory 4

Control Theory 5

Control Theory 6

Control Theory 7

Control Theory Control inputs are based on the mathematical model 8

Hybrid Control Theory

Examples High performance flight Air traffic control Maneuvering through modes Grouping and conflict classification [Shaw] [Seamster] [Kahah] [Itani, Gray, Moasser] HER 2 inhibition in breast cancer Multiple equilibria

1. High Performance Flight 11

Reachable Sets Backwards Reachable Set unsafe In red, system may become unsafe In blue, system will stay safe On boundary, apply control to stay out of red

Example: Collision Avoidance Pilots instructed to attempt to collide vehicles

Example: Back-Flip Recovery Drift Impulse

Back-flip Recovery Drift Impulse

Back-Flip: Results

These methods assume a model…. • What if the model is not well known? – Dynamics not well characterized – Human input • Can the model be learned from data? 17

Learn models from data… • … but stay safe while learning • Safety: – A nominal model with error bounds – Reachable sets computed to ensure safety in worst case – Reachable sets computed using Model Predictive Control (MPC) • Performance: – Use online learning to update nominal model – Cost function used to generate control action within the safe set • Learning-based Model Predictive Control

Learning-based Model Predictive Control • Unknown system dynamics represented using an oracle • At each time step – Optimization solved, Oracle updated

Learning-based Model Predictive Control • Unknown system dynamics represented using an oracle • At each time step – Optimization solved, Oracle updated Performance LBMPC Safety

Example: Learning to fly • Linear model – Physics for structure – Experimental coefficients • Physics improve statistics – Fewer parameters – Less noise

Example: Learning to fly video

2. Air Traffic Control 23

Closely Spaced Parallels 750 ft separation San Francisco Airport

Keeping the humans in the loop NASA Ames The FAA predicts commercial operations to increase 2. 1% annually 1 “The FAA is trying to take controllers so far out-of-theloop… that they can't get back into the loop when the computer quits. ” Don Brown, former air traffic controller, Safety Rep for National Air Traffic Controllers Assoc. 2 Improving automation requires maintaining controller awareness, which requires models of controller cognition 1 FAA Forecast Fact Sheet – Fiscal Years 2011 -2031 2 Don Brown, “Can the FAA Get Rid of Air Traffic Controllers? ” The Atlantic Online, March 6, 2011

Initial Studies deviated aircraft intruder [Alex Bayen]

Cognitive Analysis Grouping, conflict classification, and maneuvers Qualitative Models Quantitative Models ? ? Air traffic controller cognitive strategies are known, but it’s very difficult to get parameters for quantitative models. Seamster, T. , Redding, R. , Cannon, J. , Ryder, J. , and Purcell, J. , “Cognitive Task Analysis of Expertise in Air Traffic Control. ” The International Journal of Aviation Psychology, No. 3, 1993.

• Infeasible to get data from real controllers • Most experiments use retired controllers or student volunteers • Retired controllers are rare, students get bored, where to get more data? Trajectories, aircraft states, player inputs Contrails: Air traffic control game for Android Replay Engine on Server

The advantages of Big Data: A Typical ATC experiment 1 Contrails to date 28 participants 168 trials (6 each) 1391 installs 10, 391 games played Local US college students 10+ countries Max individual sample (est): 100 planes Most active user: 9489 planes Contrails install base as of 2/14/2012 Android Market Statistics Users by country, as of 2/14/2012 1 M. Stone et al. , “Prospective memory in dynamic environments: Effects of load, delay, and phonological rehearsal. ” Memory, 2001.

3. Treating breast cancer 30

Western Blots Tens of data points

Reverse Phase Protein Arrays (RPPA) Tens of Thousands of data points [Gordon Mills, MD Anderson Cancer Center]

Mass Cytometry – Time of Flight (Cy. TOF) Inductively Coupled Plasma (ICP) (Time Of Flight) mass spectrometer

Cy. TOF data Tens to hundreds of millions of data points [Brend, Eli]

(a) HER 2 inhibition is persistent, but its effects on HER 3 and AKT inhibition are transient (b) After 48 hours of applying Gefitinib, HER 3 is transferred from the cytoplasm to the cell membrane (c) p. HER 3 does not survive the application of Gefitinib when AKT is activated [Sergina et al, 2007]

Model identification • Identifying network structure • Reactions modeled as simple mass action or catalytic equations • Abstract variables modeling the transport mechanism

Model implications Control engineering point of view: • steer the state of the cells to a new equilibrium • low AKT correlated with cell death, seek equilibrium with low AKT • low membrane HER 3 may prevent recovery of AKT • different drugs could be applied at different times: 1. reduce membrane HER 3 AKT

Model implications Control engineering point of view: • steer the state of the cells to a new equilibrium • low AKT correlated with cell death, seek equilibrium with low AKT • low membrane HER 3 may prevent recovery of AKT • different drugs could be applied at different times: 1. reduce membrane HER 3 AKT

Model implications Control engineering point of view: • steer the state of the cells to a new equilibrium • low AKT correlated with cell death, seek equilibrium with low AKT • low membrane HER 3 may prevent recovery of AKT • different drugs could be applied at different times: 1. reduce membrane HER 3 2. inhibit HER 2 first drug is used to get the cells to a state more vulnerable to second AKT lapatinib

Experimental Results Preliminary test on SKBR 3, 250 n. M Lap was applied after treatment with HRG. Apoptosis rates shown: Treatment Lap Lap Ave. Scheme Ave % apoptosis 13. 56 6. 89 11. 3 41. 9 51. 85 59. 35 0. 1 ng/ml HRG (to 10 ng/ml HRG (over model the HRG in the activation) body) 51. 0

Conclusions • “Physics-based” models not always available: – Systems that involve human action – Systems with thousands of variables • Big Data has and will augment our abilities to identify, interact with, and control these systems • Current projects: – Action. Webs: • Energy-efficient HVAC control • Energy-efficient Air Traffic Control • Learning human action from data – Biology: • Cancer • Development • Metabolic networks 41