Dynamic Inverse Models in HumanCyberPhysical Systems Sam Burden

Dynamic Inverse Models in Human-Cyber-Physical Systems Sam Burden S. Shankar Sastry currently: Postdoc in EECS at UC Berkeley Sep 2015: Asst Prof in EE at UW Seattle Dean of Engineering at UC Berkeley Embedded Humans: Provably Correct Decision Making for Networks of Humans and Unmanned Systems (N 00014 -13 -1 -0341)

Human interaction with the physical world l Increasingly mediated by automation – – Augmented by hardware and software Machines adapt to collaborate and assist Human-Cyber-Physical Systems (HCPS)

Embedding humans amid automation l Can lead to performance degradation – pilot-induced oscillations in rotory/fixed-wing aircraft Mc. Ruer, Krendal 1974; Hess J. Guid. Cont. Dyn. 1997 Pavel et al. Prog. Aero. Sci. 2013 – overreliance on adaptive cruise control in cars Rudin-Brown, Parker Trans. Rch. F: Traffic Psych. and Behav. 2004 l Requires predictive models for human behavior

Predictable behavior from internal models l Popular paradigm posits pairs of internal models – forward model predicts sensory effect of motor action Sutton, Barto Psych. Rev. 1981; Jordan, Rumehlart Cog. Sci. 1992; Wolpert, et al. Science 1995 – inverse model computes motor command expected to yield desired behavior Kawato Curr. Opin. Neurobio. 1999; Thoroughman, Shadmehr Nature 2000; Conditt, Mussa-Ivaldi PNAS 1999 – Theoretical and empirical evidence for paired forward + inverse models Bhushan, Shadmehr Bio. Cybern. 1999; Sanner, Kosha Bio. Cybern. 1999 Hanuschkin, Ganguli, Hahnloser Front. Neural Circ. 2013; Giret, Kornfeld, Ganguli, Hahnloser PNAS 2014 l Parallels in control theory, robotics, AI – Internal models, adaptive control, learning Francis, Wonham Automatica 1976; Sastry, Bodson Prentice Hall 1989; Sutton, Barto, Williams IEEE CSM 1992 Crawford, Sastry UCB EECS 1996; Atkeson, Schaal ICML 1997; Papavassiliou, Russell IJCAI 1999

Instantiating internal models l Forward model ( M : U Y ): static vs. dynamic – static map (linear or nonlinear) Hanuschkin, Ganguli, Hahnloser Front. Neural Circ. 2013 Giret, Kornfeld, Ganguli, Hahnloser PNAS 2014 – dynamic map depends on intermediate state Thoroughman, Shadmehr Science 2000 Wolpert, Diedrichsen, Flanagan Nature Neurosci. 2011 l Inverse model ( M-1 : Y U ): hard to define – – static map may fail to be one-to-one or onto dynamic map may be acausal or need state estimate

Dynamic inverse models in HCPS l Today’s talk: dynamic inverse models from the perspective of mathematical control theory 1. derivation of dynamic inverse model 2. properties and implications for design of HCPS

Single input/single output forward model l Consider forward model in control-affine form: – – l x in Rn, u in R, y in R f, g in Cr(Rn, Rn), h in Cr(Rn, R) Suppose model has strict relative degree g in N: – Expressed in terms of Lie derivatives Lf h(x), Lg h(x): – intuitively, input affects g-th derivative of output e. g. g =2 for Lagrangian mechanical systems – applicable to interaction with physical world

Transformation of forward model Forward model: l Suppose model has strict relative degree g in N: l – l e. g. g =2 for Lagrangian mechanical systems Then model is linear in new coordinates: – There exists such that in coordinates forward model has the form simpler forward model – Choosing yields

Dynamic inverse model Forward model: l Given desired output yd, we seek desired input ud l – – l Note that exact tracking is too stringent – l Since y =x 1, there exists unique vd such that States z rendered unobservable by input vd! need initial cond. But it’s easy to achieve exponential tracking applying input yields – Ø How does tracking affect unobservable states z ?

Tracking with stable model pair Forward model: l Dynamic inverse model: l l Theorem: If forward and inverse models are exponentially stable, then feedforward input from dynamic inverse of internal model achieves exponential tracking for physical system. – – Trajectories converge for stable model pairs (M, M-1) Feedforward input “asymptotically inverts” dynamics

Tracking with stable model pair (M, M-1) l Theorem implies: – – For stable model pair, trajectories x, x’ converge to Feedforward input “asymptotically inverts” dynamics

Application to provably-correct interventions l Suppose human (H) implements inverse model: – – can infer desired task y. H from observed input u. H nominal forward model becomes: – automation can intervene to improve performance by minimizing cost function J: Rn R using input a – guarantees performance improvement following intervention in human-cyber-physical system

Dynamics of humans embedded w/ machines l Today: predictive models for interaction l Future: enhance human ability to interact with and control the built world – – – Human-Cyber-Physical Human Intranet Cybathlon Humans are the enabling technology

Appendix - Extensions and Generalizations - Properties of dynamic inverse model - Behavioral repertoire of humans

Extensions and generalizations Forward model: l Dynamic inverse model: l l Results easily extend to accommodate: – – multiple inputs / multiple outputs (small) perturbations in dynamics Sastry Springer 1999 – approximate input-output linearization Hauser Ph. D Thesis 1989; Hauser, Sastry, Kokotovic IEEE TAC 1992; Banaszuk, Hauser SIAM JCO 1996 – learning / adaptation / estimation of dynamics Sutton, Barto, Williams IEEE CSM 1992; Papavassiliou, Russell ICJAI 1999 Sastry, Bodson Prentice Hall 1989; Vrabie, Vamvoudakis, Lewis IET 2013

Properties of dynamic inverse model Forward model: l Dynamic inverse model: l l Property: dynamic inverse model is unique – – Exact tracking input determined by yd Independent of how internal model is represented or obtained (e. g. reinforcement learning, adaptive ctrl. ) Sutton, Barto, Williams IEEE CSM 1992; Papavassiliou, Russell ICJAI 1999 Sastry, Bodson Prentice Hall 1989; Vrabie, Vamvoudakis, Lewis IET 2013 – Impossible to learn if inverse model is unstable

Behavioral repertoire of humans l Too rich to model from first principles – Spans computational, algorithmic, & physical “levels of analysis” Marr, Poggio MIT AI MEMO 1976 – Influenced by neurophysiological state (cognitive load, hunger) La. Pointe, Stierwalt, Maitland Int. J. Speech-Lang. Pathology 2010; Danziger, Levav, Avnaim-Pesso PNAS 2011 l Can reduce dramatically during particular tasks – Bernstein posed the “problem of motor redundancy” Bernstein Pergamon Press 1967. – Perhaps instead we “exploit the bliss of motor abundance” e. g. using synergies, uncontrolled manifolds, optimality Latash Exp. Brain Rch. 2012; Ting, Macpherson J. Neurophys. 2005; Scholz, Schoner, Exp. Brain Rch. 1999 Todorov, Jordan Nature Neurosci. 2002; Diedrichsen, Shadmehr, Ivry Trends Cog. Sci. 2010 – For instance, locomotion naturally reduces dimensionality Burden, Revzen, Sastry IEEE TAC 2015