Estimating Physical Intent For Exoskeleton Control or Reading

Estimating Physical Intent For Exoskeleton Control or Reading The Operator’s Mind Chris Atkeson, CMU

Estimating Physical Intent • Anticipate movement • We do it all the time: martial arts, partner dance, … • “Tells” in poker: Gaze, head and body movement, facial expression, … • Feel or see muscles tense, postural change, … • Suppose we could instrument our partner/opponent? What signals would we go after? What sensors would we use? • Predict muscle forces, and body movement. • Predict a state at a time: x(t), a trajectory, or a probability distribution: p(x(t)) • Estimate/track human decision process: what behavior is chosen, with what parameters? What utility function is the human optimizing? • Uses: exoskeleton control system; online operator situation awareness; cognitive assistance and physical guidance; command squad visualization; situation awareness, coordination, and cognitive assistance; after-action reports; and new operator training and operator learning from practice and coaching;

Time Scales • 10 sec - Behavior selection. Rely on perceiving situation and thinking like your operator and your opponents. What are the probabilities of various attacks? Responses? • 1 sec - Behavior selection and reflex responses. Measure set/tells to estimate probability of particular behaviors and reflexes. Maintain probability distribution of possible behaviors and reflexes. • 100 msec - Muscle activation. EMG, muscle activation modeling force helpful here. • 10 msec - Biomechanics. Joint position, velocity, contact forces, internal muscle and tissue forces. • 1 msec - Local and global accelerations and local deformation dynamics. Distributed MEMs IMUs on operator sensing local accelerations, angular velocities, vibration, impacts, shock waves, and other high frequency mechanical phenomenon.

Signals • perceive what the operator sees, hears, feels, and smells. • outward facing superhuman perception (RF signals, UV, IR, ultrasound imaging at contacts, . . . ) • brain signals, spinal signals (motorneuron pools), motor nerve signals, sensory nerve signals • muscle electrical signals (EMG) • muscle force signals (FMG), muscle force at tendons, muscle internal pressure • implanted tissue markers, other implanted sensors • tissue deformation tracked by tissue imaging (ultrasound, optical) • operator internal forces • operator-exoskeleton contact force • operator-exoskeleton contact deformation, deformation rate • train user to emit special signals (play a videogame with their body, hands, eyes, neural signals, muscle signals, . . . )

Timing (changing load in hand) • Fast reflexes (involuntary) - 30 -50 ms. • “Long loop reflexes" (can be voluntarily enabled/disabled) - 50 -80 ms. • Triggered reactions - 80 -120 ms. Reaction to an object slipping in a grasp is about 80 ms. [Johansson and Westling 1984] • Voluntary responses - 120 -180 ms. • Visual reaction times in the 200 ms range. • Typically, (full) force production trails EMG by about 30 ms in fast muscles (electromechanical delay, due to calcium ion dynamics traveling from sarkoplasmatic reticulum to cross bridge binding sites.

Implanted EMG electrodes (similar to what we put in pets) • • • http: //neural. iit. edu/research/imes/ http: //ieeexplore. ieee. org/xpl/login. jsp? tp=&arnumber=4716669 http: //link. springer. com/chapter/10. 1007%2 F 978 -3 -319 -08072 -7_12 http: //www. ncbi. nlm. nih. gov/pubmed/19224729 http: //www. ncbi. nlm. nih. gov/pmc/articles/PMC 2878855/

Muscle force • Implanted pressure sensor • Ultrasound tracking of tendon stretch • Optic fiber-based tendon strain measurement • http: //jap. physiology. org/content/99/2/603 • Finni T, Komi PV, and Lukkariniemi J. Achilles tendon loading during walking: application of a novel optic fiber technique. Eur J Appl Physiol Occup Physiol 77: 289– 291, 1998. • Komi PV, Belli A, Huttunen V, Bonnefoy R, Geyssant A, and Lacour JR. Optic fibre as a transducer of tendomuscular forces. Eur J Appl Physiol Occup Physiol 72: 278– 280, 1996.

Muscle movement • Implanted sensors measuring distance • Continuous imaging • Ultrasound (transmission vs. reflective? implanted? ) http: //www. jove. com/video/50595/a-novel-application-of-musculoskeletal-ultrasound-imaging

Soft contact force sensors (Yong Lae Park, CMU)