Bilateral Teleoperation of Multiple Cooperative Robots over Delayed

Bilateral Teleoperation of Multiple Cooperative Robots over Delayed Communication Network: Application Dongjun Lee Mark W. Spong Oscar Martinez-Palafox d-lee@control. csl. uiuc. edu, {mspong, pomartin}@uiuc. edu Research partially supported by the Office of Naval Research (N 00014 -02 -1 -0011 and N 00014 -05 -1 -0186), the National Science Foundation (IIS 02 -33314 and CCR 02 -09202), and the College of Engineering at the University of Illinois.

Outline 1. Review of the Proposed Control Framework 2. Simulation Results 3. Semi-Experimental Results 4. Conclusions

Bilateral Teleoperation of Cooperative Multi-Robots Combine advantages of - bilateral teleoperation: human intervention in uncertain environments - multi-robot cooperation: mechanical strength/dexterity & robustness/safety - applications: remote construction/maintenance of space/under-water/civil structures in possibly hazardous environments

Semi-Autonomous Teleoperation behavior of overall group (and grasped object) Locked System Passive Coupling: dropping object!!! decoupling internal formation shape (cooperative grasping) Shape System - Passive Decomposition [Lee&Li, CDC 03] decomposes slave dynamics into decoupled shape (formation shape) and locked (overall group motion) systems - Local grasping control of decoupled shape system: secure/tight grasping regardless of human command via delayed comm. Channel - Bilateral teleoperation of locked system: by operating the master robot of manageably small DOF, human can tele-control the behavior of the grasped object over the delayed comm. channel while perceiving external forces

System Modelling and Grasping Shape Function Dynamics of a single master (m-DOF) inertia Coriolis velocity control human force Dynamics of multiple slave robots (n 1+n 2+…+n. N-DOF) Stack -up n-DOF product system (n=n 1+n 2+…+n. N-dimensional) Grasping Shape Function: Rn→Rn-m master’s DOF desired (constant) grasping shape function describes internal group formation shape

Passive Decomposition and Local Grasping Control Decomposed Slave Dynamics passive decoupling locked system Locked system: abstracts overall behavior of multiple slave robots and grasped object Shape system: describes internal group formation of slave robots (i. e. cooperative grasping) shape system Local Grasping Control desired grasping shape FF cancellation of internal force: although dynamics is decoupled, other effects (e. g. object’s inertia) can still perturb the shape system through internal force FE

Scattering-Based Teleoperation of Locked System Dynamics of Master Robot and Slave Locked System (both are m-DOF) control human/combined external forces Locked System Shape system (locally controlled) Scattering-Based Teleoperation of Locked system: - humans can tele-control the behavior of the grasped object over delayed comm. channel while perceiving external forces acting on the object and slaves - asymptotic position coordination/static force reflection

Outline 1. Review of the Proposed Control Framework 2. Simulation Results 3. Semi-Experimental Results 4. Conclusions

Simulation Settings 3 -DOF Master Three 3 -DOF Slave Robots agent 1 deformable object (no friction) Delay 0. 5 s (x, y)-translation yaw rotation agent 2 agent 3 - grasping shape function is defined s. t. three slaves form an equilateral triangle (w/ side length L) whose rotation is specified by the heading of agent 2 - human operator can tele-control the position and rotation of the triangle by operating 3 -DOF master robot (translation and yaw) - 10% identification errors for inertias of robots (nominal: m=1 kg, I=1 kgm 2)

Simulation: Importance of Decoupling Without Passive Decoupling Control With Passive Decoupling Control - no grasped object (just motion coordination) w/ PD-based grasping control - without decoupling control, grasping shape (i. e. shape system) is perturbed by human command overall group behavior - slight grasping shape distortion w/ decoupling is due to inertial uncertainty

Simulation: Heavy Object Fixtureless Manipulation With Feedforward Cancellation of Internal Force Without Feedforward Cancellation of Internal Force - even if dynamics is decoupled, inertial effect of object (w/ frictionless contact) perturbs cooperative grasping through the internal force FE - this perturbation can be cancelled out by feedforward cancellation of the internal

Heavy Object Manipulation: Contact/Human Force good load balance due to grasping rigidity due to grasping shape deformation - human can perceive the total inertias of the grasped object and the slave robots - human can also perceive sensation of grasping loss - better load-balancing is achieved w/ FF-cancellation of the internal force F E, as grasping shape becomes more rigid

Simulation: Force Reflection Three 3 -DOF Slave Robots human force agent 1 deformable object external force agent 2 agent 3 due to object’s deformation - external forcing (x-direction) on the grasped object is faithfully reflected to the human operator (i. e. haptic feedback) - load balancing among slaves is degraded as the grasped object is deformed in the rigidly-maintained grasping shape

Outline 1. Review of the Proposed Control Framework 2. Simulation Results 3. Semi-Experimental Results 4. Conclusions

Semi-Experiment Setting 2 -DOF Master Three 2 -DOF Slave Robots agent 2 deformable object Delay 0. 5 s external force Delay 0. 5 s PHANTo. M Desktop: constrained on plane (i. e. (x, y)-translation) agent 1 agent 3 - three slave robots: 2 -DOF point mass dynamics (only x, y translations) - Phantom Desktop is used as master with its workspace constrained on (x, y)plane - Grasping shape function: : specifies rotation and shape of the triangle formed by the three slaves

Semi-Experiment: Deformable Object Manipulation human perceives inertias of object/slaves secure/precise grasping w/ FF-term due to object deformation - x-directional motion (full-range) w/ fixtureless grasping - grasping security is preserved regardless of human command - human can perceive the combined inertia of slaves and grasped object - increase of some slaves' contact force due to inertia/deformation of object

Semi-Experiment: Obstacle Perception human perceives external force due to object deformation Secure/precise grasping w/ FF-term - external force (x-direction) on the grasped object center - force generated by the PI-action in the local impedance controls - object’s deformation again leads in unbalanced load sharing among slaves

Conclusions We propose a control framework for bilateral teleoperation of multiple cooperative robots over delayed master-slave comm. channel: - passive decomposition: the decoupled shape (cooperative grasping) and locked (behavior of the grasped object) systems - local grasping control for the shape system: high precision cooperative grasping regardless of human command/comm. delays - scattering-based bilateral teleoperation of the locked system: human can tele-control behavior of the cooperatively grasped object by operating a small-DOF of the master robot, while perceiving combined force on the slaves and the grasped object over the delayed comm. channel - enforce energetic passivity: interaction safety and stability - Semi-experiment and simulation results are presented and validate efficacy of the proposed control framework Possible impacts on emerging or traditional applications: - remote construction/maintenance of space/under-water/civil structures in hostile/hazardous environments