Networked Robotic Systems An Overview Dr Thad Roppel

Networked Robotic Systems An Overview Dr. Thad Roppel

CRR Lab - A Brief History • Outgrowth of work started at Eglin AFB in 1992 • Infrared / Millimeter-Wave Radar Sensor Fusion • Follow-on funding DARPA e-NOSE • Best sensor platform? Robots • Many robots are better than one robot. .

SENSOR FUSION LABORATORY Problem Complexity: Human vs. Machine • Object recognition • Linguistics • Extraction of Relevant Features from Sensor Arrays MACHINE HARD EASY • Judging Maximum Potential Benefit • Thresholding • Tallying EASY HUMAN • Arithmetic • Logic HARD

IR / MMW DATA FUSION Support: AFOSR 1992 -93 Project Goal: Improved identification of military vehicles from aerial scenes. T-62 Tank M-113 Armored Personnel Carrier (APC) LANCE Missile Launcher

IR / MMW Fusion, cont’d APPROACH: IR SCENE PIXELS APC TANK LAUNCHER NEURAL NETWORK MMW RADAR DATA OVERALL RESULT: 14 % improvement with sensor fusion PERFORMANCE ASSESSMENT: • Multiple permutations • Confusion matrix • Average result A A + T L - T + - L +

Chemical Sensor Arrays Support: DARPA 1997 -99 PROJECT GOAL: Improved identification and detection of chemical plumes in non-laboratory conditions. COMMAND STATION PLUME RF LINK WIND SENSORS VEHICLE ROAD

Canine Training at IBDS Auburn is world-renowned for training of detection dogs at the Institute for Biological Detection Systems.

Chemical Sensor Arrays, cont’d Sensor Array Dynamic Response 5 4. 5 Sensor Voltage 4 3. 5 3 2. 5 2 1. 5 1 0. 5 0 0 100 200 300 400 Timestep Odor Sensor Array Sensor Outputs 500

Chemical Sensor Arrays, cont’d Preprocessing Below Threshold Above Threshold 4. 5 2 4 4 Sensor Number Sensor Voltage 5 3 2. 5 2 1. 5 Sensors 1 -15 1 8 10 12 14 0. 5 0 0 6 100 200 300 400 Timestep Raw Output 500 10 20 30 Timestep 40 50 Thresholded Binary Output

Chemical Sensor Arrays, cont’d Sample 2 Sample 3 1 1 20 20 xyl pth oil g 93 g 89 g 87 dal amm ace Sample 1 5 10 Sensor # 15

Chemical Sensor Arrays, cont’d network response Time Evolution of Confusion Matrix: Forward Sequence 1 timestep ace amm dal g 87 g 89 g 93 oil pth xyl 5 timesteps input categories 20 timesteps ace amm dal g 87 g 89 g 93 oil pth xyl Trained for 20 timesteps 50 timesteps 1 0. 9 0. 8 0. 7 0. 6 0. 5 Ideal Response 0. 4 0. 3 0. 2 0. 1 0

Chemical Sensor Arrays, cont’d network response Time Evolution of Confusion Matrix: Random Sequence Trained for 20 timesteps 1 timestep ace amm dal g 87 g 89 g 93 oil pth xyl input categories 20 timesteps 50 timesteps 1 0. 9 0. 8 0. 7 0. 6 0. 5 0. 4 Ideal Response 0. 3 0. 2 0. 1 0

BIOMIMETICS Support: Under discussion with AF Advanced Guidance Division, Munitions Directorate at Eglin AFB PROJECT GOAL: Learn sensor fusion from animals. Apply this to flying a drone to target using onboard video. Flies land accurately Bats catch evading insects in flight Bees find flowers

CRR Lab – History, Cont’d • Feb. 2006: Invited Joe Albree – Math Prof. at AUM - to speak to HKN about history of the engineering profession in USA. • I didn’t know he co-authored a book about the history of West Point with… • Gen. Chris Arney, ARO program in Multi-Agent Systems, who was organizing… • LIMES 2006 at West Point. Language for Intelligent Machines.

Cooperative Autonomous Robots for Reconnaissance White Paper for Chris Arney, ARO Prepared 8/29/2005 by Thad Roppel, ECE Dept. , Auburn University Contact: roppeth@auburn. edu, (334) 844 -1814

Hardware Testbed for Collaborative Robotics using Wireless Communication Chris Wilson – MS Dec 2009

Mounted optical mice and batteries

Wifistix (top card) and Gumstix (bottom card).

Eric Hildebrand ELEC 5530 HW 4 November 10, 2010 Dominion The year was 2143, and humanity was at the will of a single man. Known only as “Roppeth”, an evil mastermind had created an army that defeated everything humanity threw at it. No one knew where this army came from, but it could only be assumed that Roppeth created the first generation, and each new generation was spawned by the previous. What made the army so overpowering was the fact that they were autonomous robots controlled by the will of their leader but could act and behave independently from his control. These robots were bipeds, slightly larger than an average human, but completely overpowering to any human counterpart. …

Good News… • How to Survive a Robot Uprising

ASIMO • Highly functional biped • The future…? • Video • But for now, cooperation is more like this….

Oct. 2008 – Robotics and Autonomous Systems - Special Issue on Network Robot Systems • A probabilistic framework for entire WSN localization using a mobile robot • Action evaluation for mobile robot global localization in cooperative environments • Autonomous functional configuration of a network robot system • Framework and service allocation for network robot platform and execution of interdependent services • Robots in the kitchen: Exploiting ubiquitous sensing and actuation • Human behavior recognition using unconscious cameras and a visible robot in a network robot system • End-to-end congestion control protocols for remote programming of robots, using heterogeneous networks: A comparative analysis

NRS Definition • The IEEE Society of Robotics and Automation Technical Committee on Networked Robots provides the following definition of Networked Robots • Physical embodiment: Any NRS has to have at least a physical robot which incorporates hardware and software capabilities • Autonomous capabilities: A physical robot must have autonomous capabilities to be considered as a basic element of a NRS. • Network-based cooperation: The robots, environment sensors and humans must communicate and cooperate through a network. • Environment sensors and actuators: Besides the sensors of the robots, the environment must include other sensors, such as vision cameras and laser range finders, and other actuators, such as speakers and switches • Human-robot interaction: In order to consider a system as NRS, the system must have a human-robot related activity.

NRS Definition Expanded Two subclasses of Networked Robots: (1) Tele-operated *human supervisors send commands and receive feedback via the network. -Medicine, education, search & rescue, … (2) Autonomous, *robots and sensors exchange data via the network. *sensor network extends the effective sensing range of the robots *allows them to communicate with each other over long distances to coordinate their activity. *The robots in turn can deploy, repair, and maintain the sensor network to increase its longevity, and utility. *Broad challenge: develop a science base that couples communication to control to enable such new capabilities

Network Robot Types Three types of network robots: • Visible - can be seen – humanoid, pet, stuffed animal, etc. • Virtual - acts in a cyber space and makes use of information available on Internet. – avatar agent on a mobile phone or info kiosk • Unconscious - users are not aware of the presence of the robot – camera or a sensor embedded in infrastructure

Ubiquitous robotics network system for urban settings (URUS)

Physically Embedded Intelligent Systems (PEIS)

Japan NRS consists of four major Japanese companies: • NTT - telecommunications; • Toshiba - home appliances; • Mitsubishi Heavy Industries - industrial robots • ATR- telecommunication and social robotics R&D

NRS in USA • Net. Bot Lab at TAMU (Prof. Dezhen Song) • Ghostrider video clip • DARPA, JPL • Georgia Tech

WSN Localization

WSN Localization, cont’d (a)Scheme of the approach. The signal strength is used to estimate the position of the nodes of the network. The mobile robot computes centrally an initial estimation employing a separate Particle Filter for each node. In the second step, a decentralized Information Filter integrates information received from neighbor nodes and the robot, at each node. (b): An example, a ground robot (Romeo) driving through the network.

WSN Localization, cont’d

Robots in the Kitchen

ROS (Willow Garage) • Willow Garage

Conclusion Robots everywhere!!