Exploration Robot with Stereovision Vladislav Richter richterv 2fit

Exploration Robot with Stereovision Vladislav Richter richterv 2@fit. cvut. cz Miroslav Skrbek FIT, CTU in Prague skrbek@fit. cvut. cz Vladislav Richter, Miroslav Skrbek PESW 2014

Outline • • • Introduction Robot's hardware Control Application Sensor visualization Kinect Vladislav Richter, Miroslav Skrbek PESW 2014 2

Exploration Robot Android phone or tablet Wi-Fi Android 3 D image (anaglyph) Commands forward backward rotate set_wheel_speed say get_eye_image … Vladislav Richter, Miroslav Skrbek PESW 2014 3

Robot's Hardware Speaker (speech synthesis) Roboard RB 110 (1 GHz PC, Linux) Audio codec with an amplifier Wi. Fi antena Cameras (stereo pair) 80 cm distance IR sensor Eye Movement (up/down) Vladislav Richter, Miroslav Skrbek PESW 2014 Accelerometer, gyroscope, battery voltage, battery current and temperature sensors inside 4

Remote Control Application • Directed for tablets and mobile phones • Android based application – Earlier version Android 2. 3 – Latest version Android 4. 4 • • Motor control Sensor visualization Sound localization Voice output Vladislav Richter, Miroslav Skrbek PESW 2014 5

Sensor Visualization • • • Accelerometer Gyroscope Motor speed System temperature Battery – Voltage – Current • IR distance sensors • Sound source location Vladislav Richter, Miroslav Skrbek PESW 2014 6

Accelerometer Data Visualizations Accelerometer data Terrain The model follows the movement of the robot Vladislav Richter, Miroslav Skrbek PESW 2014 7

Sound Source Localization Δt 1 Cross correlation I Localization Cross correlation II Δt 2 Bar with four microphones (two stereo pairs) Vladislav Richter, Miroslav Skrbek PESW 2014 8

Camera Images 3 D View Cameras: 640 x 480, 30 fps, YUYV Anaglyph Vladislav Richter, Miroslav Skrbek PESW 2014 9

Nearest object detection Left image Right image Open. CV based stereo correspondence calculation Depth map Nearest objects are white Vladislav Richter, Miroslav Skrbek PESW 2014 10

Motor Control Virtual Joystick Movement keypad Vladislav Richter, Miroslav Skrbek PESW 2014 11

Advanced Feature – GUI Adaptation Android get_xml_info XML file Description of sensors and commands available on the robot n io t a UI pt a ad G Vladislav Richter, Miroslav Skrbek PESW 2014 12

Kinect on the Robot UDP multi-cast streams - color image - depth map - infrared image - skeleton tracking data - accelerometer - audio Wi-Fi Server 15 fps UDP stream - motor control - stream control Vladislav Richter, Miroslav Skrbek PESW 2014 13

Kinect - Depth Map Image depth is 12 bits. To view the full range, a color scheme is used. Depth map Vladislav Richter, Miroslav Skrbek PESW 2014 14

Kinect - Color Image Head-up Display Vladislav Richter, Miroslav Skrbek PESW 2014 15

Kinect – Infrared Image Vladislav Richter, Miroslav Skrbek PESW 2014 16

Kinect - Skeleton Data Vladislav Richter, Miroslav Skrbek PESW 2014 17

Conclusion • • • Four-wheeled robot hardware Stereo camera pair for distance measurement Reach set of sensors Sound source location Voice output Android based control application – 3 D view by anaglyph – Sensor data visualization – Motor control Vladislav Richter, Miroslav Skrbek PESW 2014 18

Thank you for your attention Vladislav Richter, Miroslav Skrbek PESW 2014 19