FPGAbased Platform for RealTime Stereo Vision Sergiy Zhelnakov

FPGA-based Platform for Real-Time Stereo Vision Sergiy Zhelnakov, Pil Woo (Peter) Chun, Valeri Kirischian Supervisor: Dr. Lev Kirischian Reconfigurable Embedded Systems Lab Ryerson University, Toronto, ON

Motivation § Areas of application: – Real-time Stereo-Vision Systems – Telematic Systems: Remote Control of Manipulators in Hazardous Areas – Virtual Reality Systems and Simulators – UAV Navigation Systems – Telemedicine – Surveillance / Security Systems

Objectives § Development of the Run-Time Reconfigurable Platform for implementation, testing and realtime verification of algorithms for stereo-vision stereo-image recognition, and visualization of 3 D images § Implementation and test of real-time stereovideo processing algorithms (e. g. Edge Detection in moving objects)

Specification § Functional specification: – The system performs: image capture from two color cameras, stereo video visualization with shutter glasses, image processing (edge detection) § Technical specification – Input: video data with § § § spatial resolution 640 x 480 pixels frame rate: 30 fps color, 8 -bit resolution – Output: standard SVGA § resolution: 640 x 480 pixels § frame rate: not less than 60 fps Fastest process: 40 n. S per pixel output

Platform Components and Links Stereo-Image Capturing & Video Pre-processing Module Run-time Reconfigurable Multi-Stream Video Processor Video Processing Module Multi-Channel Post -processing Videooutput Module Shutter Glasses CRT Monitor LCD Projector

Implemented Algorithms § Color matching: (Bayer pattern) – at 2 x 30 frames/sec and 640 x 480 resolution

Implemented Algorithms § Color matching

Implemented Algorithms § Edge detection (Robert Cross) background 512 pixels Max (|d-a|, |b-c|) , where 480 pixels a b c d

Implemented Algorithms § Edge detection (Robert Cross) 512 pixels Block RAM 2/1 (previous row data) Block RAM 1/2 (current row data) Temporary Storage Calculation = a 1 b 1 c 1 d 1 e 1 f 1 …………… x 1 y 1 z 1 X 2 Y 2 Z 2 512 pixels A 2 B 2 C 2 D 2 E 2 F 2 ……………… ……… P C-b 1 P : Previous camera data C : Current camera data camera DATA C or P-a 1

Platform Assembly and Implementation Results § Implementation results – Image capture and visualization on the FPGA based Reconfigurable Functional Unit (RFU) (1 st stage - XCV 50 E; 2 nd - XC 2 V 1000) – Edge detection algorithm (Robert Cross) implemented

Summary § The Run-Time Reconfigurable Platform was developed for different Stereo-Vision applications. § All components of the Platform were tested by implementation of real-time stereo-image capture, image processing and visualization on stereovideo output display system § Perspectives of the platform development: stereopanoramic vision system