Depth Analysis With Stereo Cameras Timo Hohn and

Depth Analysis With Stereo Cameras Timo Hohn and Leo Nickerson Group 5

Functionality • Take stereo images of a scene and calculate the distance objects are away from the two cameras. • This distance will be represented on an output image as a color gradient that is displayed on an external VGA monitor. Motivation • 3 D depth analysis for remote robot imagery, where more information of the scene is necessary. • Object detection and avoidance applications for robot navigation where IR systems are not available.

Hardware Block Diagram

Software Flow Diagram Start (User Initiates Analysis) Collect JPEGS from Both Cameras Trim Images To Overlap Pixel Recognition Depth Analysis of Matched Pixels Update output image Modify Video Buffer to Include Output Image as Overlay

Design Challenges • Interfacing the Cameras` NTSC video in and RS 232 communication to the DE 2 board Adapting the camera communication library to work well with the DE 2 board Using the onboard TV decoder to collect frame buffers from one of the two cameras Sending manipulated frames to VGA out. Application Notes are available from previous projects • Pixel Recognition Open source libraries from the Open. CV project will be used for matching pixels May perform image subtraction to reduce number of pixels required to analyze

Included FPGA and External Components • Cyclone II Processor (currently economy version) • VGA and TV decoder accessed as Avalon-MM slaves • Use of onboard SRAM modules and the possibility of larger SDRAM storage depending on memory use • GPIO pins (Tx, Rx, and GND pins for each camera, which use 3. 3 V logic) • Additional +5 V power supply for camera modules (either USB wall plug or battery packs) • A single onboard push button for initiating image capture

Example C++ Module For Finding Distances #include "Dist. Find. h" #include <cmath> Dist. Find: : Dist. Find(int x_res, int fov_degree, double width){ x_res = x_res; fov_rad = (double)fov_degree*PI/180; rad_per_pixel = fov_rad/x_res; width = width; } double Dist. Find: : find(int x_camera 1, int x_camera 2){ double offset 1 = rad_per_pixel/2 + abs(x_camera 1320)*rad_per_pixel; double offset 2 = rad_per_pixel/2 + abs(x_camera 2320)*rad_per_pixel; … … (x_camera 1>320) ? offset 1*=-1 : ; (x_camera 2<320) ? offset 2*=-1 : ; double phi 1 = PI/2 + offset 1; double phi 2 = PI/2 + offset 2; double angle_c = PI - phi 1 - phi 2; double sine_ratio = width/sin(angle_c); double side 1 = sine_ratio*sin(phi 2); double side 2 = sine_ratio*sin(phi 1); return (phi 1<phi 2) ? sin(phi 1)*side 1 : sin(phi 2)*side 2; }

Unit Testing Integration Testing • Using stored images to test depth and pixel recognition tasks • Sending an image out the VGA port to test that interface • Collect raw data from NTSC video and observe its accuracy • Take raw video input from one camera and route it directly to the VGA port • Starting with a basic, high contrast environment with a single object. • Work our way up to more complex environments involving overlapping objects.

Optional Features Additional Features • Performing depth calculations on real time video (additional hardware required) • Automation and optimization of frequent image captures (eliminate the need for user initialization) Secondary Features • Removal of the NTSC video overlay and only display still images • Simplification of analysis environment if pixel recognition algorithms deem inaccurate

Questions?