Terrain Mapping and Obstacle Detection for Unmanned Autonomous

Terrain Mapping and Obstacle Detection for Unmanned Autonomous Ground Vehicle Without Sensor Stabilization October 20, 2006 Amit Puntambekar Advisor: Dr. Arun Lakhotia Team Cajun. Bot www. cajunbot. com

Presentation Overview ► Introduction and motivation ► Related work ► Terrain mapping and obstacle detection algorithm ► Sensor error handling ► Algorithm evaluation ► Conclusion and future work Estimated presentation time: 50 minutes

DARPA Grand Challenge ► History ► Autonomous Ground Robots ► Application of AGV’s ► Examples Top: Mars Rover by NASA, Bottom: i. Gator by i. Robot

DARPA Grand Challenge ► Grand Challenge 2004 ► Grand Challenge 2005

Components of Autonomous System ► Hardware – Sensors, Electronics, etc. ► Software Obstacle Detection Path Planning Steering

Obstacles ► Man Made ► Natural

Types of Obstacles ► Static - Rocks, Cones, Steep Slopes, etc. ► Dynamic – Moving Cars, Gate, etc. ► Negative Obstacles – Ditches, Potholes, etc.

Motivation ► Timely Obstacle Detection -Top speed of vehicle: 25 mi/hr (11. 17 m/s). -Even a second delay in detecting obstacle might be fatal. ► Static and Dynamic Obstacles ► Negative Obstacles

Sensors ► GPS § Position information ► INS § Orientation ► LIDAR § Range

GPS/INS ► Principle of Operation ► Data Format ► Erroneous Conditions GPS INS Position + Position 5 Hz Orientation 100 Hz

LIDAR 0 degree 180 degree

LIDAR Terminologies Time Stamp ► LIDAR § Beams Scan • 1 Scan = 180 beams • 75 Scans per Sec Range Laser Beams

LIDAR Mounting Ø Parallel to Ground ØUsed by CMU ØMinimum ØSlopes obstacle size as obstacles ØSensitive to ØVibrations ØMountings ØAir Pressure

LIDAR Mounting ► Sweeping the terrain § Scans sweep terrain § Successive scans are geographically close Consecutive scans on flat ground

LIDAR Mounting ► Vertical Mounting § Team GRAY § Data Discontinuity ► Combination Mounting

Algorithms for Sweeping LIDARs ► Consecutive § § ► Detect Discontinuity in data Team GRAY, METIOR Plane Fitting § § ► The laser scans incrementally sweep the surface Analyzing the consecutive scans to determine change in geometry of the terrain Data Discontinuity § § ► scan analysis Best fitting plane computation Virginia Tech Slope Computation § § Change in slope of the scans is computed CMU, Team ENSCO

Prior Work Review ► Dependent on § Incremental scan sweeping ►Flat terrain § Sensor mountings ►Will not work if sensor mounting is changed

Off-Road Conditions- Bumps ► Indoor Vs. Off-road Environments Ø Effect of Bumps 2 3 4 1 Scattered scans due to bumps

Sensor Stabilization ► Specific Sensor Stabilizers ► Vehicle Suspensions ü 22 out of the 23 2005 Grand Challenge finalist team had vehicle suspensions or hardware sensor stabilizers to mitigate bumps. ü Teams like CMU, IRV had both ü Cajun. Bot was the only entry without sensor stabilizer and suspensions Top: Sandstorm from CMU; Bottom: IRV from Indiana Robotics

Sensor Stabilizers ► Cost - The cost of the CMU Gimbal is approximately $70, 000. ► Single Point of Failure

Research Contribution ► Off-Road Obstacle Detection System § Without sensor stabilization § Not sensitive to sensor mountings § Accounts for GPS errors § Scales well with number of sensors

Core Algorithm ► Obstacle Detection Algorithm § Theory § Implementation for a real time system – Cajun. Bot § Error Handling

Algorithm Theory Points Triangle Formation Slope Computation

Algorithm Theory (Continued) Ø High Absolute Slope Ø High Relative Slope Ø Height Discontinuity Obstacle Triangle Analysis

Obstacle Detection ► High Absolute Slope - Large surfaces where triangles can be formed Eg: Wall, cars, etc. ► High Relative Slope - Obstacles on slope - When obstacles are not large enough to register three LIDAR beams to form triangles - Negative obstacles ► High Elevation Change - Narrow obstacles like poles - Negative Obstacles angle Top: Virtual Triangle on a wall like obstacle Bottom: Obstacle on a slope

Position . . . . Data Fusion + . . . . (Range, Angle), 75 n tio uta mp Co pe Slo Real Time System

Effect of Bumps Ø Scans Scattered due to bumps Ø Consecutive scans might be geographically far apart 2 3 4 1

Position, Location . . . Data Fusion + . . . …. . . . Slope Grid (Range, Angle), 75 n tio uta mp co pe Slo Spatial Griding

Ø Temporal of GPS Data Consistency accuracy ØGPS Drift 2 3 4 1

Sensor Error - GPS Drift X Axis: Time (s) Y Axis: Height (m) ► What is GPS Drift ? Moving § Gradual drift in the GPS data ► Effects Drift? 0. 13 of GPS § Only temporally close data can be compared ► Factors causing GPS Drift § Hardware and connectivity with satellites 0. 20 Stationary Graph: GPS Z Vs. Time Data Collected on a flat parking lot. Vehicle traveling at 3 m/s

Handling GPS Drift ► Temporal Data Ordering ► GPS stable for 3 -4 seconds

Obstacle Detection Obstacle Cell Analysis Ø Absolute Slope Ø Relative Slope Ø Height Discontinuity Terrain Obstacle Map (TOM) Grid

Obstacle Detection – TOM Grid Analysis ► High Num of Hits) …. . Relative Slope Difference in Orientation > Threshold & Difference in Height > Threshold Elevation Change Max Height, Num of Centroids, Absolute Slope Absolute Orientation > Threshold Min Orientation, Min Height, Potential Obstacle Determination ► High (Max Orientation, . . . …. . Difference in Height > Threshold Terrain Obstacle Map Confidence Factors

Dynamic Obstacles ► Dynamic Obstacles § registered as obstacle at every location § Refresh Grid ► Grid New Data, T_new Last Access Time Stamp, T_old …. . Refreshing - TOM in a spatio-temporal grid - Refresh TOM Cells if existing data and new data are not temporally close - Aging based on access time stamp . . . …. . Terrain Obstacle Map (TOM)

Sensor Error GPS Spike ► What is GPS Spike ? Graph: GPS (Z) Vs. Time -Sudden change in the GPS data in a very short time interval. -The elevation data is more prone to GPS Spikes ► Causes Spike for GPS 30 m -Weak Signal -After ‘Dead Reckoning’ X-axis : Time(s) NQE 2005 Data Y-axis: Height(m)

Effect of GPS Spike Graph: GPS Z Vs. Time GPS Spike Data Playback

GPS Spike: Reason for False Obstacles Ø Corrupted data enters system Ø Slope computation gets erroneous Data Filter

Detecting GPS Spike Ø Median filter monitors INS Data Ø Erroneous data is discarded

Core Algorithm- Revised ► Terrain Modeling ► Obstacle Detection

Effect of Bumps - II ► INS, LIDAR data fusion -Mounting INS on LIDAR -Good Suspensions -Sensor Stabilizers -Rigid Platform GPS LIDARS INS Rigid Platform

Effects of Bumps ► INS, Angle LIDAR Data Rate Mismatch 0 X LIDAR INS 0 Time t 2 t 1 t 3

Sensor Fusion Interpolation ► CBWare - Data interpolation support LIDAR X Angle INS ► Robots with sensor stabilizers can fuse the most recent data from sensors ► In Cajun. Bot data is interpolated based on time of production Time t 1 t 2

Algorithm Evaluation ► Ability to utilize bumps to see further ► Accuracy of results ► Algorithm complexity ► Scalability ► Different obstacle types ► Sensor orientation independence

Data Sets ► Logged data from 2005 GC ► Testing in a controlled environment with Cajun. Bot-II ► Testing in a simulated environment - CBSim

Evaluation – Effects of Bumps ► Obstacle detection distance increases linearly with severity of bumps experienced

Testing in a controlled environment with Cajun. Bot-II Effect of bumps Experimental Setup With Bumps Without Bumps Distance 42. 6 m 28. 5 m False Obstacles NIL Comparison table Obstacle detection without Bumps Obstacle detection with bumps

Scalability ØSensor Specific Computation ØData Specific Computation Results based on analyzing Cajun. Bot-II logged data on a Dell machine with 3. 2 GHz Intel Processor and 1 GB RAM with full load (all other Cajun. Bot software modules running) on Fedora Core 2 operating system

Scalability and Bumps Results based on analyzing the 2005 GC final run logged data on a Dell machine with 1. 6 GHz Intel Processor and 1 GB RAM with full load (all other Cajun. Bot software modules running) on Fedora Core 2 operating system

Sensor Orientation Independence Ø Run 1 ØTop sensor Orientation (r, p, h)=(0, -1. 5, 1) Offsets (X, Y, Z)=(0. 25, 1, 0. 25) ØBottom Sensor Orientation (r, p, h)=(0, -3, 1. 5) Offsets (X, Y, Z)=(0. 25, 1. 5, -0. 5) Ø Run 2 ØTop sensor Orientation (r, p, h)=(2, -4. 5, 0) Offsets (X, Y, Z)=(0, 1, 0. 5) ØBottom Sensor Orientation (r, p, h)=(-2, -4, 2) Offsets (X, Y, Z)=(0, 2, 0) Cajun. Bot with two distinct sensor orientations

Comparison with different obstacle shapes at varying speed HD: Height Discontinuity, AS: Absolute Slope, RS: Relative Slope Ø Considerable increase in speed, negligible decrease in efficiency Ø 4%, 3. 2%, 3. 9% decrease in detection distance among the three shapes when the speed increases by 150 %

Limitations of the algorithm ► High precision INS required ► Sensitive to Boresight Misalignment § Angle at which LIDAR is mounted w. r. t INS § Fusion data from multiple LIDARs ► Limited LIDAR reflectivity § Water § Black surfaces – tar roads, etc.

Conclusion ü An Obstacle Detection system § Without sensor stabilization ► Takes § § advantage of bumps to see further Not sensitive to sensor mountings Accounts for GPS errors Scales well with number of sensors Handles dynamic obstacles Efficient Spatio-Temporal Grid ü Evaluation of system ü ü On logged data, live on Cajun. Bot-II, and simulator

Future Work ► Dynamic obstacle detection § Detecting trajectory and speed ► Obstacle Classification § Vegetation, mesh, etc

Questions ?

Thank you

Backup Slides

Boresight Angles / Offsets GPS Body on which the sensor is mounted P x INS p. R Computed Ground Line H Laser sensor n y Hs Actual Ground Line Laser beam

Limitations of the algorithm ► High precision INS required ► Sensitive to Boresight Misalignment § Angle at which LIDAR is mounted w. r. t INS § Fusion data from multiple LIDARs ► Water, black surfaces do not reflect LIDAR

LIDAR Errors ► Boresight Misalignment L 4 L 3 L 5 L 6 L 7 L 2 L 1 Effect of Mounting Angles on Sensor Data

Sensors Used ► LIDARs ► GPS ► INS

GPS/INS ► Principle of Operation ► Data Format ► Erroneous Conditions GPS INS Position + Position 5 Hz Orientation 100 Hz

LIDARS ► Principle of Operation – Time of Flight ► Data Format – 75 Hz at 0. 25 degree offset ► Erroneous Conditions Laser Beams

Data Handling ► To keep multiple copies of 2 minute worth of data in memory would require 550 MB of RAM ► Updating multiple copies is expensive

Data Handling

LIDAR Terminologies LIDAR Beams Laser Scan . . . 1 scan = 180 beams 75 scans per sec