Combining Visual and Spatial Appearance for Loop Closure

Combining Visual and Spatial Appearance for Loop Closure Detection in SLAM Kin Leong Ho, Paul Newman Oxford University Robotics Research Group

Motivation • Loop Closing – the task of deciding whether a vehicle has returned to a previously visited area • Popular approaches – nearest neighbour statistical gate, joint compatibility test

Image Loop Closure • Closing loops with visually salient features to avoid dependence on global position estimate

Closing the loop

Image Feature Extraction Process MSER detector Saliency detector

Demonstration of wide-baseline stability of visually salient features under perspective distortion and variation in illumination conditions

Matching Performance Query Image Tentative Match Similar posters found in the environment. [Newman, Ho ICRA 2005] Tentative Match

Results from Image Retrieval System

Limitations of Image Matching - Repetitive visual artifacts in urban environments such as posters, signs and wall pattern - False triggering of loop closure event based solely on image matching Tentative Match Query Image Tentative Match

Incorporating Spatial Information -Spatial information can be used to disambiguate visually confusing locations

Spatial Descriptors • Reduced a laser scan patch into a set of descriptor • Describe curvature of shape • Describe complexity of shape • Describe spatial configuration of laser scan

Segmentation • Laser scan is divided into smaller but sizeable segments • Segments are formed due to break in boundary or occlusions Original Laser Scan Set of Descriptors

Cumulative Angular Function • A plot of the cumulative change in turning angle versus the arc length of the segment Turning Angle • Invariant to rotation and translation Arc length of Segment

Entropy of CAF • A measure of complexity of segment • Weight descriptors to prefer between complex versus simple shapes CAF Histogram of Turning Angle

Inter-Segment Descriptors • Extract critical points: Critical points are points along a segment where there are sharp changes in cumulative angular function • Distances and relative orientations between critical points form links between segments

Descriptor Comparison 1 • Angular function disparity – minimum error between two cumulative angular functions

Descriptor Comparison 2 • entropy disparity – Kullback-Leiber distance

Edge Comparison • Matching of links • Links that are matched are coloured in black • Links that are not matched are coloured in blue

Spatial Similarity Score • Shape similarity metric comprises of two parts: shape similarity and spatial similarity

Results from Spatial Retrieval System

More Results

MSER Detector Query Image Query Laser Scan Selected Regions Saliency Detector Segmentation SIFT Descriptor Laser Scan Database Image Database Similarity Measure Combined Similarity Scores Similarity Measure

Visual Similarity Matrix

Spatial Similarity Matrix

Combined Similarity Matrix

Demonstration

Issues • Setting of threshold values • Principled way of combining similarity scores • At present limited to planar environments Current Extensions • Removal of repetitive images by spectral decomposition • Successful Application to 3 D laser mapping and SLAM

Questions Thank you!