Removing Moving Objects from Point Cloud Scenes Krystof

Removing Moving Objects from Point Cloud Scenes Krystof Litomisky and Bir Bhanu International Workshop on Depth Image Analysis November 11, 2012

Motivation: SLAM Du 2011 Where is everyone? Henry 2010 Andreasson 2010 Henry 2012 Wurm 2010

Moving objects can cause issues… • Registration • Localization • Mapping • Navigation GOAL: A SLAM algorithm that ignores moving objects, but creates accurate, detailed, and consistent maps.

One Solution Remove moving objects before registration!

Overview Identifying and removing arbitrary moving objects from two point cloud views of a scene.

Plane Removal • Why? – Not moving – Helps segmentation • How? RANSAC. • Iteratively remove the largest plane until the one just removed is approximately horizontal

Euclidean Cluster Segmentation Two points are put in the same cluster if they are within 15 cm of each other

Viewpoint Feature Histograms

1 6 11 16 21 26 31 36 41 46 51 56 61 66 71 76 81 86 91 96 101 106 111 116 121 126 131 136 141 146 151 156 161 166 171 176 181 186 191 196 201 206 211 216 221 226 231 236 241 246 251 256 261 266 271 276 281 286 291 296 301 306 Finding Correspondences Allow Warping 5 bins (1. 6%)

Dynamic Time Warping Euclidean distance Dynamic Time Warping Iteratively take the closest pair of objects (in feature space) until there are no objects left in at least one cloud

Correspondences • Some objects will have no correspondences • Object motion:

Correspondences • Some objects will have no correspondences • Camera motion:

Correspondences • Some objects will have no correspondences • Occlusion:

Recreating the Clouds • Each cloud is reconstructed from: – Planes that were removed – Objects that were not removed original recreated, viewpoint changed

Experiments

Results input output

Results input output

Results input output

Results input output

Results input output

Object ROC Plot 1. 0 TPR: 1. 00 FPR: 0. 47 0. 9 0. 8 True Positive Rate 0. 7 0. 6 Dynamic Time Warping 0. 5 Histogram Difference 0. 4 0. 3 0. 2 0. 1 0. 0 0. 1 0. 2 0. 3 0. 4 0. 5 0. 6 False Positive Rate 0. 7 0. 8 0. 9 1. 0

Fraction of Static Points Retained 1. 0 Mean: 0. 85 0. 8 0. 6 0. 4 0. 2 0. 0 vincent hallway 1 hallway 2 couch bathroom albert at door hallway 3 hallway 4 jhon and jhon door jhon couch jin at door suresh and door ben's desk Xiaoqing ben at jhon desk

Conclusions & Future Direction • Remove moving objects from point cloud scenes – Arbitrary objects – Allow camera motion • Considerations: – Just look for people? – Runtime speed

Questions? Thank you.

References H. Du et al. , “Interactive 3 D modeling of indoor environments with a consumer depth camera, ” in Proceedings of the 13 th international conference on Ubiquitous computing - Ubi. Comp ’ 11, 2011, p. 75. H. Andreasson and A. J. Lilienthal, “ 6 D scan registration using depthinterpolated local image features, ” Robotics and Autonomous Systems, vol. 58, no. 2, pp. 157 -165, Feb. 2010. P. Henry, M. Krainin, E. Herbst, X. Ren, and D. Fox, “RGB-D mapping: Using Kinect-style depth cameras for dense 3 D modeling of indoor environments, ” The International Journal of Robotics Research, p. 0278364911434148 -, Feb. 2012. K. M. Wurm, A. Hornung, M. Bennewitz, C. Stachniss, and W. Burgard, “Octo. Map: A probabilistic, flexible, and compact 3 D map representation for robotic systems, ” in Proc. of the ICRA 2010 Workshop on Best Practice in 3 D Perception and Modeling for Mobile Manipulation, 2010. P. Henry, M. Krainin, E. Herbst, X. Ren, and D. Fox, “RGB-D Mapping: Using depth cameras for dense 3 D modeling of indoor environments, ” in the 12 th International Symposium on Experimental Robotics (ISER), 2010.