Cognitive Mapping and Planning for Visual Navigation Saurabh

Cognitive Mapping and Planning for Visual Navigation Saurabh Gupta^", James Davidson", Sergey Levine^", Rahul Sukthankar", Jitendra Malik^" - UC Berkeley^, Google" CVPR 2017 Presented by Jason Xie

Problem ● Visual navigation in novel environments ● Environment map and topology are unknown to the mobile robot ○ ○ Geometric Context: ■ Goal defined as node on graph in robot’s coordinate frame Semantic Context: ■ Goal given as semantic command ● i. e. ‘go to a chair’, ‘go to a door’, ‘go to a table’

Traditional Approaches ● SLAM ○ Use Li. DAR point clouds, depth, Sf. M to create HD Maps

Traditional Approaches ● SLAM ○ ○ ● Use Li. DAR point clouds, depth, Sf. M to create HD Maps 2 step independent process: ■ Create map ■ Plan path through map Drawbacks ○ ○ ○ Requires direct observation Dependence of planner on mapper ■ e. g. a few missing textures might lead to failure of entire navigation system Does not account for predictable structure of environments LSD-SLAM: Large-Scale Direct Monocular SLAM (Engel et al. , ECCV 2014)

Recent Approaches ● Reinforcement Learning ○ ○ “Pixels to actions” Bypasses state estimation Target-driven Visual Navigation in Indoor Scenes using Deep Reinforcement Learning (Zhu et al. , ICRA 2017)

Recent Approaches ● Reinforcement Learning ○ ○ ● “Pixels to actions” Bypasses state estimation Drawbacks ○ Requires exploration before it can begin to successfully navigate Target-driven Visual Navigation in Indoor Scenes using Deep Reinforcement Learning (Zhu et al. , ICRA 2017)

Approach ● Want to retain structure of geometry and experience from learning ○ Rats build “cognitive maps” when navigating mazes ■ (E. C. Tolman. Cognitive maps in rats and men. ) ● Neural network policy for visual navigation: ○ ○ spatial memory to capture layout of the world a planner that can operate given partial observations joint unified architecture where the mapper is motivated by the planner’s needs is end-to-end trainable

Setup ● Simulated, static real world environment ● Grid coordinate system - robot takes actions in nodes on grid ● 4 discrete macro-actions ○ ○ a 0 a 1 a 2 a 3 : stay in place : rotate left by θ : rotate right by θ : move forward x cm ● Robot modeled as fixed cylinder with RGB/depth cameras (fixed height/pitch) ● Assumes perfect odometry

Goal ● Learn a policy that conveys best action to take given egomotion, image, target ○ ○ Position P t at time t At each timestep, ■ Robot receives image of its environment E ■ ● It = I(E, P t) Target ● Geometric coordinate frame: ● Semantic target

Mapping ● Fuses observed camera images to create a top-down 2 D egocentric belief map ○ ○ Creates latent representation (for relevant spatial memory as needed by planner) Can approximate free space map ● Free space estimate of the map given by accumulates free space prediction from the current view with the accumulated prediction from previous time steps transforms the free space prediction from the previous time step f t-1 according to the egomotion in the last step e t takes the current image It and outputs an estimate of the free space based on the view of the environment from the current location

Mapping ● Backward flow field ρ(e t) maps from current image f t to its relative location in f t 1 ○ W uses bi-linear sampling to apply ρ(e t) to f t-1 , which allows backpropagation of gradients ● Φ is learned through a CNN ○ ○ Can use semantic cues (presence of floors, walls, furniture, size/shape of common objects) Allows for estimation of free spaces even in the presence of partially visible textures accumulates free space prediction from the current view with the accumulated prediction from previous time steps transforms the free space prediction from the previous time step f t-1 according to the egomotion in the last step e t takes the current image It and outputs an estimate of the free space based on the view of the environment from the current location

Mapping

Planning ● Value iteration networks (Tamar et al. , NIPS 2016) ○ Differentiable w/ respect to inputs max pooling over channels ○ ○ ○ reward with inputs state s, action a probability of the next state given current s, a discount factor (0, 1) state information conveyed by the mapper value of each state is iteratively recalculated by taking a max over the values of its neighbors plus the reward of the transition to those neighboring states since planner is not modeled, it can learn policies which account for partially observed environments

Planning ● Value iteration networks (Tamar et al. , NIPS 2016) ○ ○ value of each state is iteratively recalculated by taking a max over the values of its neighbors plus the reward of the transition to those neighboring states can be seen as a trainable and differentiable generalization of Dijkstra’s algorithm

Planning ● Extension of hierarchical planner presented by Tamar et al. ○ ○ ○ Reduces complexity and learning difficulty at a cost of planning accuracy/resolution Conduct l value iterations in k-times downsampled environment Output is center cropped, upsampled, and value iterations are repeated at a finer scale

Joint Architecture ● Mapper optimizes for planner decision making instead of mapping for entire ground truth

Experimental Setup ● Stanford large-scale 3 D Indoor Spaces (S 3 DIS) dataset (Armeni et al. ) ○ ○ 3 D scans of 6 large-scale indoor areas from 3 different buildings of educational and office use. 2 buildings for training, last building for testing ● Res. Net 50 pre-trained on Image. Net to represent RGB images ○ Depth images acquired through cross modal distillation ● Trained using DAGGER ● Geometric goal: ○ Start node is sampled, end node is sampled within 32 steps (usually in another room or hallway) ● Semantic goal: ○ Agent is given one-hot vector representing the object category it must go to

Evaluation Metrics ● Nearest Neighbor Trajectory Transfer ○ ○ ○ At each time step, location in training set most similar to what the agent perceives is chosen The optimal action is executed and the process repeats for each time step Poor results (difference of median of 25 steps) - demonstrate differences between the training and testing set ● No image, goal location only with LSTM ○ Only consider’s robot coordinates without the first person camera image ● Reactive Policy ○ ○ Single Frame: Considers image at current time step Multiple Frames: Considers 3 images in addition to current view ● LSTM agent (only metric used for semantic task) ○ ○ Uses LSTM based memory For semantic task, evaluated on normalized Manhattan distance

Representative Successes (Geometric) ● CMP is able to traverse long distances and reason that the direction the agent initially picked was unlikely to succeed

Representative Failures (Geometric) Missed opening Tight spaces No progress Tight spaces

Semantic Task Chair in light gray region Success Moves through chair

Performance against analytic mapper ● Trained against analytic mapper (projects points in depth image into top view)

Takeaway ● CMP can outperform analytic mapper ● Single scale CMP outperforms multi scale CMP for a tradeoff of increased planning cost ● Limitations a. Assumes perfect odometry - real world robots have uncertainty b. Planning does not scale well for large environments ■ Multi scale can compensate, but at a tradeoff of decreased resolution and accuracy c. Goal may not be specified geometrically, discretized graph planning may have difficulty ● Results are promising and this may motivate more research for planning in optimal environments using learning methods

Questions?