CS 294 142 Talking to Robots John Canny

CS 294 -142: Talking to Robots John Canny Fall 2017 CCN: 67236

Time-Contrastive Networks: Self-Supervised Learning from Multi-View Observation Pierre Sermanet, Corey Lynch, Jasmine Hsu, Sergey Levine (Google)

What should a visual representation do? • Explicit: represent image content in an obvious way: i. e. objects in the image and their relationships. • But this is a “homuncular” approach: It assumes there is some general intelligence downstream to consume the results. • Implicit: represents image content in a way that supports a task. • Simpler: the representation has fewer (task) requirements • More useful: the representation is part of the solution

Task Requirements This representation is geared toward robot imitation learning: • It captures shape information (pose), and contact (grasping) • It ignores viewpoint, occlusion, lighting, motion blur, etc. Nao Robot, U. Nantes

Triplet Loss An embedding (mapping from images to a vector) is trained on triplets:

Triplet Loss Background Used in Google’s Face. Net: • Uses an “anchor” instance x, a positive instance x+, and a negative instance x • 99. 93% accuracy on “Labeled Faces in the Wild” dataset.

Triplet Loss Background

Time-Contrastive Networks • The network is not trained end-to-end. • A vision front-end (trained e. g. on the Image. Net dataset) is placed in front of the trainable embedding network.

Triplet Loss An embedding (mapping from images to a vector) is trained on triplets:

Time-Contrastive Networks • Positives: give a different view of the same scene, and so variation from view is discouraged in the embedding: view, occlusion, lighting, motion, blur, … • Negatives: are taken from different times, and encourage differences that depend on time for a moving robot: differences in pose, contact, fullness in a container etc.

TCN • Need only multiple views of the scene. • But its important that the cameras move relative to each other, and relative to a world frame, to create variation in lighting and view. • Can do this with cameraphones, or with robot-held cameras.

TCN Architecture ‘… consists of the Inception model up until the layer “Mixed 5 d” (initialized with Image. Net pre-trained weights), followed by 2 convolutional layers, a spatial softmax layer [9] and a fully connected layer. The embedding is a fully connected layer with 128 units added on top of our custom model. ’

Evaluation – Alignment metric Designed to capture accurate timing of events in the videos: i. e. using the time of an event in v 1 (e. g. liquid starting to flow), find the best matching image in v 2, and compare its timestamp with a human labeling of that event.

Evaluation – Attributes metric Hand-label a subset of images with these attributes: • is the hand in contact with the container? (yes or no) • is the container within pouring distance of the recipient? (yes or no) • what is the tilt angle of the pouring container? (values 90, 45, 0 and 45 degrees) • is the liquid flowing? (yes or no) • does the recipient contain liquid? (yes or no)

Evaluation

Evaluation – breakdown by attributes

Qualitative Evaluation - k. NN

End-to-end Pose Imitation Training signals:

End-to-end Pose Imitation

Experiment Results

Types and Amounts of Supervision

Results: TC + Self

T-SNE Embedding

Videos sermanet. github. io/tcn

Take-aways • Triplet Loss provides a powerful way to custom-design a representation to emphasize certain differences, while minimizing others. • TCN generates a representation that emphasizes pose and contact changes, and minimizes view effects. • TCN also de-emphasizes the agent (human vs. robot), and supports imitation learning.

Opportunities • TCN should be a good front-end for other applications: pure reinforcement learning. • TCN should also simplify and improve the task of visual prediction, with or without action information. This has broad application, including reinforcement learning and planning.