A Visual Stepping Stone to AI Larry Zitnick

Alex. Net, 2012 Recognition 2016 Neocognitron, 1983 1984 Data GPUs Backprop

AI 2048 Recognition 2016 1984 Data GPUs Backprop

AI 2048 Recognition 2016 1984 Data GPUs Backprop More data More compute Backprop

A giraffe standing in the grass next to a tree. A man riding a

An airplane is parked on the tarmac at an airport. A man riding a

Recognition 2016 1984 Data GPUs Backprop AI ?

There is a cat in this image. I’ll give you a $1 if you

Semi-supervised learning A yellow Vespa parked in a lot with other cars. A store

Unlikely Likely fence pink Learning Visual Classifiers using Human-centric Annotations Misra et al. ,

Reinforcement learning Maze. Base: A sandbox for experimenting with reinforcement learning, reasoning, and planning.

Unsupervised learning Correct Wrong Unsupervised Learning using Sequential Verification for Action Recognition, Misra et

UNDERSTANDING Mary went to the hallway. John moved to the bathroom. Mary travelled to

PREDICTING Learning Physical Intuition of Block Towers by Example

DATA We Are Humor Beings: Understanding and Predicting Visual Humor, Chandrasekaran et al. ,

MEASURING Visual Question Answering Is this a vegetarian pizza? Does this person have 20/20

Data GPUs Backprop 1984 Recognition 2016 Learning Reasoning AI 2048

Data GPUs Backprop 1984 Recognition 2016 Reasoning Learning AI 2048

Recognition 2016 1984 Learning Reasoning AI 20**

Slides: 28

Download presentation

A Visual Stepping Stone to AI Larry Zitnick Facebook AI Research

Recognition? Neocognitron, 1983 1984

Alex. Net, 2012 Recognition 2016 Neocognitron, 1983 1984 Data GPUs Backprop

AI 2048 Recognition 2016 1984 Data GPUs Backprop

AI 2048 Recognition 2016 1984 Data GPUs Backprop More data More compute Backprop

A giraffe standing in the grass next to a tree. A man riding a wave on a surfboard in the water. Mind’s Eye: A Recurrent Visual Representation for Image Caption Generation, Chen and Zitnick, CVPR

An airplane is parked on the tarmac at an airport. A man riding a motorcycle on a beach. Building Machines That Learn and Think Like People,

MIRROR

AI 2048 Recognition 2016 1984 Data GPUs Backprop More Data More Compute Backprop

Recognition 2016 1984 Data GPUs Backprop AI ?

LEARNING

There is a cat in this image. I’ll give you a $1 if you find a cat image. Supervised Reinforcement I didn’t look closely but I think there’s a cat. Semisupervised I’m not telling you anything!!! Unsupervised

Supervised

Semi-supervised learning A yellow Vespa parked in a lot with other cars. A store display that has a lot of bananas on sale.

Unlikely Likely fence pink Learning Visual Classifiers using Human-centric Annotations Misra et al. , CVPR 2016

Reinforcement learning Maze. Base: A sandbox for experimenting with reinforcement learning, reasoning, and planning. Maze. Base: A Sandbox for Learning From Games

Unsupervised learning Correct Wrong Unsupervised Learning using Sequential Verification for Action Recognition, Misra et al. ,

l a u s i V ^ REASONING

UNDERSTANDING Mary went to the hallway. John moved to the bathroom. Mary travelled to the kitchen. Where is Mary? Dialog-based Language Learning, Weston, Ar. Xiv

PREDICTING Learning Physical Intuition of Block Towers by Example

DATA We Are Humor Beings: Understanding and Predicting Visual Humor, Chandrasekaran et al. ,

MEASURING Visual Question Answering Is this a vegetarian pizza? Does this person have 20/20 vision? VQA: Visual Question Answering Antol et al. , ICCV 2015

Looking forward

Data GPUs Backprop 1984 Recognition 2016 Learning Reasoning AI 2048

Data GPUs Backprop 1984 Recognition 2016 Reasoning Learning AI 2048

Recognition 2016 1984 Learning Reasoning AI 20**

Facebook AI Research