Advanced Topics In Computer Vision Spring 2016 Presented

Advanced Topics In Computer Vision, Spring 2016 Presented by: Sima Sabah

Video Representation [Peng, X. , et al. 2014], [Jan van Gemert, Uv. A]

• HOG (Histogram of Oriented Gradients) [Dalal and Triggs, 2005]

• HOG (Histogram of Oriented Gradients) • Optical Flow and Histogram of Optical Flow (HOF) [Laptev et al. 2008]

• HOG (Histogram of Oriented Gradients) • Optical Flow and HOF • Trajectories

• HOG (Histogram of Oriented Gradients) • Optical Flow and HOF • Trajectories • MBH (Motion Boundary Histogram) [Dalal et al. ECCV 2006]

• HOG (Histogram of Oriented Gradients) • Optical Flow and HOF • Trajectories • MBH (Motion Boundary Histogram)

• Neural Networks

Hang Wang, Cordelia Shmid ICCV 2013

Trajectories: normalized displacement vectors [Wang et. al. IJCV’ 13]

• SURF (green) and optical flow (red)

Original optical flow

Successful examples Failure cases Removed trajectories (white) and foreground ones (green)

• Part-based Human detector [Prest et al. 2012]

Karen Simonyan, Andrew Zisserman NIPS 2014

224 x 3 224 x 2 L

• Optical Flow Stacking • Trajectory Stacking

• HOF, MBH • Can be learned Single convolutional layer (containing orientation sensitive filters) followed by rectification and pooling layers • Trajectory • Can be an input using Trajectory stacking • Still missing: • Local pooling over spatio-temporal tubes centered at the trajectories • Camera motion compensation

• UCF-101 – optical flow representation • Two-Stream Conv. Net

X. Wang, A. Farhadi, A. Gupta CVPR 2016

• Loss:

Spatial stream Conv. Net Temporal stream Conv. Net

• Initialize network weights with pre-trained Two-Stream Network. • Repeat: • Forward propagation and feature computing for each frame • Search Latent variables: • such that • Calculate joint loss • Perform back-propagation

• Objective: Spatial stream Conv. Net Spatial Distance Score Temporal stream Conv. Net • Model fusion: • 2 x. Temporal. Score + Spatial. Score Temporal Distance Score

• UCF-101 • HMDB 51 • ACT

• ACT • 11648 videos • 43 classes • 16 super classes