Efficient Visual Object Tracking with Online Nearest Neighbor

Efficient Visual Object Tracking with Online Nearest Neighbor Classifier Many slides adapt from Steve Gu

Application Fields • Motion-based recognition ---human identification based on gait; • automated surveillance • --monitoring a scene to detect suspicious activities; • video indexing • --automatic annotation and retrieval of the videos in multimedia databases; • human-computer interaction -- gesture , eye gaze tracking for data input to computers; • Robot or vehicle navigation --video-based path planning and obstacle avoidance capabilities.

Main contributions • A tracking-by-detection framework is proposed that combines nearest-neighbor classification of bags of features • Efficient sub-window search • A framework that handles occlusion, background clutter, scale and appearance change State-of-art results on challenging sequences Demo

Outline • Object tracking and its challenges • The proposed tracking-by-detection framework –Bag-of-Features model –Online nearest neighbor classifier –Efficient sub-window search • Analysis and results • Result on our data

Challenges in object tracking • • Occlusion Scale change Background clutter Appearance change —loss of information from 3 D world on a 2 D image, —scene illumination changes —complex object motion, —nonrigid or articulated nature of objects, —real-time processing requirements.

Occlusion

Scale change

Background Clutter

Appearance change

Main Contributions • A simple yet effective visual tracker, combine nearest-neighbor classification of bags of features • A framework that handles occlusion, background clutter, scale and appearance change • Can be implemented efficiently with ESS.

-----The main advantages of tracking by detection come from the flexibility and adaptability of its underlying representation of appearance.

Tracking-by-detection framework • Appearance Model

The Objective Given: • the object model in the previous frame: Ok-1 • the background model. B, which is static • the location of the tracked window: Wk Estimate • the updated object model: Ok

The Motion Model Given • –the object model in the previous frame: Ok-1 • –the background model B, which is static • –the window in the previous frame: Wk-1 • –the current test window W Compute • –the matching score between Wand Wk-1 given Ok-1

Tracking with ESS • We modify the quality function: • Easy to show that the quality function satisfies the criteria for branch and bound

Limitation • SIFT descriptor cannot handle uniform regions and motion blur • No advanced motion model is utilized –e. g. Kalmanfilter, particle filter, etc • current tracker cannot localize objects very precisely when the object’s shape deforms.

Comparison with MIL

Application in our project Application background: • Robot walks around, taking pictures intermittently; so the • View, scale of object change when robot is approaching, leaving, walking around the object. • As robot walking around , the background changes

Changes in view (appearance), scale, occlusion and background

SIFT

Revised • Feature , from sift to color sift and dense sift • Update the background model

Tracking result with dense-sift

How to improve • Object representation • Features representation