UCF REU Presentation SUPERVISED TEMPORAL CONTRASTIVE LEARNING KALI

UCF REU Presentation SUPERVISED TEMPORAL CONTRASTIVE LEARNING KALI CARTER, ISHAN DAVE

Project Overview

Project Overview

Project Timeline • ECCV Vipriors Workshop Challenge • Training with ~3 k clips • No pretrained weights allowed • Have a strong supervised baseline using Cross Entropy Loss [done] • Experiment with different architectures [done] • Submission 1 [done] • Experiment with ensemble [done] • Experiment with optical flow TV-L 1 and design two stream model [done] • Experiment with multi-validation strategy [done] • Training with Supervised Contrastive Loss and fine tune with Cross Entropy Loss [done] • Final Submission for competition [done] • Training with Supervised Temporal Contrastive Loss • Download HACS dataset

Tasks Completed • Submitted technical report for competition • Cleaned up code for submission upon their request • Got blocked twice for HACS dataset download – brainstorming new way to approach this • Understood Thumos (UCF 101) dataset annotations • Wrote Contrastive loss function- trying understand the vectorized implementation

Supervised Temporal Contrastive Active-clip (when there is an activity) (Zi) should be 1. Attract to Active-clip (Zj) of the other video of the same class 2. Repel to the Inactive-clips (Zk) of the same video and other videos of the same class

Next Steps • Find a workaround to download the HACS dataset • Finish dataloader for temporal annotations • Start experiment with the Temporal Contrastive Loss • Another experiment with using Memory Bank

Approach Overview