WEEK 7 2020 CRCV HSAP PRESENTATION MEGAN SHAH

WEEK 7 2020 CRCV HSAP PRESENTATION MEGAN SHAH AUGUST 4, 2020 - AUGUST 12, 2020

WHAT I HAVE DONE • Primarily worked on data collection for UTRAP project– taking various objects and using a white box adversarial attack technique to make the computer misclassify the object (example: submarine as a cat) • Classes: Warplanes, Military Uniforms, Tanks, Assault Rifles, Submarines, Aircraft Carriers, Cannons • Collected around 1000 images per class for warplanes, tanks, submarines and 500 images for cannon • Mass selected and downloaded images using BULKR • Continued with Data Collection– sorted and entered You. Tube IDs, dates, and uploader names into Excel spreadsheet– collected a total of approx. 400 total videos • Collected Ground and Aerial Videos for Lima and Washington DC (10 ground videos, 10 aerial videos per city)

RESEARCH PAPER READING • Continued reading paper titled “Video Description: A Survey of Methods, Datasets, and Evaluation Metrics”– have read 25 pages • The availability of datasets determines the pace of advancement of research– datasets are categorized into four main classes: cooking, movies, videos in the wild, and social media • Cooking: MP-II Cooking– 65 fine grained images of participants making dishes, annotated, when scene is not annotated ‘background activity’ is generated; You. Cook– more challenging viewpoint because diff. backgrounds, divided into different cooking styles; TACo. S– subset of MP-II Composites, manipulation of cooking ingredients; TACo. S- Multilevel– detailed descriptions of video, short description, single sentence; You. Cook II– open domain • Movies: MPII-MD– audio descriptions extracted from Hollywood movies; M-VAD– short clips from many different movies

RESEARCH PAPER READING • Social Media: Video. Story– multi-sentence description; Activity. Net entities– grounded to bounding boxes and annotations • Videos in the Wild: MSVD– clips with human annotated sentences, audio is muted; MSR-VTT– open domain videos used for video captioning; Charades-- wide variety of indoor household activities; VTW– one sentence description with highly complex words; Activity. Net Captions– dense natural language descriptions, multiple descriptions for each video • Evaluation Metrics– performed over machine generated captions/descriptions, automatic evaluations and human evaluations are the two main categories of evaluations • • Automatic Evaluations: • ROUGE– text summary evaluation, calculates recall score of generated sentences BLEU– measures the correspondence between human and machine-generated text

RESEARCH PAPER READING • METEOR– addresses shortcomings of BLEU, uses semantic matching instead of exact lexical matching that BLEU uses, score computation shows how accurate the generated sentence is in comparison to reference sentence • CIDEr– recently introduced metric for image captioning, converts sentences into root forms

THANK YOU!!! ANY QUESTIONS?