Querybased video summarization UCF REU 2016 Jacob Laurel
Query-based video summarization UCF REU 2016 Jacob Laurel Graduate Student Mentor: Aidean Sharghi Professor: Dr. Boqing Gong
Overview �Create fully summarized data set for egocentric videos for nearly 50 queries �Implement Baseline detection algorithm on dataset that uses a Determinantal Point Process to automatically generate diverse summaries
Weekly Progress �Finally obtained results from data set �As hypothesized, annotations displayed very good inter-user agreement. �Shot Detection was unsuccessful, likely due to quality of video frames
Intersection over Union for measuring inter-user agreement/consistency �
35 30 0 1 52 103 154 205 256 307 358 409 460 511 562 613 664 715 766 817 868 919 970 1021 1072 1123 1174 1225 1276 1327 1378 1429 1480 1531 1582 1633 1684 1735 1786 1837 1888 1939 1990 2041 2092 2143 How many annotations for a given image? Total Number of distinct annotations per image (NO redundancies) 25 20 15 Series 1 10 5
4500 0 Baby Beach Bed Blonde Book Building Car Chair Chocolate Computer Cup/glass Dance Desk Drink Exercise Face Flower Food Garden Grass Hall Hands Hat Kids Lady Lake Market Men Musical instrument Ocean Office Park Party Pets/animal Phone Room School Shoes Sign Sky Sports Street Student Sun Toy Tree Window Histogram for the Dataset Frequency of Selected Annotations 4000 3500 3000 2500 2000 1500 Series 1 1000 500
Shot Detection Attempts �Used ffmpeg for first attempt and a Chisquared color histogram frame comparison for second attempt �Despite varying the selectivity of the shot criteria, ffmpeg produced too many isolated shots (overly selective) �Open. CV implementation of Chi-squared Histogram comparison inaccurately classified most scene fades
Shot Detection Results Histogram-based method falsely detected a shot due to lighting change FFmpeg falsely detected shot due to blurry motion
Next Steps �Now that Annotations are acceptable, we need to proceed to gathering the user-created summarizations �Refine baseline algorithm upon obtaining the summarizations
- Slides: 9