Video Summarization Video Highlights Extraction Video Browsing Video

Video Summarization Video Highlights Extraction Video Browsing Video Representation Multimodal Analysis Video Retrieval

content access data flow Video Summarization Highlights based Summarization Video Browsing Table of Contents based Summarization Video Representation Top-down Multimodal Analysis Video Content Bottom-up Video Retrieval

Highlight Groups Grouping Highlight Candidates Audio-visual markers association A highlight Audio-Visual Markers Key audio-visual object detection Visual Marker Audio Marker Play / Break Feature extraction & segmentation Pla y Break Video with Audio Track

Browsing To. C based Summarization Highlights based Summarization Retrieval To. C Index Highlights Highlight Groups Scenes Visual Groups Semantic Highlight Candidates Shots Audio-Visual Markers Key frames Camera motion Play/Break

Video Audio Markers Detection Visual Markers Detection Applause Cheers Soccer Goal Post Baseball Catcher Excited Speech Golfer Bending to Hit A-V Marker Negotiation Which sport is it? Highlight Candidates Finer Resolution Highlights Golf Swings Golf Putts Non-highlights Strikes, Balls Non-highlights Ball-hits Corner Kicks Penalty Kicks Non-highlights

Soccer video markers Baseball video marker Golf video markers

Audio Class Recognition (GMM) Time Domain Input Audio Frequency Domain Classify the input audio into one of 5 Audio Markers using GMM Applause Feature Extraction Cheering Music MFCC Coefficients. Training Audio Clips Feature Extraction Comparing Likelihoods Speech Excited Speech Audio Class Training the GMM Classifiers using MFCC features and BIC.

Audio Class Recognition (GMM) Time Domain Input Audio Frequency Domain Classify the input audio into one of 2 Audio Markers using GMM Excited Speech Feature Extraction Other MFCC Coefficients. Training Audio Clips Feature Extraction Comparing Likelihoods (Applause, Cheering, Music, Speech) Task = Sports Highlights Audio Class Training the GMM Classifiers using MFCC features and CV

Audio Class Recognition (GMM) Time Domain Input Audio Frequency Domain Classify the input audio into one of 5 Audio Markers using GMM Applause Feature Extraction Cheering Music MFCC Coefficients. Training Audio Clips Feature Extraction Comparing Likelihoods Speech Excited Speech Audio Class Training the GMM Classifiers using MFCC features and BIC.

MDCTs Input Audio Feature Extraction Task Importance Level Class Label Audio Classifier Importance Level Calculation

Generic Audio Classification Applause Cheering Music MDCTs Speech Excited Speech Training Audio Clips Feature Extraction Training Data for GMMs Compare Likelihoods Class Label

MDCTs Input Audio Feature Extraction Task Importance Level Class Label Audio Classifier Importance Level Calculation

Task Specific Audio Classification Excited Speech Other MDCTs Compare Likelihoods Class Label (Applause, Cheering, Music, Speech) Training Audio Clips Feature Extraction Task = Sports Highlights

Filter 1 Filter 2 Filter 3 Goal post view 1 Filter 3 Filter 2 Goal post view 2 Soccer Models Interpretation Filter 1 Filter 2 Catcher Model Interpretation Filter 1 Filter 2 Golfer view 2 Golf Models Interpretation

Pla y Break Video clip A-V markers catcher bat swing cheer excited speech Index