Bringing Internet Video to Prime Time Aditya Ganjam

Bringing Internet Video to Prime Time Aditya Ganjam Vice President of Engineering Conviva

Background Aditya Ganjam – Vice President of Engineering at Conviva Startup founded by Hui Zhang (Prof. at CMU) and Ion Stoica in 2006 Conviva optimizes video quality for premium content properties such as HBO, ESPN, ABC, Disney, and Turner through network wide real-time visibility and real-time actions. Conviva's technology has powered some of the world's largest on-line events such as Olympics, FIFA World Cup, NCAA College Basketball March Madness, Major League Baseball, and Academy Awards.

2005: Beginning of the Internet Video Era Launch of You. Tube 100 M streams first year First popular mobile video device Premium Sports Webcast on Line Live 8 concert online

2006 – 2011: Internet Video Going Prime Time 2006 2007 2008 2009 2010 2011

2011 Internet Traffic Distribution IPTV VOD Internet Video P 2 P Video Calling Web, email, data File transfer Online gaming Vo. IP Business Internet Business Intranet 66% Internet Traffic is Video Source: Akamai

2011 and Beyond: A World Full of Elephants What Does It Mean For the Internet If 95% Traffic is Video? Video (100 x traffic growth) Other Applications (10 x traffic growth) 2011 2016

Macro Changes in Internet Video Technology and Business Technology enablers in place Broadband penetration Standard software & hardware platforms Real business model emerging Premium content (ESPN, HBO) Advertising & subscription (mlb. com, netflix, Hulu) Online audiences rival broadcast for major events Olympics, Inauguration, Michael Jackson, World Cup Convergence of TV, Internet, Mobile Internet connected TVs (Xbox, Playstation, Apple. TV, Roku, Sony, Samsung) Internet connected smart mobile devices (i. Pad, i. Phone, Android) “TV Everywhere” over the top (HBO, ESPN, Turner, Comcast)

This Talk Delivering high quality video over the Internet and Conviva’s approach to addressing this challenge Three sections … Understanding today’s Internet video eco-system State of the art of Internet video quality A strategy for delivering high quality video

Understanding Today’s Internet Video Eco-system

Internet Video Ecosystem : Video Data-plane Key components: Video Player & CDN Video Source Screen Video Player Encoders & Video Servers CMS and Hosting ISP & Home Net Content Delivery Networks (CDN)

How a CDN works … - A CDN is a large distributed content cache acting as an overlay multicast Content Delivery Networks network - Data flows from the content origin through mid-tier servers to edge servers - Content is cached along the way to achieve scale - Many CDNs use DNS to abstract away the complexity of server selection Content Origin DNS Edge Servers Video Player - Major CDNs are Akamai, Limelight, and Level 3 - Many ISPs are also building their own CDNs

Video Uses CDNs A Little Differently Live streaming Multi-bitrate streaming Multi-CDN streaming Screen Video Player 500 Kbps 1 Mbps 2 Mbps 800 Kbps 1. 5 Mbps 3 Mbps

Player & Device Eco-System PC Mobile Game Consoles Set-top Boxes Significant adoption, especially Flash Rapid growth in market share Rapid growth and significant market share Moderate growth Connected TVs Moderate growth, but will pick up quickly

Sophistication of the Video Player 3 rd Targeting – ie. Party Ad Networks CMS CDN 9 10 1 2 Tokenization / License server 5 4 3 11 Video Player Ad Proxy 6 7 12 8 Video CDN Ad CDN 13 CDN Verification 14

What is the quality of Internet video today ? What is high video quality? Prevent video startup failures Start the video quickly Play the video smoothly and without interruptions Play the video at the highest bit rate possible What is the best way to measure Internet video quality? Claim: Collecting statistics from the video player is the best way to measure video quality Reason 1: The video player interacts with multiple services owned by multiple companies and is the only single point that has state across all interactions Reason 2: With multi-bit rate and multi-CDN technologies, a single server or CDN does not have the complete quality information for a client

Video Player Monitoring : Player Model Join. Time (JT) Buffering. Ratio(BR) Rate. Of. Buffering(RB) Avg. Bitrate(AB) Rendering. Quality(RQ)

Player Application Video Player Monitoring : Data Collection • Content Manager Messaging & Serialization UI Controller Player Insight Streaming Module Automatic Monitoring HTTPS Automatic and consistent monitoring of default streaming modules – – – Flash: Net. Stream, Video. Element Silverlight: Media. Element, Smooth. Streaming. Media. Element i. OS: MPMovie. Player. Controller To backend

Video Player Monitoring : Cross-platform Challenges. Platform Independent Platform Specific Language Translator C# Smooth. Streaming, … Comm. Timer Storage Silverlight Action. Script Net. Stream, … Comm. Timer Storage Flash Java. Script <video/>, <audio/> Comm. Timer Storage HTML 5 Objective C MPMovie. Player. Controller Comm. Timer Storage i. OS C++ • • • Consistency in metric computation across languages and platforms Benefit from stronger type checking from C# Readable output: preserve comments, white space, formatting Provides control for language-specific fragments Compiles real code and unit tests

Video Player Monitoring : Data Model

State of The Art of Internet Video Quality

We’ve seen patterns across many sites … and billions of streams

Example Video Site Quality Summary 31. 68% of views had quality issues Total Views video did not start Total Views = 66, 44, 79, 19 Total Viewers = 3, 291, 204 Total Minutes Viewed = 290, 260, 395 0. 68 5. 7 32. 2% of viewers had recurring quality issues Viewers with good quality watched 1. 5 X more video than viewers with poor quality Good video quality for all viewers can add 10. 9% more minutes of viewed video Good video quality can add $120 K more revenue per month* * Assumes $30 CPM ad every 8 minutes Minutes per Viewer 25. 3 video buffered 62. 2 100. 9 video had low resolution 0 50 Poor Quality Viewers 100 Good Quality Viewers 68. 3 video had good quality 150 32. 2 67. 8

Example Video Site Quality Summary 31. 68% of views had quality issues Total Views video did not start Total Views = 66, 44, 79, 19 Total Viewers = 3, 291, 204 Total Minutes Viewed = 290, 260, 395 0. 68 5. 7 32. 2% of viewers had recurring quality issues Viewers with good quality watched 1. 5 X more video than viewers with poor quality Good video quality for all viewers can add 10. 9% more minutes of viewed video Good video quality can add $120 K more revenue per month* Minutes per Viewer 25. 3 video buffered 62. 2 100. 9 video had low resolution 0 50 100 Poor Quality Viewers 68. 3 video had good quality 150 Good Quality Viewers 50 40 30 20 10 0 1 2 3 4 5 6 7 8 9 Poor experience on 9 different sites * Assumes $30 CPM ad every 8 minutes

Opportunity of Going Higher Speed Untapped Brand Enhancement and Viewer Experience <100 100 -300 300 -500 500 -700 Actual Rate Consumed Actual. Bit Rate Consumed Available Bandwidth 700 -1000 -1600 -2200 -3000 >3000

Viewers Watch Longer When Video is Not Interrupted by Buffering Live Vo. D Concert TV 1. com 261% more Impacted Avg Min 0 3 9 12 15 18 21 24 27 Avg Min 30 0 3 6 9 12 15 18 21 24 27 30 TV 2. com Sports 191% more Jan 2011 32 min Impacted 0 3 9 12 17 min Good Views Impacted 11 min 6 31% more Jan 2011 Good Views Avg Min 16 min Impacted 6 min 6 27 min Good Views 22 min Good Views 69% more 15 18 21 24 27 30 Avg Min 0 3 13 min 6 9 12 15 18 21 24 27 30

Viewers Return More When Video Is Not Interrupted by Buffering 1% difference in buffering between two ISPs 68% monthly loss in uniques for ISP with poor performance Even 1% increase in buffering leads to more than 60% loss in audience

Engagement vs Join Time Join time is critical for user retention

Engagement vs Buffering Ratio 1% increase in buffering reduces engagement by 3 minutes

CDNs Vary in Performance over Geographies and Time 25% 50% • Used one month aggregated data-set • Considered 31, 744 DMAASN-hours with > 100 views in each CDN There is no single best CDN across geographies, networks, and time

0. 0% Washington DC (Hagerstown): CMCS(33657) Milwaukee: ROADRUNNER-CENTRAL(20231) Green Bay - Appleton: SCRR-7015(7017) Denver: ASN-QWEST(209) Charlotte: SCRR-11426(11426) Washington DC (Hagerstown): ASN-CXA-ALL-CCI-22773 -RDC(227. . . Philadelphia: VZGNI-TRANSIT(19262) San Diego: SBIS-AS(7132) Las Vegas: ASN-CXA-LV-13432 -CBS(13432) Madison: CHARTER-NET-HKY-NC(20115) Indianapolis: SBIS-AS(7132) Providence - New Bedford: ASN-CXA-ALL-CCI-22773 -RDC(22773) Washington DC (Hagerstown): VZGNI-TRANSIT(19262) Hartford & New Haven: SBIS-AS(7132) Houston: SBIS-AS(7132) Grand Rapids - Kalamazoo - Battle Creek: SBIS-AS(7132) Atlanta: BELLSOUTH-NET-BLK(6389) Honolulu: HAWAIIAN-TELCOM(36149) Atlanta: COMCAST-7725(7725) Washington DC (Hagerstown): SPCS(10507) Orlando - Daytona Beach - Melbourne: EMBARQ-WNPK(2379) Denver: CMCS(33652) Norfolk - Portsmouth - Newport News: ASN-CXA-ALL-CCI-22773 -. . . Saint Louis: CHARTER-NET-HKY-NC(20115) Cincinnati: FUSE-NET(6181) Phoenix: ASN-QWEST(209) Dallas - Fort Worth: VZGNI-TRANSIT(19262) Pittsburgh: CCCH-3(7016) Saint Louis: SBIS-AS(7132) San Francisco - Oakland - San Jose: SBIS-AS(7132) San Diego: ROADRUNNER-WEST(20001) Greenville - Spartansburg - Asheville - Anderson: BELLSOUTH-NE. . . Kansas City: SBIS-AS(7132) Columbus - OH: SCRR-10796(10796) Louisville: INSIGHT-COMMUNICATIONS-CORP-AS 1(36727) Chicago: ATT-INTERNET 3(6478) Kansas City: ASN-CXA-ALL-CCI-22773 -RDC(22773) Miami - Fort Lauderdale: BELLSOUTH-NET-BLK(6389) Cleveland: NEO-RR-COM(11060) Chicago: VZGNI-TRANSIT(19262) Columbia - SC: SCRR-11426(11426) Dallas - Fort Worth: SBIS-AS(7132) Detroit: SBIS-AS(7132) Kansas City: SCRR-11955(11955) Los Angeles: CHARTER-NET-HKY-NC(20115) Miami - Fort Lauderdale: COMCAST-20214(20214) Cleveland: SCRR-10796(10796) West Palm Beach - Fort Pierce: COMCAST-20214(20214) San Diego: ASN-CXA-ALL-CCI-22773 -RDC(22773) Seattle - Tacoma: ASN-QWEST(209) New York: CABLE-NET-1(6128) Portland - OR: ASN-QWEST(209) Oklahoma City: SBIS-AS(7132) Phoenix: ASN-CXA-ALL-CCI-22773 -RDC(22773) Chicago: SBIS-AS(7132) Greensboro - High Point - Winston-Salem: SCRR-11426(11426) Albany - Schenectady - Troy: RR-NYSREGION-ASN-01(11351) Tyler - Longview (Lufkin & Nacogdoches): SUDDENLINK-COMMU. . . Grand Rapids - Kalamazoo - Battle Creek: CMCS(33668) Houston: CMCS(33662) Hartford & New Haven: COMCAST-7015(7015) Eugene: ASN-QWEST(209) San Francisco - Oakland - San Jose: CMCS(33651) CDNs Vary in Performance over Geographies and Time 100. 0% 90. 0% 80. 0% 70. 0% 60. 0% 50. 0% 40. 0% 30. 0% 20. 0% 10. 0%

CDN Streaming Failures Are Common Events

Summary of Results On most sites 15 -30% of viewers do not get an uninterrupted high quality stream Quality has a substantial impact on viewer engagement Buffering ratio is most critical across genres (for live event: 1% increase in buffering reduces 3 min play time) Join time impacts engagement at viewer level CDN performance varies minute by minute and region by region

A Strategy for Delivering High Quality Video Over the Internet

Refresh: What is high quality? Prevent startup failures Start the video quickly Play the video smoothly and without interruptions Play the video at the highest bit rate possible

Three Concepts for High Quality Video Delivery Continuous measurement and optimization using a control infrastructure decoupled from the delivery infrastructure Multi-bit rate streams delivered using multiple CDNs Optimization algorithms based on individual client and aggregate statistics working at multiple time scales

One-time DNS Re-direction vs Continuous Optimization CDN Select a server intelligently at start time May still incur connection or streaming failure or missing asset Continuous optimization Select the best CDN to successfully access the asset without any failure • Continuously monitor video quality from client side • Use global audience based diagnostics to predict quality issues • Switch bit-rates or sources

Possible Optimization Architecture Real-time Alerts Real-time Global Data Aggregation and Correlation (Streaming Map-reduce) Real-time and historical Insights Historical Data Aggregation and Analysis (Hadoop+Hive+Spark) Localize issues by region, network, CDN, and time DMA Inference Engine DMA ASN ASN Level 3 DMA Akamai Global Inference, Decision & Policy Engine Limelight Real-time global optimizations Decision Engine Bit Rates Time of day CDNs Continuous real-time measurements from every client Optimize viewer performance by selecting the best option within the set of bit rates and CDNs

Akamai 100 80 ASN 60 40 2000 4000 6000 8000 Peak Concurrent Viewers 10000 12000 14000 16000 18000 20000 Network Saturation Point Unsaturated Network 40 35 30 25 20 15 10 5 0 Limelight Level 3 0 5000 10000 15000 20000 25000 30000 35000 Geo ASN 0 Bandwidth Fluctuation Geo ASN Bandwidth Fluctuation Example Optimization Using Aggregate Statistics Peak Concurrent Viewers This ASN/Geo is saturated on all three CDNs Don’t switch CDN. Reduce bit rates and maintain

Conviva Optimization in the Wild 25. 00% Reduced views impacted by buffering from 16. 13% to 5. 56% … 2800 Average Bit Rate Views Impacted by Buffering 20. 00% 15. 00% 10. 00% 5. 00% 0. 00% 2600 2400 2200 2000 1800 1600 … and raised engagement by 36% … increased average bit-rate from 1. 7 Mbps to 2. 1 Mbps…

Concluding Remarks All indications show that we are in the middle of a key transition of main-stream video to the Internet Video quality presents opportunity and challenge Follow the traffic: 60% Internet traffic today, will be more than 95% in the next 2 -3 years Premium video will be consumed by lean-back experience on big screens zero tolerance for poor quality Video player continuous monitoring and optimization driven by player-level and global algorithms has the best chance of delivering high quality video