Introduction to DASH Dynamic Adaptive Streaming over HTTP

Introduction to DASH Dynamic Adaptive Streaming over HTTP NUS. SOC. CS 5248 -2017 Roger Zimmermann

Dynamic Adaptive Streaming over HTTP (DASH) Christian Timmerer and Christopher Müller Alpen-Adria Universität Klagenfurt (AAU) Faculty of Technical Sciences (TEWI) Institute of Information Technology (ITEC) Multimedia Communication (MMC) http: //research. timmerer. com http: //blog. timmerer. com mailto: christian. timmerer@itec. uni-klu. ac. at 02 May 2011 Acknowledgment: Thomas Stockhammer (QUALCOMM), Mark Watson (Netflix) – reused their presentations from MMSys’ 11 accessible via http: //www. mmsys. org/

User Frustration in Internet Video • Video not accessible – – – • Low Quality of Experience Behind a firewall Plugin not available Bandwidth not sufficient Wrong/non-trusted device Wrong format • Fragmentation – Devices – Content Formats – DRMs – – – Long start-up delay Frequent Re-buffering Low playback quality No lip-sync No DVD quality (language, subtitle) • Expensive – Sucks my bandwidth – Need a dedicated devices – Other costs… One way to build confidence ➪ Open Standards Ack & ©: Thomas Stockhammer Christian Timmerer, Alpen-Adria-Universität 2010/05/02 Klagenfurt, Austria 3

What is DASH? http: //en. wikipedia. org/wiki/Dash_(disambiguation) 2010/05/02 Christian Timmerer, Alpen-Adria-Universität Klagenfurt, Austria 4

HTTP Streaming of Media Server Client 1 MF DF ISOBM FF M 2 TS 2010/05/02 MF 2 easy conversion DF easy conversion ISOBMFF … ISO Base Media File Format (e. g. , mp 4 – others: avi) M 2 TS … MPEG-2 Transport Stream (e. g. , DVB, DMB) MF … Manifest Format (e. g. , MPD, FMF) DF … Delivery Format (e. g. , F 4 F, 3 gs) Christian Timmerer, Alpen-Adria-Universität Klagenfurt, Austria ISOBM FF M 2 TS 5

Adaptive Streaming in Practice Ack & ©: Mark Watson 2010/05/02 Christian Timmerer, Alpen-Adria-Universität Klagenfurt, Austria 6

Dynamic Adaptive Streaming over HTTP (DASH) Proprietary Solutions Apple HTTP Live Streaming Adobe HTTP Dynamic Streaming Microsoft Smooth Streaming Netflix Int’l Standard Solutions V 1 3 GPP Rel. 9 Adaptive HTTP Streaming OIPF HTTP Adaptive Streaming Int’l Standard Solutions V 2 3 GPP Rel. 10 DASH MPEG DASH Akamai Movenetworks’ Movestreaming Amazon . . . time http: //multimediacommunication. blogspot. com/2010/05/http-streaming-of-mpeg-media. html 2010/05/02 Christian Timmerer, Alpen-Adria-Universität Klagenfurt, Austria 7

Outline • Introduction – DASH Design Principles – Scope: What is specified – and what is not! • DASH Data Model – Media Presentation Description – Segment Indexing • Dynamic & Adaptive – Video on Demand vs. Live – The Adaptation Problem • Conclusions & Future Work 2010/05/02 Christian Timmerer, Alpen-Adria-Universität Klagenfurt, Austria 8

DASH Design Principles • DASH is not – system, protocol, presentation, codec, interactivity, client specification • DASH is an enabler – It provides formats to enable efficient and high-quality delivery of streaming services over the Internet – It is considered as one component in an end-to-end service – System definition left to other organizations (SDOs, Fora, Companies, etc. ) • Design choices – Enable reuse of existing technologies (containers, codecs, DRM etc. ) – Enable deployment on top of HTTP-CDNs (Web Infrastructures, caching) – Enable very high user-experience (low start-up, no rebuffering, trick modes) – Enable selection based on network and device capability, user preferences – Enable seamless switching – Enable live and DVD-kind of experiences – Move intelligence from network to client, enable client differentiation – Enable deployment flexibility (e. g. , live, on-demand, time-shift viewing) – Provide simple interoperability points (profiles) Ack & ©: Thomas Stockhammer Christian Timmerer, Alpen-Adria-Universität 2010/05/02 Klagenfurt, Austria 9

What is specified – and what is not? Ack & ©: Thomas Stockhammer Christian Timmerer, Alpen-Adria-Universität 2010/05/02 Klagenfurt, Austria 10

DASH Data Model Segment Info Initialization Segment http: //www. e. com/ahs-5. 3 gp Media Presentation Period, start=0 s Media Segment 1 Period, • start=100 • base. URL=http: //www. e. com/ … … Representation 1 Period, start=100 s … 100 kbit/s … start=0 s http: //www. e. com/ahs-5 -1. 3 gs • bandwidth=500 kbit/s • width 640, height 480 Media Segment 2 … Segment Info 500 kbit/s Representation 2 Period, start=295 s Representation 1 duration=10 s Template: . /ahs-5 -$Index$. 3 gs start=10 s http: //www. e. com/ahs-5 -2. 3 gs Media Segment 3 start=20 s http: //www. e. com/ahs-5 -3. 3 gh … … Media Segment 20 start=190 s Ack & ©: Thomas Stockhammer 2010/05/02 http: //www. e. com/ahs-5 -20. 3 gs Christian Timmerer, Alpen-Adria-Universität Klagenfurt, Austria 11

Media Presentation Description • Redundant information of Media Streams for the purpose to initially select or reject Groups or Representations – Examples: Codec, DRM, language, resolution, bandwidth • Access and Timing Information HTTP-URL(s) and byte range for each accessible Segment Earliest next update of the MPD on the server Segment availability start and end time in wall-clock time Approximated media start time and duration of a Media Segment in the media presentation timeline – For live service, instructions on starting playout such that media segments will be available in time for smooth playout in the future – – • Switching and splicing relationships across Representations • Relatively little other information Ack & ©: Thomas Stockhammer 2010/05/02 Christian Timmerer, Alpen-Adria-Universität Klagenfurt, Austria 12

DASH Groups & Subsets Group id="grp-1" Representation id="rep-2" . . . Representation id="rep-n" Group by codec, language, resolution, bandwidth, views, etc. – very flexible (in combination with xlink)! § Ranges for the @bandwidth, @height and @frame. Rate Subset id="ss-1" Contains group="grp-1" Group id="grp-2" Contains group="grp-4" Representation id="rep-1" Contains group="grp-7" Representation id="rep-2" . . . Representation id="rep-n" . . . Group id="grp-m" Representation id="rep-1" 2010/05/02 Subsets § Mechanism to restrict the combination of active Groups § Expresses the intention of the creator of the Media Presentation Christian Timmerer, Alpen-Adria-Universität Klagenfurt, Austria Representation id="rep-2" 13

Segment Indexing • Provides binary information in ISO box structure on – Accessible units of data in a media segment – Each unit is described by • Byte range in the segments (easy access through HTTP partial GET) • Accurate presentation duration (seamless switching) • Presence of representation access positions, e. g. IDR frames • Provides a compact bitrate-over-time profile to client – Can be used for intelligent request scheduling • Generic Data Structure usable for any media segment format, e. g. ISO BMFF, MPEG-2 TS, etc. • Hierarchical structuring for efficient access • May be combined with media segment or may be separate Ack & ©: Thomas Stockhammer 2010/05/02 Christian Timmerer, Alpen-Adria-Universität Klagenfurt, Austria 14

Segment Indexing Segment Index in MPD only <MPD>. . . <URL source. URL="seg 1. mp 4"/> <URL source. URL="seg 2. mp 4"/> </MPD> seg 1. mp 4 seg 2. mp 4. . . <MPD>. . . <URL source. URL="seg. mp 4" range="0 -499"/> <URL source. URL="seg. mp 4" range="500 -999"/> </MPD> seg. mp 4 Segment Index in MPD + Segment <MPD>. . . <Index source. URL="idx. mp 4"/> <URL source. URL="seg. mp 4"/> </MPD> idx. mp 4 Segment Index in Segment only <MPD>. . . <Base. URL>seg. mp 4</Base. URL> idx </MPD> Christian Timmerer, Alpen-Adria-Universität 2010/05/02 Klagenfurt, Austria seg. mp 4 15

Adaptive Streaming Summary • For on demand • – Chunks are unnecessary and costly – Byte range requests have caching and flexibility advantages – Separate audio/video essential for language support • For live For both – Chunks are unavoidable – Still value in decoupling request size from chunk size – Multiple language audio tracks are rare – May need manifest updates – Switch point alignment required for most CE decoding pipelines Ack & ©: Mark Watson and Thomas Stockhammer Christian Timmerer, Alpen-Adria-Universität 2010/05/02 Klagenfurt, Austria 17

Adaptation Problem Choose sequence and timing of requests to minimize probability of re-buffers and maximize quality Ack & ©: Mark Watson 2010/05/02 Christian Timmerer, Alpen-Adria-Universität Klagenfurt, Austria 18

Conclusions • Asynchronous delivery of the same content to many users is a first-class network service – HTTP CDNs may not be the “perfect” architecture, but it’s working pretty well at scale • Many variations on HTTP Adaptive Streaming theme in deployed systems and emerging standards – DASH provides sufficient flexibility here • DASH is rich and simple at the same time – Understand more detailed market needs – Create profiles as considered necessary – Collaborate with system creators on how to integrate DASH Ack & ©: Mark Watson and Thomas Stockhammer 2010/05/02 Christian Timmerer, Alpen-Adria-Universität Klagenfurt, Austria 19

Potential Future Work Items • MMSys’ 11 Keynote – HTTP Adaptive Streaming in Practice by Mark Watson (Netflix) – Future work • Good models for future bandwidth • Tractable representations of future choices - how to efficiently search the 'choice space’ • What are the quality goals? • Call for adaptation logics – Efficient implementations of the actual adaptation logic which is responsible for the dynamic and adaptive part of DASH http: //multimediacommunication. blogspot. com/2011/02/beta-version-of-vlc-dash-plugin. html • Get it deployed and adopted (e. g. W 3 C, DVB – what is necessary? ) • Join this activity, everyone is invited – get involved in and exited about DASH! 2010/05/02 Christian Timmerer, Alpen-Adria-Universität Klagenfurt, Austria 20

Implementations • Reference Software – Open Source, ISO Copyright – Currently not publicly available • GPAC Implementation – GNU Lesser General Public License – http: //gpac. wp. institut-telecom. fr/ • VLC Plugin – GNU Lesser General Public License – http: //www-itec. uni-klu. ac. at/dash/ 2010/05/02 Christian Timmerer, Alpen-Adria-Universität Klagenfurt, Austria 21

Thank you for your attention . . . questions, comments, etc. are welcome … Ass. -Prof. Dipl. -Ing. Dr. Christian Timmerer Klagenfurt University, Department of Information Technology (ITEC) Universitätsstrasse 65 -67, A-9020 Klagenfurt, AUSTRIA christian. timmerer@itec. uni-klu. ac. at http: //research. timmerer. com/ Tel: +43/463/2700 3621 Fax: +43/463/2700 3699 © Copyright: Christian Timmerer 2010/05/02 Christian Timmerer, Alpen-Adria-Universität Klagenfurt, Austria 22

Adaptation Schemes ABR – Adaptive Bitrate Adaptation NUS. SOC. CS 5248 -2017 Roger Zimmermann

Dynamic Adaptive Video Streaming over HTTP (DASH) Abdelhak Bentaleb bentaleb@comp. nus. edu. sg Supervisor Prof. Roger Zimmermann School of Computing National University of Singapore

Introduction 25 Real-time Entertainment Streaming video and audio, More than 65% of Internet traffic at peak periods Popular Services You. Tube (16. 9%), Netflix (34. 9%), Amazon Video (2. 95%), Hulu (2. 5%) All delivered over the top (OTT) Network operators are looking for novel ways to optimally utilize the resources What happens when multiple client compete with each other? Source: Global Internet Phenomena Report (DEC 2015) Sandvine Source: Trends and Analysis Report (May 2015) CISCO

Push and Pull-Based video delivery source 26 Traditional Video Streaming: Pushbased Initiate, manage and control a video streaming session between the client and server Adaptive HTTP Video Streaming (HAS): Pull-based Uses HTTP and TCP to fetch data Push-Based Delivery Pull-Based Delivery Source (Server) Broadcasters/serv ers like: Windows Media, Apple Quick. Time, Real. Networks Helix, Cisco VDS/DCM Web/FTP servers such as: LAMP, Microsoft IIS, Adobe Flash, Real. Networks Helix, Cisco VDS Protocols RTSP, RTP, UDP HTTP, RTMPx, FTP Video Monitoring RTCP for RTP and User Tracking transport (Currently) Proprietry Multicast Support Yes No Caching Support No Yes for HTTP

What is Streaming? 27 Definition Streaming is transmission of a continuous content from a server to a client. Simultaneous consumption by the client. Two Main Characteristics Client consumption rate may be limited by real-time constraints as opposed to just bandwidth availability. Server transmission rate (loosely or tightly) matches to client consumption rate.

HAS Video Delivery Benefits 28 Dynamic Adaptation to the network condition. Reuse of existing Internet infrastructure. Adaptation logic located at the client side. HTTP protocol: simplifies getting through NATs and firewalls.

30 HTTP Dynamic Adaptive video Streaming (DASH) ü Request the MPD ü Download segments using HTTP GET ü Fragment the video into small fixed duration (2 to 10 s) segments ü Encode and store segments at different bitrate and resolution levels ü select bitrate based on TCP bandwidth ü List the encoded segments in a Media estimation Presentation Description (MPD) ü Adapt to network resources dynamically ü Send segments using HTTP POST

31 HTTP Dynamic Adaptive video Streaming (DASH) Adaptation to dynamic network conditions Adapts to dynamic conditions anywhere on the path through the Internet and/or home network. Decrease continual connections between S and C’s while increase scalability of clients. Improved end-user Quality of Experience (Qo. E) Enables faster start-up and seeking. Reduces freezes. Use of HTTP/TCP Provides easy traversal for all kinds of middleboxes. Enables cloud access, leverages existing HTTP caching infrastructure (Cheaper CDN costs). HTTP TCP

Overview 32

Client-side Bitrate (BR) Adaptation 33 The bitrate adaptation logic is fully controlled by the client (purely client-driven). Bitrate adaptation heuristics based on: 1. Available bandwidth 2. Playback buffer size 3. Chunk scheduling 4. Hybrid-based Interesting algorithms Li et al (2014), Liu et al (2011) Huang et al (2014), Mueller et al (2015) Jiang et al (2012), Chen et al (2013) Yin et al (2015), Li et al. (2014), De Cicco et al. (2013), Miller et al. (2012), Zhou et al. (2012)

Adaptation Comparison (1) 34 • “A Comparison of Quality Scheduling in Commercial Adaptive HTTP Streaming Solutions on a 3 G Network” Haakon Riiser, Håkon S. Bergsaker, Paul Vigmostad, Pål Halvorsen, Carsten Griwodz, ACM Mo. Vid '12, proceedings of the 4 th Workshop on Mobile Video, pp. 25 -30. • This paper compares four commuter scenarios:

Adaptation Comparison (2) 35 Netview

1. Available Bandwidth BR Adaptation 36 Uses the available bandwidth estimation as an heuristic in the bitrate adaptation logic algorithm. Interesting algorithms: Li et al. (2014), Liu et al. (2011) Challenges and drawbacks: Blind available bandwidth sharing between competing clients q Distribute the available bandwidth equally between clients q Each client strives to fetch the high chunk bitrate => unfairness, congestion Heterogeneous devices with various capabilities case? The bandwidth estimation is not accurate DASH scalability issues q Lack of a central manager May suffer from buffer starvation and undesirable Qo. E Worst decisions when number of clients increase

2. Buffer Level Size BR Adaptation 37 Uses the current buffer level size as an criterion in the bitrate adaptation logic algorithm. Interesting algorithms: Huang et al. (2014), Mueller et al. (2015) Challenges and drawbacks: Suffer from frequent bitrate switch with a low perpetual quality q Can not deal with rapid and/or sudden bandwidth variations q Low end-user Qo. E Very slow detection very slow detection, eliminate the available bandwidth estimation Can not support many clients

3. Chunk Scheduling BR Adaptation 38 Divides the bitrate adaptation algorithm into many components and uses scheduling approach to select a suitable bitrate level. Interesting algorithms: Jiang et al. (2012), Chen et al. (2013) Challenges and drawbacks: Is exposed to instability issue when the number of players increases q Inaccurate bandwidth estimation Ignored responsiveness to bandwidth fluctuations q Buffer undershoot issue and stalls Achieves unpleasant end-user Qo. E Hard to implement in a real word without any central control manager that schedules bitrate decisions for each client

4. Hybrid BR Adaptation 39 The client makes its bitrate selection based on combination between available bandwidth and buffer level heuristics. Interesting algorithms: Yin et al. (2015), Li et al. (2014), De Cicco et al. (2013), Miller et al. (2012), Zhou et al. (2012) Challenges and drawbacks: Supports few number of clients Avoid just one or two of scalability issues q e. g. , consistent quality without taking into account fair share of bandwidth Lack of optimal bitrate decisions q Does not consider any metric of user satisfaction q Suffers from video instability and stalls q Achieves a poor end-user Qo. E

Server-side Bitrate Adaptation 40 The bitrate adaptation logic is fully controlled by the server (purely server-driven) Uses traffic shaping mechanism to assign the bitrate for each client, q e. g. , based on some pricing rules Not requiring any cooperation from the client Like traditional video streaming systems Some interesting algorithms Akhshabi et al. (2013) Houdaille and Gouache (2012) De Cicco et al. (2011) bps K 0 60 0 Kbps 30 900 Kbps 1000 Kbp s Traffic Shaper DASH Server DASH client

SVC-based Bitrate Adaptation 41 Each segment can be split into a subset of bit-streams. The client selects the base layer of the lower quality and increases the quality by downloading more enhancement layers. Downloading more layers is based on network conditions. Interesting algorithms Kreuzberger et al. (2015), Sieber et al. (2013), Andelin et al. (2012)

SDN-based Bitrate Adaptation 42 Software defined networking is used. Network resources control and monitoring capabilities simplify/rapidity of network resource programming and deployment Avoid the purely client-driven bitrate adaptation issues The bitrate adaptation logic is controlled, monitored and assisted by a central coordinator called SDN controller. Qo. E improvement => interaction between a SDN controller and DASH client All Interesting algorithms: Georgopoulos et al. (2013), Farshad et al. (2015), Arefin et al. (2013), Nam et al. (2014), Wang et al. (2014), Ferguson et al. (2013), Bari et al. (2013), Gorlatch et al. (2015), Gudipati et al. (2013), Yap et al. (2013)

In-network based Bitrate Adaptation 43 Bitrate adaptation logic is based on in-network decisions. In-network process needs a special component. q Agent and/or proxy deployed in the network q Offer the required information that allow bitrate adaptation algorithm to use efficiently the network resources. Interesting algorithms Mok et al. (2012), Eckert and Knoll (2013), Bouten et al. (2015), Petrangeli et al. (2015), Thomas et al. (2015), Joseph and de Veciana (2014), El Essaili et al. (2013)

Commercial Solution Bitrate Adaptation 44 Microsoft smooth player (MSS) Apple HTTP Live streaming player (HLS) Current available bandwidth, device capabilities as heuristics during the bitrate adaptation logic process. Adobe OSMF Current available bandwidth, playback windows resolution and CPU load as heuristics for bitrate adaptation logic. Adapts to the network variations based on the available bandwidth and device processing capabilities. Akamai HD Adapts to the network variations based on the available bandwidth and CPU load.

Comparison 45

46