Outline Streaming over Internet 1 st Generation Progressive

Outline Streaming over Internet 1 st Generation Progressive 2 nd Generation UDP based Real-Time Protocols (RTP, RTCP, RTSP) • 3 rd Generation TCP based • • – Proprietary – Adobe – Adaptive HTTP based

Challenges § TCP/UDP/IP , best-effort, with no guarantees on expectation or variance of packet delay § Streaming applications delay of 5 to 10 seconds is typical and has been acceptable, but performance deteriorate if links are congested § Real-Time Interactive requirements on delay and its jitter have been satisfied by overprovisioning (providing plenty of bandwidth), what will happen when the load increases? . . .

On-Demand Streaming § Important and growing application due to reduction of storage costs, increase in high speed net access from homes, enhancements to caching § Interactive control by user (but often with long response time) § Ubiquitous on the web: – – You. Tube, Netflix, Vimeo Television networks, Hollywood, etc. Most local radio & TV stations Virtually everywhere on websites 4

First Generation: HTTP Download § A simple architecture is to have the Browser request the object(s) and after their reception pass them to the player for display – No pipelining

Helper Application – i. e. media player § Displays content, which is typically requested via a Web browser; typical functions: – Decompression – Jitter removal – Error correction: use redundant packets to be used for reconstruction of original stream – GUI for user control § Examples: – – – Real. Player Adobe Flash Player Windows Media Player Quick. Time Div. X Web Player

First Gen: HTTP Progressive Download (2) § Alternative: set up connection between server and player; player takes over § Web browser requests and receives a Meta File (a file describing the object) instead of receiving the file itself; § Browser launches the appropriate Player and passes it the Meta File; § Player sets up a TCP connection with Web Server and downloads or streams the file

Meta file requests 8

HTTP Progressive Download § With helper application doing the download, playback can start immediately. . . § Or after sufficient bytes are buffered § Sender sends at maximum possible rate under TCP; retransmit when error is encountered; Player uses a much larger buffer to smooth delivery rate of TCP

HTTP Progressive Download

Buffering Continuous Media § Jitter = variation from ideal timing § Media delivery must have very low jitter – Video frames every 30 ms or so – Audio: ultimately samples need < 128 ms jitter § But network packets have much more jitter that! § Solution: buffers – Fill them with best effort – Drain them via low-latency, local access

Ti m e Buffer Duration "Good" Region: smooth playback Buffer Size oo th Pl ay ba = allowable jitter Sm "Bad": Buffer overrflows ck Max Buffer Duration Max Buffer Size File Position Streaming, Buffers and Timing "Bad": Buffer underflows and playback stops Buffer almost empty Time

The myth of SBR • Any single bitrate you choose for a video is by definition the wrong video for 99% of users. Average Throughput for your Site Visitors 100 90 These users will rebuffer 80 70 These have lower quality than they could sustain 60 50 40 30 20 100 300 500 700 900 1100 1300 1500 1700 1900 2100 2300 2500 2700 2900 3100 3300 3500 3700 3900 4100 4300 4500 4700 4900 5100

Drawbacks of HTTP Progressive Download (2) § HTTP connection keeps data flowing as fast as possible to user's local buffer – May download lots of extra data if you do not watch the video – TCP file transfer can use more bandwidth than necessary § Mismatch between whole file transfer and stop/start/seek playback controls. – However: use file range requests to seek to video position § Cannot change video quality (bit rate) to adapt to network congestion

2 nd Generation: Real-Time Streaming § This gets us around HTTP, allows a choice of UDP vs. TCP and the application layer protocol can be better tailored to Streaming; many enhancements options are possible

Legacy Video Streaming Approaches • RTSP and RTP

Example: Real Time Streaming Protocol (RTSP) • For user to control display: rewind, fast forward, pause, resume, etc… • Out-of-band protocol (uses two connections, one for control messages (Port 554) and one for media stream) • RFC 2326 permits use of either TCP or UDP for the control messages connection, sometimes called the RTSP Channel • As before, meta file is communicated to web browser which then launches the Player; Player sets up an RTSP connection for control messages in addition to the connection for the streaming media

RTSP Operation 18

RTSP Media Stream § Stateful Server keeps track of client's state § Client issues Play, Pause, . . . , Close § Steady stream of packets – UDP - lower latency – TCP - may get through more firewalls, reliable Credit: some content adapted from Alex Zambelli 19

RTSP Exchange Example C: SETUP rtsp: //audio. example. com/xena/audio RTSP/1. 0 Transport: rtp/udp; compression; port=3056; mode=PLAY S: RTSP/1. 0 200 1 OK Session 4231 C: PLAY rtsp: //audio. example. com/xena/audio. en/lofi RTSP/1. 0 Session: 4231 Range: npt=0 (npt = normal play time) C: PAUSE rtsp: //audio. example. com/xena/audio. en/lofi RTSP/1. 0 Session: 4231 Range: npt=37 C: TEARDOWN rtsp: //audio. example. com/xena/audio. en/lofi RTSP/1. 0 Session: 4231 S: 200 3 OK 20

Example 2: RTMP - Real-Time Messaging Protocol § § Proprietary Adobe protocol Runs over TCP Manages audio, video, and other Multiplex multiple streams over TCP connection 21

Flash Streaming Server § Flash Streaming Server communicates with its clients using the Adobe patented RTMP over TCP, which manages a two-way connection, allowing the server to send and receive, video, audio and data between client and server § Typically used with FLV encoded files

Variations of RTMP § RTMP§ § Standard, unencrypted RTMP. The default port is 1935. RTMPT- RTMP “tunneled” over HTTP The default port is 80 RTMPS- RTMP sent over an SSL enables secure TCP/IP connections. The default port is 443 RTMPE- Enhanced and encrypted version of RTMPTE- Encrypts the communication channel, tunneling over HTTP

Drawbacks of RTSP, RTMP § Web downloads are typically cheaper than streaming services offered by CDNs and hosting providers § Streaming often blocked by routers § UDP itself often blocked by firewalls § HTTP delivery can use ordinary proxies and caches § Conclusion: rather than adapt Internet to streaming, adapt media delivery to the Internet 24

Bandwidth is never constant § § Even preselecting bitrates will lead to rebuffering problems as bandwidth fluctuates over the duration of the vide, especially for long form content Ideally we would like to switch bitrates constantly to always give the user the highest quality they can sustain at any point in time. Buffering

§ § Bandwidth is never constant Even preselecting bitrates will lead to rebuffering problems as bandwidth fluctuates over the duration of the vide, especially for long form content Ideally we would like to switch bitrates constantly to always give the user the highest quality they can sustain at any point in time. Adaptive delivery 1500 kbps 1000 kbps Buffering 500 kbps

3 rd Generation: HTTP Streaming § Other terms for similar concepts: Adaptive Streaming, Smooth Streaming, HTTP Chunking § Client-centric architecture with stateful client and stateless server – Standard server: Web servers – Standard Protocol: HTTP – Session state and logic maintained at server § § Video is broken into multiple chunks Chunks begin with keyframe so independent of other chunks A series of HTTP progressive downloads of chunks Playing chunks in sequence gives seamless video

Adaptive Bit Rate with HTTP Streaming § Encode video at different levels of quality/bandwidth § Client can adapt by requesting different sized chunks § Chunks of different bit rates must be synchronized – All encodings have the same chunk boundaries and all chunks start with keyframes, so you can make smooth splices to chunks of higher or lower bit rates

How does segmentation work? 3 1 Incoming video 500 kbps 1000 kbps 2 is split by an encoder 2000 kbps into multiple short blocks. Each block holds the same section of video, encoded at a different size and bitrate.

How does adaptive delivery work? The segmented video is stored on a server, along with a text file which describes the names of each segment. This text file is called a manifest. A player downloads the manifest and then begins requesting individual segments of video. It makes its choice based on bandwidth conditions, grabbing the best quality it can at the time. SERVER CLOUD PLAYER

Media Source Extension – the future of browser video § A W 3 C standard - https: //dvcs. w 3. org/hg/html -media/raw-file/tip/media-source/mediasource. html § Supported today by Chrome, IE 11 and Opera § MSE allows Java. Script to - fetch and parse a manifest - retrieve video, audio and text segments using XHR - feed them to the <video> tag to be decoded and rendered. MSE gives developers incredible control over playback, buffering and adaption logic of in-page video.

Pareto principle in video popularity • Also known as the “ 80 -20” (or 90 -10) rule [http: //en. wikipedia. org/wiki/Pareto_principle] • Associated with highly skewed distributions • Caused by positive-feedback effects (“the rich get richer”) • Does non-UGC content (say movies from Net. Flix) follow a similar pattern? • A practical implication of this skewness: caching applies!

Caching works… • Let’s build CDNs • Take advantage of locality and relativity • A practical implication of this skewness: caching works! • Akamai • You tube • Netflix based on Amazon infrastracture

Adaptive HTTP Streaming System (Protocol) § Server – Can be standard web server – Media segment can be prepared in-line or offline § Client – Sends series of HTTP GET segment requests and receives segments – Performs rate adaptation before sending a new GET segment request

Advantages of HTTP Streaming • Easy to deploy: it's just HTTP, work with existing caches/proxies/CDN/Firewall • Fast startup by downloading lowest quality/smallest chunk • Bitrate switching is seamless • Many small files – Small with respect to the movie size – Large with respect to TCP • 5 -10 seconds of 1 Mbps – 3 Mbps 0. 5 MB – 4 MB per chunk 35

Which adaptive streaming formats exist today? § HTTP Live Streaming (HLS) - Controlled by Apple - Known as HLS, it is supported well by i. OS, Safari and half-heartedly by Android. § HTTP Dynamic Streaming (HDS) - Controlled by Adobe - Known as HDS, it is played back only by Flash clients with custom apps. § Smooth Streaming (SMOOTH) - Controlled by Microsoft - Played back via Silverlight clients, Xbox, § MPEG-DASH (DASH) - An ISO standard - Playback via MSE in browsers, native apps on i. OS, Android and WIn. OS.

Simple HLS example <video id="player" controls poster="sintel. jpg" preload="none" style="width: 480 px; height: 204 px"> <source src="http: //multiplatformf. akamaihd. net/i/multi/april 11/sintelhd_, 512 x 288_450_b, 640 x 360_700_b, 768 x 432_1000_b, . mp 4. csmil/master. m 3 u 8" type='application/x-mpeg. URL' /> <source src="http: //akamaicorppmd. edgesuite. net/flashvod/video/mp 4/sintel. mp 4" type='video/mp 4'/> <source src="http: //akamaicorppmd. edgesuite. net/flashvod/video/mp 4/sintel. webm" type='video/webm' /> </video>

Example of HLS Meta Data

Other Video Streaming Approaches • RTMP

Adaptive video streaming over HTTP �Several players support it �Several commercial content providers use it �Microsoft Smooth Streaming Player �Netflix Player �Adobe OSMF/Zeri Player �Apple Player

HTTP Adaptive Video Streaming

HTTP Adaptive Video Streaming: Manifest File and Fragments

Smooth Streaming Player

Smooth Streaming Player • Sample HTTP Request: – GET /mediadl/iisnet/smoothmedia/Experience/Big. Buck. Bunny 7 20 p. ism/Quality. Levels(2040000)/Fragments(video=400000 000) HTTP/1. 1

Adobe OSMF/Zeri Player

Adobe OSMF Player • Sample HTTP Request: – GET /content/inouteditmbr/inoutedit_h 264_3000 Seg 1 -Frag 5 HTTP/1. 1

What Makes Netflix Interesting? • Commercial, feature-length movies and TV shows § and not free; subscription-based • Nonetheless, Netflix is huge! § § 25 million subscribers and ~20, 000 titles (and growing) consumes 30% of peak-time downstream bandwidth in North America § Maintains only a small “in-house” facility for key functions • A prime example of cloud-sourced architecture § § Majority of functions are sourced to Amazon cloud (EC 2/S 3) § § § e. g. , subscriber management (account creation, payment, …) user authentication, video search, video storage, … DNS service is sourced to Ultra. DNS Leverage multiple CDNs (content-distribution networks) for video delivery § Akamai, Level 3 and Limelight

Netflix Architecture • Netflix has its own “data center” for certain crucial operations (e. g. , user registration, billing, …) • Most web-based user-video interaction, computation/storage operations are cloud-sourced to Amazon AWS • Video delivery is out/cloud-sourced to 3 CDNs • Users need to use MS Silverlight player for video streaming

Netflix Videos and Video Chunks • Netflix uses a numeric ID to identify each movie – IDs are variable length (6 -8 digits): 213530, 1001192, 70221086 – video IDs do not seem to be evenly distributed in the ID space – these video IDs are not used in playback operations • Each movie is encoded in multiple quality levels, each is identified by a numeric ID (9 digits) – various numeric IDs associated with the same movie appear to have no obvious relations 49

Netflix Player

Netflix Player �Sample HTTP Request: �GET /sa 2/946/1876632946. wmv/range/22120592252058? token=1283923056_d 6 f 6112068075 f 1 fb 6 0 cc 48 eab 59 ea 55&random=1799513140 HTTP/1. 1

Netflix Videos and Video Chunks • Videos are divided in “chunks” (of roughly 4 secs), specified using (byte) “range/xxx-xxx? ” in the URL path: Limelight: http: //netflix-094. vo. llnwd. net/s/stor 3/384/534975384. ismv/range/057689? p=58&e=1311456547&h=2 caca 6 fb 4 cc 2 c 522 e 657006 cf 69 d 4 ace Akamai: http: //netflix 094. as. nflximg. com. edgesuite. net/sa 53/384/534975384. ismv /range/057689? token=1311456547_411862 e 41 a 33 dc 93 ee 71 e 2 e 3 b 3 fd 8534 Level 3: http: //nflx. i. ad 483241. x. lcdn. nflximg. com/384/534975384. ismv/range/057689? etime=20110723212907&movie. Hash=094&encoded=06847414 df 0656 e 697 cbd • Netflix uses a version of (MPEG-)DASH for video streaming 52

Terms and Definitions of Adaptive HTTP Streaming § Need – Media Presentation Description (MDP) which provides metadata • For requesting (GET request) media segments • For rate adaptation purpose – Segment which may include media data or metadata to decode § Use DASH an example in the few slides

Example: DASH Client Thomas Stockhammer, Qualcomm, “DASH – Design Principles and Standards , Presentation at MMSys 2011

Meta Data § DASH uses MPD (Media Presentation Descriptor) and Index Information as metadata for DASH Access Client § Initialization and Media Segments for Media Engine – Reuse of existing container format Source: Stockhammer, Qualcomm, “DASH – Design Principles and Standards , Presentation at MMSys 2011

Media Presentation Data Model MDP - description of accessible segments and corresponding timing Source: Stockhammer, Qualcomm, “DASH – Design Principles and Standards , Presentation at MMSys 2011