Project Create a DASHcompliant Dynamic Adaptive Streaming over

Project Create a DASH-compliant (Dynamic Adaptive Streaming over HTTP) streaming system NUS. SOC. CS 5248 -2012 Roger Zimmermann

Goals (1) n Capture a video on an ASUS Transformer tablet computer and store it as an MP 4 file. n Split the MP 4 file into streamlets, i. e. , 3 second long video files. n Upload the streamlets to a web server. n Transcode the streamlets into 3 different streamlets (e. g. , low, medium, high quality). n Create a playlist on the web server. NUS. SOC. CS 5248 -2012 Roger Zimmermann

Goals (2) n ASUS Transformer runs the Android 4. 0 Ice Cream Sandwich OS. n Programming on Android is done in Java with the Eclipse IDE. n On the web server, create scripts in the PHP language. n Implement a simple Android DASH media player. NUS. SOC. CS 5248 -2012 Roger Zimmermann

Project Homepage n Descriptions and web links n Some utilities and some library source codes n Documentation (RFCs, etc. ) n IVLE Forums n TA: Wang Guanfeng NUS. SOC. CS 5248 -2012 Roger Zimmermann

Advice and Actions (1) n Form a team (3 persons). n Note: You will need to read and learn a lot. Your program code will be quite small. n HTTP POST command structure n MP 4 Parser usage to create streamlets n FFmpeg transcoder usage n Playlist. m 3 u 8 format in XML n Start early (i. e. , this week)! NUS. SOC. CS 5248 -2012 Roger Zimmermann

Actions (2): Get your Tablet n Check out your loan ASUS Transformer for the project from So. C Technical Services (on the first floor of COM 1). n There is one tablet per team (3 students). n Please make an appointment with Mr. Chow Chin Ming to get your tablet. Email: chowcm@comp. nus. edu. sg. n Tell Mr. Chow the 3 team member names. NUS. SOC. CS 5248 -2012 Roger Zimmermann

Introduction to DASH Dynamic Adaptive Streaming over HTTP NUS. SOC. CS 5248 -2012 Roger Zimmermann

DASH (1) n RTP/RTSP/RTCP streaming faces several challenges n Special-purpose server for media (complex) n Protocols use TCP and UDP transmissions (firewalls) n Difficult to cache data (no “web caching”) n Advantage n Short end-to-end latency NUS. SOC. CS 5248 -2012 Roger Zimmermann

DASH (2) n Main idea of DASH n Use HTTP protocol to “stream” media n Divide media into small chunks, i. e. , streamlets n Advantages n Server is simple, i. e. , regular web server n No firewall problems (use port 80 for HTTP) n Standard (image) web caching works NUS. SOC. CS 5248 -2012 Roger Zimmermann

DASH (3) n Original DASH implementation by Move Networks n Introduced concept of streamlets n Additional idea: make playback adaptive n NUS. SOC. CS 5248 -2012 Roger Zimmermann Encode media into multiple different streamlet files, e. g. , a low, medium, and high quality version (different bandwidth)

DASH (4) MPD: Media Presentation Description n ISO/IEC Standard: n “Information technology — MPEG systems technologies — Part 6: Dynamic adaptive streaming over HTTP (DASH)” n JTC 1/SC 29; FCD 23001 -6 NUS. SOC. CS 5248 -2012 Roger Zimmermann

DASH (5) n Web server provides a playlist n The playlist is a file in a specific format that lists all the available qualities and all the streamlets for each quality n Playlist file extension is. m 3 u 8 n Content preparation: n Original media file needs to be split into streamlets n Streamlets need to be transcoded into different qualities NUS. SOC. CS 5248 -2012 Roger Zimmermann

DASH (6) n HTTP protocol is stateless! n Server remembers “nothing” about session n Scheduling logic, etc. , is in media player! NUS. SOC. CS 5248 -2012 Roger Zimmermann

DASH (7) n DASH media player n Loads. m 3 u 8 file and then starts to download streamlets n All the scheduling logic is in the player n Render current streamlet while downloading the next streamlet before playback is done n Measure bandwidth and switch between different qualities (i. e. , adapt) n Switch servers can be done easily NUS. SOC. CS 5248 -2012 Roger Zimmermann

DASH (8) n Many media players now understand DASH streaming format n Many companies use HTTP streaming: n Move Networks, Apple, Microsoft, Netflix, … n CDNs like this approach n No need to run Quick. Time, Windows Media, Real. Networks, and Flash streaming servers n Just use web server for everything! NUS. SOC. CS 5248 -2012 Roger Zimmermann

Continuous Media Servers n Introduction n Continuous Media n Magnetic Disk Drives n Display of CM (single disk, multi-disks ) n Optimization Techniques n Additional Issues n Case Study (Yima)

What is a CM Server? Network Storage Manager Memory n Multiple streams of audio and video should be delivered to many users simultaneously.

Some Applications n Video-on-demand n Medical databases n News-on-demand n NASA databases n News-editing n Movie-editing n Interactive TV n Digital libraries n Distance Learning

Continuous Display n Data should be transferred from the storage device to the memory (or display) at a pre-specified rate. n Otherwise: frequent disruptions & delays, termed hiccups. n NTSC quality: 270 Mb/s uncompressed; 3 -8 Mb/s compressed (MPEG-2). Memory Disk

Challenge: Real-Time Media § Bandwidth requirements for different media types: 100 Mb/s 50 Mb/s 31 Mb/s 4 -6 Mb/s 1 Mb/s 20 Mb/s

High Bandwidth & Large Size n HDTV quality ~ 1. 4 Gb/s Uncompressed! Standard: SMPTE 292 M n 2 -hr HDTV ~ 1260 GB

Streaming Media Servers n Streaming media servers require a different “engine” than traditional databases because of: n Real-time retrieval and storage n Large media objects n The performance metrics for streaming media servers are: n The number of simultaneous displays: throughput N The amount of time that elapses until a display starts: n The overall cost of the system: cost per stream, C n startup latency L

Media Types n Examples of continuous media are: Audio n Video n Haptics n Continuous media are often compressed. There are many different compression algorithms, for example: n Motion Picture Experts Group: MPEG-1, MPEG-2, MPEG-4 n Joint Photographic Expert Group: Motion-JPEG n Digital Video: DV, Mini. DV n Microsoft Video 9, Div. X, … Others: n MP 3: MPEG-1 layer 3 audio – Wavelet-based codecs n Above codecs are based on – Lossless compression discrete cosine transform (DCT) n

Compression n MPEG-1 180: 1 reduction in both size and bandwidth requirement (SMPTE 259 M, NTSC 270 Mb/s is reduced to 1. 5 Mb/s). n MPEG-2 30: 1 to 60: 1 reduction. (NTSC ~ 4, DVD ~ 8, HDTV ~ 20 Mb/s) n Problem: loose information (cannot be tolerated by some applications: medical, NASA)

Media Characteristics n Data requires a specific bandwidth: n Constant bitrate (CBR) CM n Variable bitrate (VBR) CM n Easier case: CBR n Data is partitioned into equi-sized blocks which represent a certain display time of the media n E. g. : 176, 400 bytes represent 1 second of playtime for CD audio (44, 100 samples per second, stereo, 16 -bits per sample)

Assumed Hardware Platform n Multiple magnetic disk drives: n n Not too expensive (as compared to RAM) Not too slow (as compared to tape) Not too small (as compared to CD-ROM) And it’s already everywhere! Memory

Magnetic Disk Drives n An electro-mechanical random access storage device n Magnetic head(s) read and write data from/to the disk Drive Internals

Disk Device Comparison

Disk Seek Characteristic

Disk Seek Time Model If d < z cylinders If d >= z cylinders

Disk Service Time n The disk service time is dependent on several factors: n Seek time n Platter diameter (e. g. , 3. 5”, 2. 5”, 1”) n Rotational latency n Spindle speed n Data transfer time n Zone-bit recording n Read versus write bandwidth

Disk Service Time Model – – – TTransfer: data transfer time [s] TAvg. Rot. Latency: average rotational latency [s] TService: service time [s] B: block size [MB] BWEffective: effective bandwidth [MB/s]

Data Retrieval Overhead

Sample Calculations • Assumptions: – TSeek = 10 ms – BWMax = 20 MB/s – Spindle speed: 10, 000 rpm

Summary n Average rotational latency depends on the spindle speed of the disk platters (rpm). n Seek time is a non-linear function of the number of cylinders traversed. n Average rotational latency + seek time = overhead (wasteful). n Average rotational latency and seek time reduce the maximum bandwidth of a disk drive to the effective bandwidth

Continuous Display (1 disk) Retrieve from disk Display from memory X 1 X 2 Display X 1 X 3 Display X 2 Display X 3 Time n n n Traditional production/consumption problem RC = Consumption Rate, e. g. , MPEG-1: 1. 5 Mb/s. RD = Production Rate, Seagate Cheetah X 15: 40 -55 MB/s. For now: RC < RD Partition video X into n blocks: X 1, X 2, . . . , Xn (to reduce the buffer requirement)

Retrieve from Disk Display from Memory X 1 Seek Time Round-robin Display Y 3 Display X 1 Y 4 X 2 Display Y 3 Display Y 4 Time n Time period: time to display a block (is fixed). n System Throughput (N): number of streams. n Assuming random assignment of the blocks: n n n X 3 Maximum seek time between block retrievals Waste of disk bandwidth ==> lower throughput Tp=? , N=? , Memory=? , max-latency=? Y 5 Display X 3 Display Y 5

Cycle-based Display Retrieve from Disk X 1 Display from Memory Y 3 Z 5 Z 6 Y 4 X 2 Y 5 X 3 Z 7 Display X 1, Y 3, Z 5 Display X 2, Y 4, Z 6 Time n Using disk scheduling techniques n Less seek time ==> Less disk bandwidth waste ==> Higher throughput n Larger buffer requirement Tp=? , N=? , Memory=? , max-latency=? n

Group Sweeping Schema (GSS) Group 1 W 1 X 1 Subcycle 1 Group 2 Y 3 Z 5 X 2 Z 6 Y 4 W 3 X 3 Subcycle 2 Display X 1, W 1 n n n W 2 Display X 2, W 2 Can shuffle order of blocks retrievals within a group Cannot shuffle the order of groups GSS when g=1 is cycle-based GSS when g=N is round-robin Optimal value of g can be determined to minimize memory buffer requirements n Tp=? , N=? , Memory=? , max-latency=? Z 7 Y 5

System Issues n Movie is cut into equi-sized blocks: X 0, X 1, …, Xn-1. n Time required to display one block is called time period Tp. n Note: Tp is usually longer than the disk retrieval time of a block; this allows multiplexing of a disk among different displays. Server Retrieval Network Buffer Display X 0 X 1 X 0 Time X 2 X 1 X 0 Time period Buffer empty X 1 X 2 Hiccup

Constrained Data Placement n Partition the disk into R regions. n During each time period only blocks reside in the same region are retrieved. n Maximum seek time is reduced almost by a factor of R. n Introduce startup latency time n Tp=? , N=? , Memory=? , max-latency=?

Hybrid n For the blocks retrieved within a region, use GSS schema. n This is the most general approach; Tp=? , N=? , Memory=? , max-latency=? By varying R and g all the possible display techniques can be achieved. Round-robin (R=1, g=N). Cycle-based (R=1, g=1). Constrained placement (R>0, g=1), . . . A configuration planner calculates the optimal values of R & g for certain application. n n n

Display of Mix of Media Retrieve from Disk X 1 Y 3 Z 5 Display from Memory Z 6 Y 4 X 2 Y 5 X 3 Z 7 Display X 1, Y 3, Z 5 Display X 2, Y 4, Z 6 Time n Mix of media types: different RC’s: audio, video; e. g. : Rc(Y) < Rc(X) < Rc(Z) n Different block sizes: Rc(X)/B(X)=Rc(Y)/B(Y)=. . . n Display time of a block (time period) is still fixed.

Multiple-disks n Single disk: even in the best case with 0 seek time, 240/1. 5 = 160 MPEG-1 streams. n Typical applications (MOD): 1000’s of streams. n Solution: aggregate bandwidth and storage space of multiple disk drives. n How to place a video? Memory

RAID Striping n All disks take part in the X 1 transmission of a block. n Can be conceptualized as a single disk. n Even distribution of display load. d 1 d 2 d 3 n Efficient admission. n Is not scalable in throughput. X 1. 1 X 2. 1 X 1. 2 X 2. 2 X 1. 3 X 2. 3

Round-robin Retrieval d 1 d 2 d 3 n Only a single disk takes part in the transmission of each block. n Retrieval schedule n X 1 Y 1 Z 3 W 3 Retrieval Schedule d 1 d 2 d 3 Round-robin retrieval of the blocks. n Even distribution of Time display load. n Efficient admission. n Not scalable in latency. W 2 X 2 Y 2 Z 1 X 3 Y 3 Z 2 W 1 Display X 1, Y 1, W 1, Z 1 X 2, Y 2, W 2, Z 2 X 3, Y 3, W 3, Z 3

Hybrid Striping n Partition D disks into clusters of d disks. n Each block is declustered across the d disks that constitute a cluster (each cluster is a logical disk drive). n RAID striping within a cluster. n Round-robin retrieval across the clusters. n RAID striping (d=D), Round-robin retrieval (d=1).

Introduction to Yima PE Personal Edition Streaming Media System NUS. SOC. CS 5248 -2012 Roger Zimmermann

Overview n Command line server n GUI client n “Split” utility to prepare media files n RTSP communication (port 5 xxxx) NUS. SOC. CS 5248 -2012 Roger Zimmermann #. /yimaserver <Yima. PE 1. 0> begin scheduler <Yima. PE 1. 0> begin rtsps

Software Source n Directories n Server n Client n Splitter n Streams Server code Client code and GUI library Media preparation utility Sample media (WAV file) n Remove all object files (*. o) before building the executables NUS. SOC. CS 5248 -2012 Roger Zimmermann

Yima PE Server n RTSP front and backend (one process) n Scheduler + FLIB (one process) n Qpthread v 1. 3. 1 library for multi-threading n Must set LP_LIBRARY_PATH to include Qpthread n Server configuration file: config n n Where are the media files located Name, size [bytes] and duration [sec] NUS. SOC. CS 5248 -2012 Roger Zimmermann

Splitter n Input: yimaintro. wav (for example) n Output: BLOCKS sub-directory n Data block files: yimaintro. wav_1, yimaintro. wav_2, … n Each block is 256, 000 bytes and contains 500 RTP packets (of 512 bytes each) n A sample config file is created; must copy contents to the main config file NUS. SOC. CS 5248 -2012 Roger Zimmermann

Server + Splitter n Server does not care about block contents, i. e. , it does not know what kind of media data is stored (MPEG-1/2, WAVE, …) n Server sends RTP packets based on config info: n BW = size / duration n Packet-level scheduling n Need only modify splitter for MP 3 media! NUS. SOC. CS 5248 -2012 Roger Zimmermann

Linux Client n Operation: n [List] button: reads media entries from local Yima. cfg file n [Play], [Pause], [Stop] buttons execute RTSP commands to server n GUI was built with XForms library; it is message-driven, with callback functions for buttons, etc. Plays uncompressed audio (PCM). NUS. SOC. CS 5248 -2012 Roger Zimmermann

Windows Client n Operation: n [List] button: reads media entries from local Yima. cfg file n [Play], [Pause], [Stop] buttons execute RTSP commands to server n GUI was built with Visual Studio C/C++ (MFC library); it is message-driven, with callback functions for buttons. Includes MP 3 decoder. NUS. SOC. CS 5248 -2012 Roger Zimmermann

Client Structure n 3 threads Player “P” n State machine GUI “C” Buffer Network “N” Command Message Queues, e. g. , put_cmd(Cto. N, …); NUS. SOC. CS 5248 -2012 Roger Zimmermann /dev/dsp RTP RTSP