CS 414 Multimedia Systems Design Lecture 33 Media

CS 414 – Multimedia Systems Design Lecture 33 – Media Server (Part 2) Klara Nahrstedt Spring 2014 CS 414 - Spring 2014

Administrative MP 3 posted, April 15, 2014 n MP 3 deadline May 3, 5 pm (Saturday) n MP 3 presentations n ¨ n Monday, May 5, 5 -7 pm On Friday, April 18 – MP 3 discussion ¨ You are encouraged to bring your own design !!! - possibility to discuss your design with Shannon CS 414 - Spring 2014

Covered Aspects of Multimedia Image/Video Capture Audio/Video Perception/ Playback Audio/Video Presentation Playback Image/Video Information Representation Transmission Audio Capture Transmission Compression Processing Audio Information Representation Media Server Storage CS 414 - Spring 2014 A/V Playback

Video Server Flickr n Flickr – image and video hosting website n In November 2007 ¨ Flickr hosted 2 Billion Photos In August 2009, ¨ Flickr hosted 62 databases across 124 servers In September 2010, ¨ Fickr hosted more than 5 billion images ¨ Developed by Ludicorp, Vancouver, 2004, now owned by Yahoo! June 2011 ¨ 51 Million registered members In August 2011 ¨ Flickr hosted 6 billion photos n n CS 414 - Spring 2014

Outline Media Server Disk Scheduling and Admission Control n Media Server File System Issues n CS 414 - Spring 2014

Media Server Architecture Delivered data Incoming request Network Attachment Content Directory Memory Management File System Storage management (Scheduling Policy) Disk controller (Scheduling Mechanism) Storage device CS 414 - Spring 2014

EDF (Earliest Deadline First) Disk Scheduling n Each disk block request is tagged with deadline ¨ Very n good scheduling policy for periodic requests Policy: ¨ Schedule disk block request with earliest deadline ¨ Excessive seek time – high overhead ¨ Pure EDF must be adapted or combined with file system strategies CS 414 - Spring 2014

EDF Example Note: Consider that block number Implicitly encapsulates the disk track number CS 414 - Spring 2014

Elevator (SCAN) n Method Take the closest request in the direction of travel ¨ Real implementations do not go to the end (called LOOK) ¨ n 53 199 Pros ¨ n 0 Bounded time for each request Cons ¨ Request at the other end will take a while Arriving Requests in Request Queue 98, 183, 37, 122, 14, 124, 65, 67 Served Request at Disk Controller (37, 14, 0, 65, 67, 98, 122, 124, 183)

SCAN-EDF Scheduling Algorithm Combination of SCAN and EDF algorithms n Each disk block request tagged with augmented deadline n ¨ Add n to each deadline perturbation Policy: ¨ SCAN-EDF chooses the earliest deadline ¨ If requests with same deadline, then choose request according to scan direction CS 414 - Spring 2014

Implementation of SCAN-EDF n Notation: ¨ Di be deadline of disk block request ‘i’ ¨ Ni be track (block) position on disk ¨ Nmax be maximum number of disk tracks n Deadline Modification: ¨ Di + f(Ni) ¨ f(Ni) converts track number of ‘i’ into a small perturbation of deadline ¨ Perturbation small enough so that n n Di + f(Ni) ≤ Dj + f(Nj) for Di ≤ Dj Possible f(Ni) = Ni/Nmax CS 414 - Spring 2014

SCAN EDF Example (Nmax = 100) CS 414 - Spring 2014

Admission Control Client 1 retrieves K 1 blocks in one round Client 2 retrieves K 2 blocks Server Client 3 retrieves K 3 blocks Client 4 retrieves K 4 blocks CS 414 - Spring 2014

Admission Control n Disk block requests are timed ¨ Media n n n server must determine admit a stream serve (schedule) a stream without having negative effect on other streams already serviced. Deterministic Guarantees Admission control considers worst case scenario when admitting new stream ¨ Constrained Disk Placement Example: M - size of blocks, G – size of gabs, rdt – data transfer of disk ¨ CS 414 - Spring 2014

Media Server Architecture Delivered data Incoming request Network Attachment Content Directory Memory Management File System Storage management Disk controller Storage device CS 414 - Spring 2014

Multimedia File System File Placement n File allocation tables/Index tables n Additional File System Operations n ¨ Fast forward ¨ Rewind n Block sizes CS 414 - Spring 2014

Multimedia File Systems n Real-time Characteristics ¨ Read operation must be executed before well-defined deadline with small jitter n n Additional buffers smooth data File Size ¨ Can be very large even those compressed ¨ Files larger than 232 bytes n Multiple Correlated Data Streams ¨ Retrieval of a movie requires processing and synch of audio and video streams CS 414 - Spring 2014

Placement of Multiple MM Files on Single Disk n n n Popularity concept among multimedia content very important Take popularity into account when placing movies on disk Model of popularity distribution – Zipf’s Law ¨ Movies n if their probability of customer usage is C/k, ¨ n are kth ranked C = normalization factor Condition holds: C/1 + C/2 + … C/N = 1, ¨ N is number of customers CS 414 - Spring 2014

Example Assume N = 5 movies n Problem: what is the probability that the next customer picks 3 rd ranked movie? n Solution: n ¨ Solve n C from the equation C/1 + C/2 + C/3 + C/4 + C/5 = 1 ¨C = 0. 437 ¨ Probability to pick 3 rd ranked movie is C/3 = 0. 437/3 = 0. 1456 CS 414 - Spring 2014

Placement Algorithm for Multiple Files on Single Disk n Organ-Pipe Algorithms (Grossman and Silverman 1973) 1 rank (most popular movie) st 3 rd 2 nd ranked movie 4 th 5 th 6 th 7 th 8 th 9 th Middle of disk (in case of traditional disk layout) Note: In case of ZBR disk layout , place most popular disks at the outer tracks CS 414 - Spring 2014

Placement of Mapping Tables Fundamental Issue: keep track of which disk blocks belong to each file (I-nodes in UNIX) n For continuous files/contiguous placement n ¨ don’t n need maps For scattered files ¨ Need maps Linked lists (inefficient for multimedia files) n File allocation tables (FAT) n CS 414 - Spring 2014

Indexing and FAT Higher Level Index Table Per File I Frame P Frame B Frame P Frame ………. . File Allocation Table Block I 1 Location PTR Block I 2 Location PTR Block I 3 Location PTR Block P 11 Location PTR Block P 12 Location PTR Block B 1 Location PTR Block P 22 Location PTR …………. . CS 414 - Spring 2014

Constant and Real-time Retrieval of MM Data n n Retrieve index in real-time Retrieve block information from FAT Retrieve data from disk in real-time Real-time playback ¨ n Implement linked list Random seek (Fast Forward, Rewind) ¨ Implement n indexing MM File Maps ¨ include metadata about MM objects: creator of video, sync info CS 414 - Spring 2014

Fast Forward and Rewind (Implementation) n Play back media at higher rate ¨ n Not practical solution Continue playback at normal rate, but skip frames Define skip steps, e. g. skip every 3 rd, or 5 th frame ¨ Be careful about interdependencies within MPEG frames ¨ n Approaches for FF: Create a separate and highly compressed file ¨ Categorize each frame as relevant or irrelevant ¨ Intelligent arrangement of blocks for FF ¨ CS 414 - Spring 2014

Block Size Issues in File Organization n Small Block Sizes ¨ n n Use smaller block sizes, smaller than average frame size Organization Strategy: Constant Time Length Need Metadata structure, called Frame Index ¨ Frame means a time frame within a movie ¨ Under the time frame read all blocks (audio, video, text) belonging to this time frame ……… A V V A V T V CS 414 - Spring 2014 Frame index Movie Time line

Block Size Issues n Large Block Size ¨ Use large blocks (e. g. , 256 KB) which include multiple audio/video/text frames n n Organization Strategy: Constant Data Length Need Metadata structure, called Block Index ¨ Each block contains multiple movie frames Block Index A V A V V V CS 414 - Spring 2014

Tradeoffs n n n Frame index : needs large RAM usage while movie is playing, however little disk wastage Block index (if frames are not split across blocks): need low RAM usage, but major disk wastage – internal disk fragmentation Block index(if frames are split across blocks): need low Ram usage, no disk wastage, extra seek times CS 414 - Spring 2014

Conclusion Designers of VOD systems strive to achieve low access latency for customers n Challenges: n ¨ Handle large amount of customers (clients) ¨ Maintain low cost of operation ¨ Provide acceptable access latency CS 414 - Spring 2014