CS 414 Multimedia Systems Design Lecture 14 Introduction

CS 414 – Multimedia Systems Design Lecture 14 – Introduction to Multimedia Resource Management Klara Nahrstedt Spring 2012 CS 414 - Spring 2012

Administrative Reading: “Multimedia Systems”, Steinmetz and Nahrstedt, Springer 2004, Chapter 2 n HW 1 posted on Wednesday, February 22. n ¨ HW 1 due on Thursday, March 1. CS 414 - Spring 2012

Outline AV Requirements - Real-time n AV Requirements on Multimedia Communication and Operating Systems n Resource management n ¨ Resources ¨ Quality of Service (Qo. S) Concept ¨ Operations CS 414 - Spring 2012

Integrating Aspects of Multimedia Image/Video Capture Audio/Video Perception/ Playback Audio/Video Presentation Playback Image/Video Information Representation Transmission Audio Capture Transmission Compression Processing Audio Information Representation Media Server Storage CS 414 - Spring 2012 A/V Playback

Integrating Aspects of Multimedia Image/Video Capture Audio/Video Perception/ Playback Audio/Video Presentation Playback Image/Video Information Representation Transmission Audio Capture Transmission Compression Processing Audio Information Representation Media Server Storage CS 414 - Spring 2012 A/V Playback

Integrating Aspects of Multimedia Image/Video Capture Audio/Video Perception/ Playback Audio/Video Presentation Playback Image/Video Information Representation Transmission Audio Capture Transmission Compression Processing Audio Information Representation Media Server Storage CS 414 - Spring 2012 A/V Playback

For Next Four Weeks we will cover Transmission/Networks of Multimedia Image/Video Capture Audio/Video Perception/ Playback Audio/Video Presentation Playback Image/Video Information Representation Transmission Audio Capture Transmission Compression Processing Audio Information Representation Media Server Storage CS 414 - Spring 2012 A/V Playback

Multimedia System/Network Sender/Server MM Application OS/Distributed Systems/Network Receiver/Client Capture AV Code AV Display AV Decode AV Stream AV, Sync AV, Schedule AV, Queue/Buffer AV, Shape AV, Manage AV, Route AV, Retrieve AV, Pre-fetch/Cache AV, Record AV Network CS 414 - Spring 2012 MM Application OS/DS/Network

AV Requirements: Real-Time and Deadlines n Real-time system – system in which correctness of computation depends not only on obtaining the right results, but also providing them on time ¨ Examples: control of temperature in a chemical plant; control of a flight simulator n Deadline – represents the latest acceptable time for the result delivery ¨ Soft deadlines versus hard deadlines CS 414 - Spring 2012

AV Requirements: Real-Time and Multimedia n Difference between RT requirements for traditional RT systems and Multimedia systems ¨ Soft deadlines versus hard deadlines ¨ Periodic behavior versus random behavior ¨ Bandwidth requirements CS 414 - Spring 2012

Layered Partition of Multimedia Systems with respect to Required Resources and Individual Services CS 414 - Spring 2012

AV Requirements on MM Systems and Networks Transport system – guaranteed delivery with respect to metrics such as delay, reliability, bandwidth requirements n OS process management – real-time processing of continuous data, communication and synchronization between processes/ threads n CS 414 - Spring 2012

AV Requirements on MM Systems and Networks (2) Memory/Buffer management – guaranteed timing delay and efficient data manipulation n File system/Media Servers – transparent and guaranteed continuous retrieval of audio/video n Device management – integration of audio and video n CS 414 - Spring 2012

Result of AV Requirements n Need Resource Management to coordinate ¨ Transport/Network Resources, ¨ CPU/OS Resources ¨ Memory/Buffer Resources ¨ Storage/Disk Resources ¨ Device Resources CS 414 - Spring 2012

Resource Management (Why do we need resource management? ) Limited capacity in digital distributed systems despite data compression and usage of new technologies n Need adherence for processing of continuous data by every hardware and software component along the data path n Competition for resources exist in an integrated multimedia system n CS 414 - Spring 2012

Window of Resources Requirements Interactive HDTV-quality multi-view video Sufficient To abundant insufficient HDTV High-quality Audio sufficient Insufficient Sufficient But scarce Sufficient but Scarce to Sufficient insufficient abundant insufficient Sufficient Network File access Sufficient But scarce 1980 1990 2000 CS 414 - Spring 2012 abundant 2010 2020 Hardware support

Quality of Service (How to parameterize services? ) n To manage resources, we need services over resources ¨ n to schedule AV data, to shape access for AV data, to process AV data, to move AV data, etc. Multimedia systems consist of set of AV-specific services Processing (media-related) services: retrieve audio/video, record video/audio, compress audio/video, fast forward video, rewind video ¨ Transport (network) services: Stream video, fast forward video, rewind video ¨ n To provide multimedia services, services get parameterized with quality levels called Quality of Service CS 414 - Spring 2012

Examples of A/V Qo. S Parameters n Qo. S for Audio service: n n n Sample rate – 8000 samples/second (8 KHz), 44. 1 KHz Sample resolution – 8 bits per sample, 16 bits per sample Qo. S for Video service: n n n Video frame rate – 25 frames per second, 30 frames per second Frame Period – 40 ms, 30 ms, 25 ms, … Frame resolution – 320 x 240 pixels, 640 x 480 pixels, 1920 x 1080 pixels, … Pixel resolution – 24 bits per pixel, 8 bits per pixel Frame size – 64 KB Compression rate – 8: 1 CS 414 - Spring 2012

Examples of Network Qo. S n Qo. S for network service: n Throughput – Number of bits per second e. g. , 100 Mbps (Ethernet throughput) – level 1 ¨ e. g. , 10 Mbps (Wi. Fi throughput) – level 2 ¨ e. g. , 128 kbps (ISDN throughput) – level 3 ¨ n Connection setup time – time how long it take to connect the sender and receiver ¨ n e. g. , 50 ms, 10 ms, … End-to-End Delay – time interval from the time packet is sent from the sender until the time it is received at the receiver (Treceive – Tsend) ¨ e. g. , 80 ms, 100 ms, 160 ms CS 414 - Spring 2012

Layered Model for Qo. S Quality of Experience Quality of Service CS 414 - Spring 2012

Application Qo. S Parameters CS 414 - Spring 2012

System Qo. S Parameters CS 414 - Spring 2012

Network Qo. S Parameters CS 414 - Spring 2012

Qo. S Classes n Guaranteed Service Class ¨ Qo. S guarantees are provided based on deterministic and statistical Qo. S parameters n Predictive Service Class ¨ Qo. S parameter values are estimated and based on the past behavior of the service n Best Effort Service Class ¨ There are no guarantees or only partial guarantees are provided CS 414 - Spring 2012

Qo. S Classes (cont. ) Qo. S Class determines: (a) reliability of offered Qo. S, (b) utilization of resources CS 414 - Spring 2012

Deterministic Qo. S Parameters Single Value: Qo. S 1 – average (Qo. Save), contractual value, threshold value, target value • • • Throughput – 10 Mbps Pair Value: <Qo. S 1, Qo. S 2> with Qo. S 1 – required value; Qo. S 2 – desired value <Qo. Savg, Qo. Speak>; <Qo. Smin, Qo. Smax> • Throughput - <8, 12> Mbps CS 414 - Spring 2012

Deterministic Qo. S Parameter Values n Triple of Values <Qo. S 1, Qo. S 2, Qo. S 3> ¨ Qo. S 1 – best value ¨ Qo. S 2 – average value ¨ Qo. S 3 – worst value n Example: ¨ <Qo. Speak, Qo. Savg, Qo. Smin>, where Qo. S is network bandwidth ¨ Throughput <12, 10, 8> Mbps CS 414 - Spring 2012

Guaranteed Qo. S n n We need to provide 100% guarantees for Qo. S values (hard guarantees) or very close to 100% (soft guarantees) Current Qo. S calculation and resource allocation are based on: 1. 2. Hard upper bounds for imposed workloads Worst case assumptions about system behavior Advantages: Qo. S guarantees are satisfied even in the worst case (high reliability in guarantees) Disadvantage: Over-reservation of resources, hence needless rejection of requests CS 414 - Spring 2012

Predictive Qo. S Parameters n We utilize Qo. S values (Qo. S 1, . . Qo. Si) and compute average ¨ Qo. Sbound n We utilize Qo. S values (Qo. S 1, , Qo. Si) and compute maximum value ¨ Qo. SK n step at K>i is Qo. SK = 1/i*∑j. Qo. Sj = max j=1, …i (Qo. Sj) We utilize Qo. S values (Qo. S 1, , Qo. Si) and compute minimum value ¨ Qo. SK = min j=1, …i (Qo. Sj) CS 414 - Spring 2012

Best Effort Qo. S No Qo. S bounds or possible very weak Qo. S bounds n Advantages: resource capacities can be statistically multiplexed, hence more processing requests can be granted n Disadvantages: Qo. S may be temporally violated n CS 414 - Spring 2012

Relation between Qo. S and Resources Reservation Admission Translation, Scaling, Negotiation Scheduling, Rate Control Flow Control, Congestion Control, Adaptation CS 414 - Spring 2012

Conclusion Qo. S – an important concept in multimedia systems n Very different types of Qo. S parameters and values n Important relation between Qo. S and Resources n Need to understand operations on Qo. S and their impact on resource management n CS 414 - Spring 2012