CS 598 KN Advanced Multimedia Systems Lecture Multimedia

CS 598 KN – Advanced Multimedia Systems Lecture – Multimedia OS Klara Nahrstedt Fall 2018 CS 598 KN - Fall 2018

Covered Aspects of Multimedia OS in each multimedia device Audio/Video Presentation Audio/Video Playback Perception/ Playback Image/Video Capture Image/Video Information Representation Transmission Audio Capture Transmission Compression Processing Audio Information Representation Media Server Storage CS 598 KN - Fall 2018 A/V Playback

CPU Process/Thread Scheduling Maintain many programs in memory n Determine how to best schedule them n Requires information about the processes and an understanding of the system goals n Switch among processes rapidly so each user perceives direct ownership of system n CS 598 KN - Fall 2018

CPU Scheduling Evaluation Criteria n Throughput: processes completed per unit time Turnaround: from submission to termination Wait: time waiting in the ready queue Response: from user request to system response Utilization: how busy the CPU is over time n Want predictable system behavior n n CS 598 KN - Fall 2018

Scheduling Algorithms FCFS n Shortest Job First n Priority scheduling n Round-robin n Multi-level queue n For General Purpose Process CPU Scheduling Rate monotonic n Earliest deadline first n CS 598 KN - Fall 2018 For RT/Multimedia Process CPU Scheduling

Real-time/Multimedia Processing Requirements n Need to process continuous multimedia data ¨ Processing occurs in predetermined, usually periodic intervals ¨ Processing must be completed by certain deadlines n Need RT process manager ¨ Perform admission control ¨ Determine schedule ¨ Perform reservation ¨ Schedule to give processing guarantees CS 598 KN - Fall 2018

RT/Multimedia Processing Requirements Main Problem: How to find a feasible schedule? n Conflicting Goals/Problems: n How do we schedule multimedia (RT) processes so that n non-RT processes do not starve when RT process is running 2. RT process is not subject to priority inversion 1. CS 598 KN - Fall 2018

Model in RT Scheduling n n Task (Process) – schedulable unit Task characterized by Timing constraints ¨ Resource requirements ¨ n Assumptions ¨ n Periodic tasks without precedence relations Time constraints s – task starting point ¨ e – task processing time ¨ d – task deadline ¨ p – task period ¨ CS 598 KN - Fall 2018

Periodic Constant Processing Time Tasks n Example Usage Pattern: Peak Processing Time=10 ms Period = 100 ms 0 10 100 110 200 210 Time(ms) CS 598 KN - Fall 2018

Periodic Variable Processing Time Tasks n Example Usage Pattern: Period = 100 ms Peak Processing Time = 30 ms Sustainable Processing Time = 15 ms Burst Tolerance = 7 ms 0 15 100 120200210 Time(ms) CS 598 KN - Fall 2018

Model of RT Scheduling n Congestion avoidance deadline ¨ If period at (k-1) step is equal to ready (start) time of period k n n Tasks: preemptive vs. non-preemptive Main goal of RT Scheduling: ¨ find feasible schedule of all periodic tasks so that newly arriving task and all previous admitted tasks finish processing in every period according to their deadline CS 598 KN - Fall 2018

Model of RT Scheduling Must have Schedulability (Admission) Test for RT tasks n What is the performance metric for RT tasks? n ¨ Guarantee ratio : = number of guaranteed tasks/total number of tasks ¨ Process utilization (U): CS 598 KN - Fall 2018

Scheduling Policies of RT Tasks n Rate- Monotonic Scheduling (RMS) ¨ Designed/proved by C. L. Liu and Layland 1973 ¨ Policy: task with highest rate has highest priority ¨ Static and optimal, priority-driven n n Optimal means that there no other static algorithm that is able to schedule a RT task which can’t be scheduled by RMS algorithm Assumptions: ¨ ¨ ¨ Tasks are periodic Each task must complete before next request All tasks are independent Run-time of each task request is constant Any non-periodic task has no required deadline CS 598 KN - Fall 2018

Example of RMS CS 598 KN - Fall 2018

Scheduling Policies for RT Tasks n Earliest Deadline First (EDF) Policy ¨ Optimal dynamic algorithm ¨ Produces valid schedule if one exists ¨ Complexity O(n 2) ¨ Upper bound of process utilization 100% ¨ Policy: task with earliest deadline has highest priority CS 598 KN - Fall 2018

Example of EDF CS 598 KN - Fall 2018

Comparison between RMS and EDF CS 598 KN - Fall 2018

Admission Control (for preemptive tasks) n Schedulability test for RMS n Schedulability test for EDF CS 598 KN - Fall 2018

Example n Consider the following preemptive RT tasks and their characteristics ¨ T 1: p 1 = 50 ms, e 1=10 ms ¨ T 2: p 2 = 100 ms, e 2=20 ms ¨ T 3: p 3 = 200 ms, e 3=50 ms ¨ T 4: p 4 = 100 ms, e 4=20 ms n Are these tasks schedulable via RMS? ¨ If n yes, what is the feasible schedule? Are these tasks schedulable via EDF? ¨ If yes, what is the feasible schedule? CS 598 KN - Fall 2018

MULTI-CLASS AND HIERARCHICAL SCHEDULING FOR MULTIMEDIA CS 598 KN - Fall 2018

DSRT (Dynamic Soft Real-Time Scheduling) – Example of a Multimedia CPU Scheduler n Multiple CPU Service Classes ¨ Schedule real-time and not real-time tasks in an integrated fashion Execution Flow of a SRT Process n Possible Mapping CPU Service Classes into a Multiprocessor Partitioning Design n CS 598 KN - Fall 2018

CPU Service Classes Specification Parameters PCPT (Periodic Constant Processing Time) P = Period PPT = Peak Processing Time PPT PVPT (Periodic Variable Processing Time) P = Period SPT = Sustainable Processing Time PPT = Peak Processing Time BT = Burst Tolerance SPT ACPU (Aperiodic Constant Processor Utilization) PPU = Peak Processor Utilization PPU Event Relative Deadline PPTCS =598 KN Peak- Fall Processing Time 2018 Guaranteed PPT

Periodic Constant Processing Time Class n Example Usage Pattern: Peak Processing Time=10 ms Period = 100 ms 0 10 100 110 200 210 Time(ms) Processing IIIIII frames CS 598 KN - Fall 2018

Periodic Variable Processing Time Class n Example Usage Pattern: Period = 100 ms Peak Processing Time = 30 ms Sustainable Processing Time = 15 ms Burst Tolerance = 7 ms 0 15 100 120200210 Time(ms) Processing of IPBBPI frames CS 598 KN - Fall 2018

Execution Flow of a SRT Process Probing Phase Reservation Phase Execution Phase Scheduling Probe Admission Control Extract Reservation Contract Processor Binding Monitor/ Conformance Test Adaptation SRT Process Adjusted Contract CS 598 KN - Fall 2018

C++ APIs // Reservation Phase cpu. reserve(reservation); cpu. set. Adapt. Strategy(strategy); Cpu. Api cpu; Cpu. Reservation reservation; // Probing Phase; cpu. probe(); cpu. start(); for (int i=0; i<num. Probe. Iterations; i++) { do. Job(); cpu. yield(); } cpu. stop(); cpu. probe. End(); cpu. probe. Match(&reservation); // Execution Phase cpu. start(); for (; ; ) { do. Job(); cpu. yield(); } cpu. stop(); cpu. free(); CS 598 KN - Fall 2018

Smart Offline Probing (1) n Goal: Extract a reservation. ¨ Determine the most suitable Service Class and Parameters. ¨ Avoid over/under reserve resources. n Needed because: ¨ Processor usage is hardware platform dependent. ¨ Processor usage is input dependent. CS 598 KN - Fall 2018

Smart Probing (2) n n DSRT runs a few iterations of SRT applications without reservation. DSRT monitors the usage iteration by iteration. DSRT analyzes the usage history. Estimate a Reservation Processor Usage n Iteration Number CS 598 KN - Fall 2018

Smart Probing (3) n Compute average processor usage = 50 ms Compute peak processor usage = 62 ms. ¨ Max Burst = 12 ms ¨ n It is Periodic Variable Processing Time (PVPT) 40 ms 50 ms 62 ms 43 ms 40 ms 50 ms 62 ms 12 ms 55 ms 50 ms CS 598 KN - Fall 2018 50 ms 55 ms 60 ms 10 ms 50 ms

Multiprocessor Partitioning Design Guaranteed Part Non-guaranteed Part PCPT Processes Reserved Run Overrun PVPT Processes Reserved Run Burst RT Scheduler Overrun TS Processes Overrun Scheduler TS Scheduler Processor #1 RT Partition Overrun Partition TS Partition Processor #2 RT Partition Overrun Partition TS Partition CS 598 KN - Fall 2018

Admission Control Test n Given a reservation request, determine ¨ Resource Availability. (1) + (2) ¨ Processor Binding. (2) n Preemptive Earliest Deadline First: CS 598 KN - Fall 2018

Partition Scheduling (Hierarchical Top-Level Scheduling) Scheduler Proportional Sharing RT Scheduler Overrun Scheduler Multi-Level Round Robin Priority Queues Preemptive EDF TS Scheduler Kernel Scheduler Processor #1 RT Partition Overrun Partition TS Partition Processor #2 RT Partition Overrun Partition TS Partition CS 598 KN - Fall 2018

RT Partition Scheduler Processor #1 Waiting Queue: (Sorted with the earliest released time) p (2 a) finished one iteration (3) released for next iteration (1) admitted p Runnable Queue: (Sorted with the earliest deadline) p (2 b) overrunning Overrun Partition Queues CS 598 KN - Fall 2018

Overrun Partition Scheduler Highest Priority Burst Queue (FIFO) (2) Miss Deadline p Overrun Queue (FIFO) (1 a) Conformed? (1 b) Nonconformed? p Permanent Non-conforming Queue (FIFO) (1 c) Nonconformed frequently? Lowest Priority CS 598 KN - Fall 2018

Adaptation Scene #1 Scene #2 Adjust Reservation. n Adaptive Strategy n ¨ Exponential Average Frame 275 CS 598 KN - Fall 2018

Exponential Average Adaptation Strategy X 1 X 2 X 3. . . Xws-1 Xws+1 . . . X 2 ws. . . X 3 ws Adaptation Points n Iteration Number Specification: ¨ Window Size (ws). ¨ Alpha ( ) ¨ Xi= Guaranteed Parameter in a reservation. ¨ Xi-1 = Actual Usage. CS 598 KN - Fall 2018

User-level Priority Dispatch Highest Priority Fixed RT Priority (6) Timer interrupts Scheduler (5) Decrease Priority Dynamic TS Priority (5) Dispatched Process runs for T ms. (4) Scheduler sleeps. (3) Scheduler sets Timer for T ms. N DSRT Scheduler Process N-1 Dispatched Process . . . . Time Sharing Processes (2) Set Processor Affinity. . . . (1) Increase Priority. 0 RT Process Pool Lowest Priority CS 598 KN - Fall 2018

Experiment Setup n n Run 8 TS processes and 5 SRT processes concurrently. TS 1 -6: Computational intensive programs. TS 7 -8: Compilation programs. SRT 1: A MPEG player at 10 FPS. ¨ Probing (PVPT class, P=100 ms, ¨ SPT=28 ms, PPT=40 ms, BT=11 ms). ¨ Adaptation Strategy: (Statistical, f = 20%, ws = 20). CS 598 KN - Fall 2018

Experimental Setup (Cont. ) n SRT 2: A MPEG player at 20 FPS. ¨ Probing (PVPT class, P=50 ms, SPT=14 ms, PPT=21 ms, BT=6 ms) n SRT 3: A sampling program. ¨ Probing(PCPT class, P=50 ms, PPT=10 ms) n SRT 4: A Java Rocks. In. Space game. ¨ (PCPT n class, P=100 ms, PPT=30 ms). SRT 5: Misbehaving greedy program. ¨ (PCPT class, P=500 ms, PPT=10 ms). CS 598 KN - Fall 2018

Experimental Result SRT 2: MPEG player with P=50 ms. SRT 1: MPEG player With P=100 ms CS 598 KN - Fall 2018

Experimental Result (Cont. ) SRT 3: Sampling program with P=50 ms. SRT 4: Java game with P=100 ms. CS 598 KN - Fall 2018

Conclusion (Over-provisioning Approach) n Current Approach: ¨ To achieve throughput - overprovision of resources (fast processors, large disks, fast I/O bus, high-speed networks), and exclusive usage of resources ¨ To achieve timing - overprovision CPU, exclusive usage of resources, application-dependent scheduling One Possibility: With over-provisioning we get Quality of Service !! CS 598 KN - Fall 2018

Conclusion (Problems with Overprovisioning) n Violation of Timing Guarantees in Exclusive Case (head-of-line blocking) n n Violation of Timing/Throughput Guarantees in Shared Case (greedy applications, flows) n n UDP flows versus TCP Flows Last Mile Problem (not everywhere is overprovisioning possible) n n Sensory and Video Data Processing/Transmission University Campus, homes Need support of Qo. S in OS towards multimedia CS 598 KN - Fall 2018 tasks