Heavy Tails Performance Models Scheduling Disciplines Mor HarcholBalter

Heavy Tails: Performance Models & Scheduling Disciplines Mor Harchol-Balter Carnegie Mellon University Computer Science 1

Q: Which minimizes mean response time? jobs FCFS “size” = service requirement load r < 1 jobs PS jobs SRPT 2

Q: Which best represents scheduling in web servers ? jobs FCFS “size” = service requirement load r < 1 jobs PS jobs SRPT 3

IDEA: Use SRPT instead of PS in Web servers client 1 “Get File 1” 1 client 2 “Get File 2” Internet 1000 Linux 0. S. client 3 “Get File 3” APACHE 2 WEB 3 SERVER Request for file Requested file 4

Objections to SRPT: Need to know “size” of request Ö Unfairness to requests for big files 5

Outline of Talk THEORY I: THEORY II: Unfairness in All Scheduling Policies IMPLEMENT Investigating Unfairness in SRPT (M/G/1, c. f. m. f. v. ) [Sigmetrics 01] (M/G/1, c. f. m. f. v. ) [Performance 02], [Sigmetrics 03] III: Implementation of SRPT in Web servers [Transactions on Computer Systems 03], [ITC 03] Papers are joint with Adam Wierman & Bianca Schroeder 6

Outline of Talk THEORY I: THEORY II: Unfairness in All Scheduling Policies IMPLEMENT Investigating Unfairness in SRPT (M/G/1, c. f. m. f. v. ) [Sigmetrics 01] (M/G/1, c. f. m. f. v. ) [Performance 02], [Sigmetrics 03] III: Implementation of SRPT in Web servers [Transactions on Computer Systems 03], [ITC 03] www. cs. cmu. edu/~harchol/ 7

SRPT has a long history. . . 1966 Schrage & Miller derive M/G/1/SRPT response time: 1968 Schrage proves optimality 1979 Pechinkin & Solovyev & Yashkov generalize 1990 Schassberger derives distribution on queue length BUT WHAT DOES IT ALL MEAN? 8

SRPT has a long history (cont. ) 1990 - 97 7 -year long study at Univ. of Aachen under Schreiber SRPT WINS BIG ON MEAN! 1998, 1999 Slowdown for SRPT under adversary: Rajmohan, Gehrke, Muthukrishnan, Rajaraman, Shaheen, Bender, Chakrabarti, etc. SRPT STARVES BIG JOBS! Various o. s. books: Silberschatz, Stallings, Tannenbaum: Warn about starvation of big jobs. . . Kleinrock’s Conservation Law: “Preferential treatment given to one class of customers is afforded at the expense of other customers. ” 9

Real-world job sizes are Heavy Tailed log-log plot Heavy-tailed Property: Job size (x seconds) [Sigmetrics 96] “Largest 1% of jobs comprise half the load. ” 10

Unfairness Question Let r=0. 9. Let G: Bounded Pareto(a = 1. 1, max=1010) ? PS M/G/1 ? Question: Which queue does biggest job prefer? SRPT M/G/1 11

Results on Unfairness Let r=0. 9. Let G: Bounded Pareto(a = 1. 1, max=1010) I SRPT PS SRPT 12

Results on Unfairness Let G: Bounded Pareto(a = 1. 1, max=1010) PS SRPT 13

Unfairness – General Distribution All-can-win-theorem: For all distributions, if r < ½, E[T(x)]SRPT < E[T(x)]PS for all x. 14

All-can-win-theorem: For all distributions, if r < ½, E[T(x)]SRPT £ E[T(x)]PS for all x. Proof: x ò 2 l t f (t ) dt + l x F ( x ) 0 2 2(1 - r ( x)) 2 E[Wait(x)]SRPT x + dt ò 1 - r (t ) 0 E[Res(x)]SRPT E[T(x)]PS 15

All-can-win-theorem: For all distributions, if r < ½, E[T(x)]SRPT £ E[T(x)]PS for all x. Proof cont. x 2 x + l ò t f (t ) dt l F ( x ) 2 0 2(1 - r ( x)) 2 - x dt ò 1 - r (t ) 0 Need sufficient condition s. t. 16

All-can-win-theorem: For all distributions, if r < ½, E[T(x)]SRPT £ E[T(x)]PS for all x. Proof cont. x 2 x + l ò t f (t ) dt l F ( x ) 2 0 2(1 - r ( x)) 2 Need sufficient condition s. t. 17

All-can-win-theorem: For all distributions, if r < ½, E[T(x)]SRPT £ E[T(x)]PS for all x. Proof cont. x 2 x l ò t f (t ) dt + l F ( x ) 2 0 2(1 - r ( x)) 2 Need sufficient condition s. t. Observe: 18

All-can-win-theorem: For all distributions, if r < ½, E[T(x)]SRPT £ E[T(x)]PS for all x. Proof cont. x ò 2 l t f (t ) dt + l x F ( x ) 0 2 2(1 - r ( x)) 2 x ò 2 2 l t f (t ) dt + l x F ( x ) 0 1 - r Suffices that: 2(1 - r(x))2 > 1 - r. Suffices that r(x) < 1/2 19

Intuition 1) All-can-win-theorem: For all distributions, if r < ½, E[T(x)]SRPT £ E[T(x)]PS for all x. 2) Under Bounded Pareto distribution (a = 1. 1), r < E[T(x)]SRPT < E[T(x)]PS for all x. *HT Property* E[Wait(x)] 0. E[Res(x)]: small 20

Outline of Talk THEORY I: THEORY II: Unfairness in All Scheduling Policies IMPLEMENT Investigating Unfairness in SRPT (M/G/1, c. f. m. f. v. ) [Sigmetrics 01] (M/G/1, c. f. m. f. v. ) [Performance 02], [Sigmetrics 03] III: Implementation of SRPT in Web servers [Transactions on Computer Systems 03], [ITC 03] www. cs. cmu. edu/~harchol/ 21

What is fair? Slowdown for job of size x Response time for job of size x PS does not bias towards any particular job size. Slowdown is independent of size Definition: A policy P is fair if E[S(x)]P ≤ E[S(x)]PS for all x. Otherwise, P is unfair. 22

Classification of Scheduling Policies Always Fair for all loads and distributions * and distributions Sometimes Unfair Fair for some loads, and unfair for other loads* Always Unfair for all loads and distributions 23

Testing your intuition: Always Fair for all loads and distributions • • Sometimes Unfair Fair for some loads, and unfair for other loads* Always Unfair for all loads and distributions Where does SRPT lie? _____ What about other policies that prioritize based on remaining size? ____ What about preemptive policies that prioritize based on age? _____ What about preemptive policies that prioritize based on size? _____ * and distributions 24

Classification of Scheduling Policies SJF LJF FCFS PS Nonpreemptive FSP Always FAIR Always Unfair Sometimes Unfair Preemptive Remaining-size based Policies PLCFS SRPT LRPT Preemptive Size-based Policies Age. Based Policies FB PSJF Lots of open problems… 25

Always Unfair Theorem: Any preemptive, size based policy, P, is Always Unfair Case 1: A finite size, y, receives lowest priority Case 2: No finite size receives lowest priority (2 a) Priorities decrease monotonically -- PSJF (2 b) Priorities decrease non-monotonically. Unfair for all loads and distributions 26

Always Unfair Theorem: Any preemptive, size based policy, P, is Always Unfair Case 1: A finite size, y, receives lowest priority Unfair for all loads and distributions y V = Work in System 27

Always Unfair Theorem: Any preemptive, size based policy, P, is Always Unfair Case 2 a: Priorities decrease monotonically (PSJF) Infinite sized job has lowest priority. . . Unfair for all loads and distributions … but that job is treated fairly? 28

Always Unfair Theorem: Any preemptive, size based policy, P, is Always Unfair Case 2 a: Priorities decrease monotonically (PSJF) E[S(x)] PS Unfair for all loads and distributions PSJF 0 x Finding a hump shows PSJF is Always Unfair 29

Always Unfair Theorem: Any preemptive, size based policy, P, is Always Unfair Case 2 b: Priorities decrease non-monotonically E[S(x)] PS Unfair for all loads and distributions PSJF 0 x Find y beyond which PSJF treats all jobs unfairly. Find x > y, where x has lower priority than y. => x is treated unfairly. 30

The mysterious hump PS E[S(x)] PSJF 0 x x 0 This hump appears in many common policies, 1. PSJF 2. FB 3. SRPT 4. SJF 31

Outline of Talk THEORY I: THEORY II: Unfairness in All Scheduling Policies IMPLEMENT Investigating Unfairness in SRPT (M/G/1, c. f. m. f. v. ) [Sigmetrics 01] (M/G/1, c. f. m. f. v. ) [Performance 02], [Sigmetrics 03] III: Implementation of SRPT in Web servers [Transactions on Computer Systems 03], [ITC 03] www. cs. cmu. edu/~harchol/ 32

From theory to practice: What does SRPT mean within a Web server? • Many devices: Where to do the scheduling? • Many jobs at once. 33

Server’s Performance Bottleneck Site buys limited fraction of ISP’s bandwidth client 1 “Get File 1” client 2 “Get File 2” client 3 Rest of Internet WEB SERVER (Apache) Linux 0. S. ISP “Get File 3” 5 We schedule bandwidth at server’s uplink. 34

Network/O. S. insides of traditional Web server Socket 1 Client 1 Network Card Client 2 Socket 2 Web Server BOTTLENECK Client 3 Sockets take turns draining --- FAIR = PS. 35

Network/O. S. insides of our improved Web server Socket 1 Client 2 Client 3 Network Card BOTTLENECK 1 st 2 nd 3 rd S Socket 2 M Socket 3 priority queues. Web Server L Socket corresponding to file with smallest remaining data gets to feed first. 36

WAN EMU 1 2 3 200 Linux WAN EMU 1 2 3 200 WAN EMU Experimental Setup 1 APACHE 2 WEB 3 SERVER switch Linux 0. S. Implementation SRPT-based scheduling: 1) Modifications to Linux O. S. : 6 priority Levels 2) Modifications to Apache Web server 3) Priority algorithm design. 37

1 2 3 200 Linux WAN EMU Experimental Setup Flash Apache 10 Mbps uplink 1 APACHE WEB 2 SERVER 3 switch Linux 0. S. Trace-based workload: 100 Mbps uplink Surge Trace-based Open system Partly-open WAN EMU Number requests made: 1, 000 Size of file requested: 41 B -- 2 MB Distribution of file sizes requested has HT property. Geographicallydispersed clients Load < 1 Transient overload + Other effects: initial RTO; user abort/reload; persistent connections, etc. 38

Results: Mean Response Time (sec) . . FAIR . SRPT . Load 39

Mean Response time (ms) Mean Response Time vs. Size Percentile Load =0. 8 FAIR SRPT Percentile of Request Size 40

Transient Overload -Mean response time FAIR SRPT 41

Transient overload Response time as function of job size FAIR SRPT small jobs win big! big jobs aren’t hurt! 42

New project: Scheduling dynamic web requests “$$$buy$$$” “buy” Internet Web Server (eg: Apache/Linux) Internet High-Priority Client Low-Priority Client Database Disks CPU(s) Locks Need to schedule the database. . . 43

Conclusion Misconceptions about unfairness Discrimination against high-performance scheduling policies Classifying policies with respect to unfairness is counter-intuitive. Good news: Many high-performing policies are also fair in practice! 44