FAIRNESS IN QUEUES Adam Wierman Carnegie Mellon University

FAIRNESS IN QUEUES Adam Wierman Carnegie Mellon University acw@cs. cmu. edu Joint work with Mor Harchol-Balter 1 Carnegie Mellon University Computer Science Department

“SCHEDULING SUCCESS STORIES” ARE EVERYWHERE users CPUs Internet Web Servers Routers Databases Locks Disks …also p 2 p, wireless, operating systems… Biersack, Rai, Urvoy-Keller, Harchol-Balter, Schroeder, Agrawal, Ganger, Petrou, Misra, Feng, Hu, Zhang, Mangharam, Sadowsky, Rawat, Dinda, Mc. Wherter, Ailamaki, & others 2 Carnegie Mellon University Computer Science Department

THE ESSENCE OF A “SCHEDULING SUCCESS STORY” web server, router, etc. Goal Minimize user response times bottleneck resource Processor Sharing (PS) Use a different scheduling policy 3 Carnegie Mellon University Computer Science Department

SRPT WINS BIG de h Scedli-ng hu Sc ing ul ? ? mean response time WHAT POLICY SHOULD WE USE? What about 0 0. 25 fairness to large jobs? PS FCFS T P R S 0. 5 load 0. 75 4 Carnegie Mellon University Computer Science Department

ARE POLICIES THAT PRIORITIZE SMALL JOBS FAIR TO LARGE JOBS? ? What does “fair” mean? Queue M/GI/1 Preempt-Resume 5 Carnegie Mellon University Computer Science Department

WHAT IS FAIRNESS? Definition: A fair decision is one that makes the proper balance of conflicting interests The “proper balance” is determined by (i) consulting the majority, (ii) employing logic, (iii) referring to the divine. 6

? Which is more fair when two jobs of the same size arrive one afer the other. . . 1 B A 2 B A 7

? 1 Which is more fair when a small job and a large job arrive at the same time. . . B A 2 B A 8

BUT, THE SETTING MATTERS OUR SETTING: Are the response times of large jobs “unfairly” long? 9

WHAT IS FAIRNESS? Definition: A fair decision is one that makes the proper balance of conflicting interests The “proper balance” is determined by (i) consulting the majority, (ii) employing logic, (iii) referring to the divine. 10

EMPLOYING LOGIC Aristotle’s notion of fairness Like cases should be treated alike, different cases should be treated differently, and different cases should be treated differently in proportion to their differences. Response times should be proportional to job sizes. T(x)/x should be constant slowdown 11

EMPLOYING LOGIC Rawls’ Theory of Social Justice All social goods should be distributed equally, unless unequal distribution is to the advantage of the least favored Share the server evenly among all jobs Processor Sharing (PS) is fair 12

EMPLOYING LOGIC Min-Max fairness (Pareto Efficiency) All jobs deserve an equal share of the resources. . . but if some jobs can use more without hurting others, that’s okay If you beat PS for all sizes, you’re still fair 13

MIN-MAX FAIRNESS Wierman and Harchol-Balter 2003 Definition: Consider an M/GI/1 queue. A policy P is min-max fair if: ? Is dividing by “x” the right scaling? ? Is 1/(1 -ρ) really a min-max criteria 14

? Is dividing by “x” the right scaling? ? Is 1/(1 -ρ) really a min-max criteria Theorem: In an M/GI/1 with an unbounded, continuous service distribution having finite E[X 2], under any non-idling policy we have and further [Wierman and Harchol-Balter 2003] 15

MIN-MAX FAIRNESS Wierman and Harchol-Balter 2003 Definition: Consider an M/GI/1 queue. A policy P is min-max fair if: ? How do common policies perform? 16

PS – FCFS – SRPT E[T(x)] / x Processor Sharing PS x 17

PS – FCFS – SRPT E[T(x)] / x First Come First Served The unfairness can be unbounded PS FCFS x Under a Pareto with ρ=0. 8, this is >80% of the jobs 18

PS – FCFS – SRPT PS SRPT x Theorem SRPT is fair for ρ<0. 5, regardless of the job size distribution. E[T(x)] / x Shortest Remaining Processing Time SRPT PS x Theorem For every service distribution with finite E[X 2], SRPT is unfair for high enough load. 19

OTHER POLICIES PS Always Fair Sometimes Fair SRPT Always Unfair FCFS 20

OTHER POLICIES PLCFS PS SRPT Prioritize Small Jobs Always Fair PSJF Sometimes Fair SJF LRPT Always Unfair Prioritize Large Jobs PLJF LCFS ROS FCFS LJF FB [Wierman and Harchol-Balter 2003, 2005] 21

OTHER POLICIES PLCFS PS Remaining size based SRPT LRPT Preemptive size based Always Fair PSJF Sometimes Fair SJF Non-preemptive size based PLJF Always Unfair Non-preemptive non-size based LCFS ! ROS FCFS Age based FB a tradeoff There’s between fairness and efficiency [Wierman and Harchol-Balter 2003, 2005] 22

FAIRNESS VS. EFFICIENCY PLCFS PS SRPT Always Fair PSJF Sometimes Fair ? more circles Is there a fair better mean response time policy with near SJF LRPT PLJF Always Unfair LCFS ROS FCFS LJF FB optimal performance? 23

Fair Sojourn Protocol (FSP) [Henderson and Friedman 2004] Guarantees that every job finishes earlier than it would under PS. 24

Fair Sojourn Protocol (FSP) “Do SRPT on the PS remaining times” 1. Compute the time when each job would finish in a virtual PS system. 2. Order the jobs according to their virtual completion times. 3. Devote the full processor to the job with the earliest PS completion time 25

Fair Sojourn Protocol (FSP) “Do SRPT on the PS remaining times” ! FSP did the same thing as SRPT 26

Fair Sojourn Protocol (FSP) “Do SRPT on the PS remaining times” ? How good are response times under FSP? 27

? How good are response times under FSP? . . . your answer here. . . 28

MIN-MAX FAIRNESS Wierman and Harchol-Balter 2003 Definition: Consider an M/GI/1 queue. A policy P is min-max fair if: ! Worries about fairness are not limited to the mean 29

E[T(x)] ? BEYOND EXPECTATION: Higher Moments X X Raw moments E[T(x)i] Central moments Var[T(x)], etc Cumulant moments [Wierman and Harchol-Balter 2005] 30

Cumulants are a descriptive statistic, similar to the moments. CUMULANTS ? They can be found as a function of the moments: or from the log of the moment generating function: Do these look familiar? 31

Cumulants have many nice properties: additivity: WHY CUMULANTS? homogeneity: 1 st cumulant is shift-equivariant & the rest are shift-invariant 32

MIN-MAX FAIRNESS Wierman and Harchol-Balter 2005 Definition: Consider an M/GI/1 queue. A policy P is min-max fair if, for all i: ? Why is the right generalization? ! Lots of open questions here 33

MIN-MAX FAIRNESS IS NOT ALWAYS APPROPRIATE FCFS is the most fair policy in some settings 34

TEMPORAL FAIRNESS It is unfair to violate the seniority of a job Definition: The politeness experienced by a job of size x under policy P, Pol(x)P, is the fraction of the response time during which the seniority of the job is respected. [Wierman 2004] 35

Definition: The politeness experienced by a job of size x under policy P, Pol(x)P, is the fraction of the response time during which the seniority of the job is respected. E[Pol(x)] FCFS – PLCFS – PS – SRPT First Come First Served 1 FCFS 1 -ρ 0 x 36

Definition: The politeness experienced by a job of size x under policy P, Pol(x)P, is the fraction of the response time during which the seniority of the job is respected. FCFS – PLCFS – PS – SRPT E[Pol(x)] Preemptive Last Come First Served 1 1 -ρ 0 FCFS PLCFS x 37

FCFS – PLCFS – PS – SRPT E[Pol(x)] Processor Sharing 1 FCFS PS 1 -ρ 0 PS PLCFS x PLCFS 0 job size %-tile 1 38

FCFS – PLCFS – PS – SRPT E[Pol(x)] Shortest Remaining Processing Time 1 FCFS SRPT 1 -ρ 0 SRPT PLCFS x ! PLCFS 0 job size %-tile 1 Prioritizing small jobs helps overall politeness 39

MANY INTERESTING QUESTIONS. . . 1) Can you think of a policy (other than FCFS) that avoids being maximally impolite to some job size? 2) How polite are scheduling heuristics & techniques? 3) What is the tradeoff between temporal fairness and proportional fairness? and efficiency? 40

PROPORTIONAL FAIRNESS min-max fairness VS TEMPORAL FAIRNESS politeness VS EFFICIENCY mean response time 41

Min-max Fairness more fair Theorem FCFS In an M/GI/1 any Always Fair policy has Politeness more polite less fair FCFS SRPT FSP PS PLCFS less polite FSP LRPT PS PLCFS SRPT LRPT 42

less polite Min-max Fairness more fair FCFS Politeness more polite less fair SRPT FSP LRPT PS PLCFS more circles better mean response time 43

WRAPUP 44 Carnegie Mellon University Computer Science Department 44

FAIRNESS IS AN AMORPHOUS CONCEPT a universal definition is unlikely to be useful (or possible) PROPORTIONAL FAIRNESS min-max ! TEMPORAL FAIRNESS politeness Prioritizing small jobs provides a balance between fairness & efficiency . . . THIS IS A VERY ACTIVE AREA OF RESEARCH, MANY OPEN QUESTIONS REMAIN 45

AN EXPLOSION OF WORK ON FAIRNESS since the introduction of min-max fairness in 2003 analyzing policies Many papers by: Wierman, Harchol-Balter, Henderson, Friedman, Biersack, Rai, Ayesta, Aalto, Nunez-Queija, Misra, Feng, Vernon, Williamson, Brown, Bansal, and others defining other fairness metrics [Wierman 2004] [Raz, Avi-Itzhak 2004] [Levy, Raz, Avi-Itzhak 04] extending definition [Sandmann 2005] to higher moments [Wierman & Harchol-Balter 2005 a, b] [Williamson, Gong 2003, 2004] [Brown 2006] and others. 46

MANY OTHER INTERESTING METRICS fair service percentage percent of service given to job i [Levy, Raz, Avi-Itzhak 04] Disc. Freq = ni + c∙mi ni = number of jobs that arrived later and completed earlier than job i mi = number of larger jobs (at the arrival of job i) that complete earlier than job i [Sandmann 2005] 47

Adam Wierman Carnegie Mellon University acw@cs. cmu. edu Papers are available at: http: //www. cs. cmu. edu/~acw 48