Dimensionality Reduction for Analyzing Cycle Stealing Priority Queueing

Dimensionality Reduction for Analyzing Cycle Stealing, Priority Queueing, Threshold Policies, and more … Mor Harchol-Balter Computer Science Dept. Carnegie Mellon University 1

Many examples multiserver sharing FIFO S L H L MH H Common Problem: 2 D-infinite Markov chain 2

Dimensionality Reduction for Markov Chains — — Approximate technique + Provides first accurate performance numbers for many common problems. Taka Osogami 03] [Sigmetrics Problem-dependent. [SPAA 03] + Very fast (less than 1 sec). Doesn’t always apply. Adam Wierman [ICDCS 03] Alan Scheller-Wolf [Tools 03 a, 03 b] [Allerton+04]< 1% error typically, and can improve. Mark Squillante [Perf. Eval 05] Li[QUESTA Zhang 05] + Models any job sizes, and any loads. [Perf. Eval 06 a] [Perf. Eval 06 b] [Mgmt Science 07] thresholds and switching costs + Adding is often easy. 3

Cycle Stealing Problem l. B jobs/sec [Sigmetrics 03, Perf. Eval. 05] l. D jobs/sec Load r. B Load r. D Betty the beneficiary Dan the donor 4

Cycle Stealing Problem l. B jobs/sec l. D jobs/sec Load r. B Load r. D switch When donor is idle, donor helps beneficiary. 5

Cycle Stealing Problem l. B jobs/sec l. D jobs/sec Load r. B Load r. D switch back When new donor job arrives, donor switches back to donor queue. 6

Generalized Cycle Stealing l. D jobs/sec l. B jobs/sec Load r. B Load r. D time KSW NBthresh Only switch if Betty has NBthresh jobs queued. time KBA NDthresh Only switch back if Dan has NDthresh jobs queued. 7

Cycle Stealing Problem l. B jobs/sec Load r. B l. D jobs/sec Load r. D What is Betty’s/Dan’s mean response time? 8

What’s so hard? l. B 0 B, 0 D l D m. D 0 B, 1 D l D m. D 0 B, 2 D l D m. B l. B 1 B, 0 D l D m. D 1 B, 1 D l D m. D 1 B, 2 D l D m. D 2 m. B l. B 2 B, 0 D l D m. D 2 B, 1 D l D m. D 2 B, 2 D 2 m. B l. B m. B l D m. D Even simplest-case chain grows infinitely in 2 D. 9

Prior work: cycle stealing & coupled-processor Truncate the chain Exponential job sizes (80’s) General Job sizes (80, 90, 00’s) Green Stanford Grassman Rao Posner Convert to Riemann-Hilbert problem Convert to Wiener-Hopf boundary problem Very complex integrals. Fayolle, Iasnogorodski, Konheim, Meilijson, Melkman. . . Very complex integrals for WORKLOAD. No numbers. Cohen, Boxma, Borst, Uitert, Jelenkovic. . . Tail Asymp or Heavy traff. Bob Foley Mc. Donald Mike Harrison Boxma Borst van Uitert Williams 10

Dimensionality Reduction Key idea: 2 D-infinite chain VERY HARD 1 D-infinite chain EASY 11

What’s so hard? l. B 0 B, 0 D l D m. D 0 B, 1 D l D m. D 0 B, 2 D l D m. B l. B 1 B, 0 D l D m. D 1 B, 1 D l D m. D 1 B, 2 D l D m. D 2 m. B l. B 2 B, 0 D l D m. D 2 B, 1 D l D m. D 2 B, 2 D 2 m. B l. B m. B l D m. D Even simplest-case chain grows infinitely in 2 D. 12

Solution l. B Cooperating servers 0 B, 0 D l. D Working independently m. B BD l. B 0 B, 1+D m. B l. B 1 B, 0 D l. D BD 1 B, 1+D 2 m. B l. B 2 B, 0 D l. B m. B BD 2 B, 1+D 2 m. B l. B m. B New type of transition. Donor Busy period: BD 1 D-infinite chain! 13

Solution Cooperating servers Working independently 0 B, 0 D l. D 0 B, 1+D l. B 1 B, 0 D m. B l. B 2 m. B 2 B, 0 D l. B l. D m. B l. B 2 m. B l. B m. B 1 B, 1+D m. B BD = g a b m. B 2 B, 1+D l. B m. B Approximation, but can be made as close to exact as desired. [Tools 03 a, Tools 03 b] 14

Markov chain is easy to generalize • Generalize service distribution • Generalize to switching times • Generalize to include thresholds Load r. B l. B jobs/sec l. D jobs/sec time NBthresh time KSW KBA Load r. D NDthresh 15

Some interesting questions Q: Exactly how does cycle stealing affect beneficiary and donor? E[TB] orig cs E[TD] cs orig r. B 16

Some interesting questions Q: How does donor job size variability affect benef. resp. time? A: Hardly, for r. B < 1 ! E[TB] C 2=50 C 2=8 Exp r. B 17

Some interesting questions Q: How should we set NBth? E[TB] A: Want NBth high, but not big deal. E[TD] Increasing NBth doesn’t hurt benef! Increasing NBth helps donor r. B Q: How should we set NDth? E[TB] r. B A: Want NDth low if r. B<1. Want NDth high if r. B>1. E[TD] Increasing NDth helps benef only in overload! Increasing NDth hurts donor r. B 18

Server farm application [SPAA 03, ICDCS 03] Size-based task assignment Shorts only. Longs only. Q: Can we enhance this further with cycle stealing? 19

Two cycle-stealing enhancements Central-Queue Immediate-Dispatch Shorts only Longs. Short if idle. Longs. Short if no long. No Immediate stealing Dispatch Central Queue 0 0. 5 1 1. 5 20

N-sharing model l. D l. B m. BD Beneficiary (Betty) m. D Donor (Dan) [Allerton 04, PER 05] cycle-stealing: Dan helps Betty with her work when he’s free. But, can do better with more aggressive cycle stealing, (if Dan processes Betty’s jobs faster than his own). Studied by: S. Bell, R. Williams, M. Harrison, M. Lopez, M. Squillante, C. Xia, D. Yao, L. Zhang, R. Schumsky, L. Green, S. Meyn, A. Ahn, D. Stanford, W. Grassman, … 21

Who gets control: man or woman? Help me when I have > TB jobs, or you’re free. TB Beneficiary Side Control TD I’ll help you when I have < TD jobs. Donor Side Control 22

Who gets control: man or woman? Help me when I have > TB jobs, or you’re free. TB Beneficiary Side Control TD I’ll help you when I have < TD jobs. Donor Side Control Answer: Mean response time E[T] lower when woman controls! 23

Which policy is more robust? Help me when I have > TB jobs, or you’re free. TB Beneficiary Side Control TD I’ll help you when I have < TD jobs. Donor Side Control Answer: Donor control helps, but even better is to let Benef. have 2 thresholds, where Donor controls which threshold is used. 24

Mean response time Results: Adaptive Dual Threshold policy TBT =6=6 TB=20 B (opt when (opt) r. D = 0. 6) (opt when r. D=0. 8) ADT: meets both goals. Dan’s load Similar idea suggested by Sean Meyn under finite queues. 25

Priority Scheduling [QUESTA 05, Perf. Eval. 06] M/PH/k with priority classes H L L MH H Goal: Mean response time per job type. 26

Prior Work: Multi-Server Priority Queues Two job classes, exponential Truncation w/Matrix Analytic or Aggregation Kao and Narayanan, Kapadia, Kazumi, Mitchell, Nishida Ngo and Lee, Leemans, Miller Generating Functions Davis, Kella & Yechiali, Feng, Kawada, Adachi, Kao and Wilson, Gail, Hantler, Taylor Multi-class simple approx. Scaling as Single -server: Buzen and Bondi Aggregation into Two classes: Mitrani and King Nishida Little known for > 2 classes or non-exponential. 27

2 Priority classes is easy via Dimensionality Reduction l. L 0 L, 0 H l. H m. L m. H l. L 1 L, 0 H l. H 2 m. L m. H l. L 0 L, 1 H l. H m. L 2 m BH l. L 0 L, 2+H l. L 2 L, 0 H l. H 2 m. L m. H l. L 1 L, 1 H l. H m. L 2 m BH 1 L, 2+H l. L 2 L, 1 H l. H m. L 2 m BH l. L 2 L, 2+H 28

What about 3 classes H, M, L? • Can get H and M response times from this chain: • Obvious Idea: Combine H and M into single class HM. L’s are second class. 0, 0 1, 0 2, 0 0, 1 1, 1 2, 1 0, 2+ 1, 2+ 2, 2+ • Need to be careful: Duration of HM busy period depends on who started it: H&H, H&M, or M&M. Must differentiate. 29

Extensions q Recursive Dimensionality Reduction extends to any number of classes and servers. q Generalize service distribution. q Simple representation allows us to derive not just mean response time, but all moments of response time. 30

Mean response time Results: 4 priority classes M/M/2 with 4 classes % error in delay 31

Effect of Variability M/G/1 L H C 2 H Mean response time M/G/2 L H C 2 H 32

Effect of more servers Mean response time M/G/k with 2 priority classes 1 server 2 servers 4 servers 8 servers C 2 H 33

Optimal number of servers (2 classes with same mean) C 2 H r 34

Optimal number of servers Smart Prioritization (Low priority jobs BIGGER) C 2 H Stupid Prioritization (Low priority jobs SMALLER) C 2 H r r 35

Lessons learned on priorities in multiserver Multiserver priority queue performance is very different from single server. Performance depends on many factors: q q Variability of job size Load Number of servers Stupid vs. Smart prioritization 36

There are many other examples of problems analyzable via Dimensionality Reduction … 2 Dinfinite chain 1 Dinfinite chain Open problem: Proving the approximation 37

FIFO S L THANK YOU! H LLH H 38

BACKUP 39

MODIFIED 1 D-chain: cycle stealing with switching costs and NBth = 3 40