Littles Law Operational Laws Littles Law Proportionality relation

Little’s Law & Operational Laws

Little’s Law • Proportionality relation between the average number of jobs (E[N]) in a system and the average system time (E[T]) of those jobs (the job arrival rate is the proportionality constant) – No assumptions about the arrival or service processes • Two similar versions: Open and closed systems • Little’s Law for an ergodic open system: E[N] = λ E[T] – λ is the average job arrival rate in the system • Little’s law for an ergodic closed system: N = X E[T] – N is constant and equal to the multi-programming level – X is the system throughput (rate of job completion) – E[T] = E[R] + E[Z], where R is response time, and Z think time – Alternative version (Response Time Law): E[R] = N/X – E[Z] 2

Applying Little’s Law – (1) 1 2 • Assume that the above system is ergodic with X = = 1+ 2 , where 1 ( 2) is he arrival rate of red (green) jobs – Ergodicity requires < • Utilization Law: system = server – Let ( i) be fraction of time server is busy (with type i jobs) – Note: This is also the average number of (type i) jobs in service – Little’s Law implies = / ( i = i/ ), or = E[S] • Note: If E[S 1] E[S 2], then i = i/ i, where i = E[Si] • Time in queue: system = queue – E[NQi] = i E[TQi] 3

Applying Little’s Law – (2) … N in out • Assume N = 10, E[Z] = 5 secs, and E[R] = 15 secs • What is the system throughput X? N = X E[T] = X(E[Z] + E[R]) X = N/(E[R] + E[Z]) = 10/(5+15) = 0. 5 jobs/sec 4

Forced Flow Law • Relates system throughput to throughput of individual device – X is system throughput, Xi is throughput of device i – Vi is average number of visit to device i per job • Forced Flow Law: Xi = E[Vi] X – Basically accounts for the fact that for every system completion, there must have been E[Vi] completions (visits) at device i, so the rate of completion at device i must be higher in that same proportion 5

Bottleneck Law • Di is the per job service demand at device i, i. e. , total demand across all visits to device i – Di = Si, 1 + Si, 2 + … + Si, Vi – This implies E[Di] = E[Vi] E[Si], if Vi and Si are independent (a typical scenario) – Note that E[Di] = Bi/C, where Bi is the busy time at device i and C is the number of service completion during that time period • Bottleneck Law: i = X E[Di] ( i = Xi E[Si] = X E[Vi] E[Si] = X E[Di]) – Rate of outside arrivals is X and each such arrival contributes E[Di] worth of service time at device i, so that device i is busy a fraction of time ( i) equal to X E[Di] 6

Summary • Open systems – E[N] = λ E[T] (Little’s Law) – E[Nsubsystem] = subsytem E[Tsubsystem] (Little’s Law) • Closed systems – N = X E[T] = X (E[R] + E[Z]) (Little’s Law) – E[R] = N/X – E[Z] (Response Time Law) – i = i/ i = i E[Si] = Xi E[Si] (Utilization Law) – Xi = E[Vi] X (Forced Flow Law) – i = X E[Di] (Bottleneck Law) 7

Building on Little’s Law • Bounds on throughput and response time for closed systems where Dmax = maxi{E[Di]} (system bottleneck) First expression holds for N small, while second expression holds for N large • N* as the N value for which the two expressions are equal, i. e. , N* = (D+E[Z])/Dmax – When N > N*, throughput is dominated by bottleneck component 8

Example (1) … N = 10, E[Z] =5 • System A: DCPU = 4. 6 and Ddisk = 4. 0 • System B: DCPU = 4. 9 and Ddisk = 1. 9 CPU Disk • Which system has higher throughput? – First compute N* (N* = (D+E[Z])/Dmax) for both systems • DA = DCPU + Ddisk = 8. 6, DA, max = 4. 6 NA* = (8. 6+5)/4. 6 = 2. 95 • DB = DCPU + Ddisk = 6. 8, DB, max = 4. 9 NB* = (6. 8+5)/4. 9 = 2. 41 • NA* & NB* << N – So in both cases, the throughput is dominated by Dmax, and System A wins because it has a lower Dmax value 9

• Example (2) Summary – – – N = X E[T] = X (E[R] + E[Z]) E[R] = N/X – E[Z] (Response Time Law) i = i/ i = i E[Si] = Xi E[Si] (Utilization Law) Xi = E[Vi] X (Forced Flow Law) i = X E[Di] (Bottleneck Law) • Measurements for interactive system with N = 20 and E[Z] = 15 secs: – – – – T = 650 secs (duration of measurements) BCPU = 400 secs Bslowdisk = 100 secs Bfastdisk = 600 secs C = CCPU = 200 jobs Cslowdisk = 2, 000 jobs Cfastdisk = 20, 000 jobs • Improvements under consideration (are they worth it? ) a) b) c) d) Faster CPU Rebalancing disks Add a second fast disk, and split the load of the original fast disk using it (Note: This is sub-optimal – should rebalance across all three disks) Combine all three improvements, including rebalancing all three disks 10

• Example (2) Original system Summary – – – N = X E[T] = X (E[R] + E[Z]) E[R] = N/X – E[Z] (Response Time Law) i = i/ i = i E[Si] = Xi E[Si] (Utilization Law) Xi = E[Vi] X (Forced Flow Law) i = X E[Di] (Bottleneck Law) • Measurements for interactive system with N = 20 and E[Z] = 15 secs: – T = 650 secs (duration of measurements) – BCPU = 400 secs, Bslowdisk = 100 secs, Bfastdisk = 600 secs – C = CCPU = 200 jobs. Cslowdisk = 2, 000 jobs, Cfastdisk = 20, 000 jobs • Intermediate quantities – – – – – E[DCPU] = BCPU/C = 400 secs/200 jobs = 2. 0 secs/job E[Dslowdisk] = Bslowdisk/C = 100 secs/200 jobs = 0. 5 secs/job E[Dfastdisk] = Bfastdisk/C = 600 secs/200 jobs = 3. 0 secs/job E[VCPU] = CCPU/C = 200 visits/200 jobs = 1. 0 visit/job E[Vslowdisk] = Cslowdisk/C = 2, 000 visits/200 jobs = 10 visits/job E[Vfastdisk] = Cfastdisk/C = 20, 000 visits/200 jobs = 100 visits/job E[SCPU] = BCPU/CCPU = 400 secs/200 visits = 2. 0 secs/visit E[Sslowdisk] = Bslowdisk/Cslowdisk = 100 secs/2, 000 visits = 0. 05 sec/visit E[Sfastdisk] = Bfastdisk/Cfastdisk = 600 secs/20, 000 visits =0. 03 sec/visit • Original system: Dmax= 3 secs/job, D = 5. 5 secs/job, N* = 20. 5/3 7 << N = 20 – So X 1/Dmax= 0. 33 jobs/sec and E[R] N Dmax – E[Z] = 45 secs 11

• Example (2 a) – Faster CPU Summary – – – N = X E[T] = X (E[R] + E[Z]) E[R] = N/X – E[Z] (Response Time Law) i = i/ i = i E[Si] = Xi E[Si] (Utilization Law) Xi = E[Vi] X (Forced Flow Law) i = X E[Di] (Bottleneck Law) • Metrics for interactive system with N = 20 and E[Z] = 15 secs: – – – – – E[DCPU] = BCPU/C = 400 secs/200 jobs = 2. 0 secs/job E[Dslowdisk] = Bslowdisk/C = 100 secs/200 jobs = 0. 5 secs/job E[Dfastdisk] = Bfastdisk/C = 600 secs/200 jobs = 3. 0 secs/job (bottleneck) E[VCPU] = CCPU/C = 200 visits/200 jobs = 1. 0 visit/job E[Vslowdisk] = Cslowdisk/C = 2, 000 visits/200 jobs = 10 visits/job E[Vfastdisk] = Cfastdisk/C = 20, 000 visits/200 jobs = 100 visits/job E[SCPU] = BCPU/CCPU = 400 secs/200 visits = 2. 0 secs/visit E[Sslowdisk] = Bslowdisk/Cslowdisk = 100 secs/2, 000 visits = 0. 05 sec/visit E[Sfastdisk] = Bfastdisk/Cfastdisk = 600 secs/20, 000 visits =0. 03 sec/visit • Faster CPU (twice as fast): E[DCPU] = 1. 0 sec/job, but Dmax remains unchanged and N* stays approximately constant – Hardly any improvement as the fast disk is the bottleneck 12

• Example (2 b) – Rebalance Disks • – – – N = X E[T] = X (E[R] + E[Z]) E[R] = N/X – E[Z] (Response Time Law) i = i/ i = i E[Si] = Xi E[Si] (Utilization Law) Xi = E[Vi] X (Forced Flow Law) i = X E[Di] (Bottleneck Law) Metrics for interactive system with N = 20 and E[Z] = 15 secs: – – – – – • Summary E[DCPU] = BCPU/C = 400 secs/200 jobs = 2. 0 secs/job E[Dslowdisk] = Bslowdisk/C = 100 secs/200 jobs = 0. 5 secs/job E[Dfastdisk] = Bfastdisk/C = 600 secs/200 jobs = 3. 0 secs/job E[VCPU] = CCPU/C = 200 visits/200 jobs = 1. 0 visit/job E[Vslowdisk] = Cslowdisk/C = 2, 000 visits/200 jobs = 10 visits/job E[Vfastdisk] = Cfastdisk/C = 20, 000 visits/200 jobs = 100 visits/job E[SCPU] = BCPU/CCPU = 400 secs/200 visits = 2. 0 secs/visit E[Sslowdisk] = Bslowdisk/Cslowdisk = 100 secs/2, 000 visits = 0. 05 sec/visit E[Sfastdisk] = Bfastdisk/Cfastdisk = 600 secs/20, 000 visits =0. 03 sec/visit Clearly, we have overloaded the fast disk. Optimal balancing is such that – E[Vslowdisk] E[Sslowdisk] = E[Vfastdisk] E[Sfastdisk] or E[Vslowdisk] 0. 05 = E[Vfastdisk] 0. 03 while keeping E[Vslowdisk] + E[Vfastdisk] = 110 – This gives E[Vfastdisk] 69 and E[Vslowdisk] 41 and consequently E[Dslowdisk] = E[Dfastdisk] = 2. 06 secs, and therefore a new value of Dmax = 2. 06 secs – D is now 6. 12 secs and N* becomes 10, which remains smaller than N. Hence, throughput and response time are still dominated by Dmax, and the system improves to X 0. 485 jobs/sec and E[R] = 26. 2 secs 13

• Example (2 c) – Add Another Fast Disk Summary – – – N = X E[T] = X (E[R] + E[Z]) E[R] = N/X – E[Z] (Response Time Law) i = i/ i = i E[Si] = Xi E[Si] (Utilization Law) Xi = E[Vi] X (Forced Flow Law) i = X E[Di] (Bottleneck Law) • Metrics for interactive system with N = 20 and E[Z] = 15 secs: – – – – – E[DCPU] = BCPU/C = 400 secs/200 jobs = 2. 0 secs/job E[Dslowdisk] = Bslowdisk/C = 100 secs/200 jobs = 0. 5 secs/job E[Dfastdisk] = Bfastdisk/C = 600 secs/200 jobs = 3. 0 secs/job E[VCPU] = CCPU/C = 200 visits/200 jobs = 1. 0 visit/job E[Vslowdisk] = Cslowdisk/C = 2, 000 visits/200 jobs = 10 visits/job E[Vfastdisk] = Cfastdisk/C = 20, 000 visits/200 jobs = 100 visits/job E[SCPU] = BCPU/CCPU = 400 secs/200 visits = 2. 0 secs/visit E[Sslowdisk] = Bslowdisk/Cslowdisk = 100 secs/2, 000 visits = 0. 05 sec/visit E[Sfastdisk] = Bfastdisk/Cfastdisk = 600 secs/20, 000 visits =0. 03 sec/visit • If we split the fast disk load across two fast disks, we get – E[Dfastdisk 1] = E[Dfastdisk 2] = 1. 5 jobs/sec – The slow disk becomes the bottleneck which yieds a new value of Dmax = 2. 0 secs – D does not change but since Dmax does, N* = (D+E[Z])/Dmax 10. Still smaller than N, so that the system improves to X = 0. 5 jobs/sec and E[R] = 25 secs 14

• Example (2 d) – Combine All Improvements • – – – N = X E[T] = X (E[R] + E[Z]) E[R] = N/X – E[Z] (Response Time Law) i = i/ i = i E[Si] = Xi E[Si] (Utilization Law) Xi = E[Vi] X (Forced Flow Law) i = X E[Di] (Bottleneck Law) Metrics for interactive system with N = 20 and E[Z] = 15 secs: – – – – – • Summary E[DCPU] = BCPU/C = 400 secs/200 jobs = 2. 0 secs/job E[Dslowdisk] = Bslowdisk/C = 100 secs/200 jobs = 0. 5 secs/job E[Dfastdisk] = Bfastdisk/C = 600 secs/200 jobs = 3. 0 secs/job E[VCPU] = CCPU/C = 200 visits/200 jobs = 1. 0 visit/job E[Vslowdisk] = Cslowdisk/C = 2, 000 visits/200 jobs = 10 visits/job E[Vfastdisk] = Cfastdisk/C = 20, 000 visits/200 jobs = 100 visits/job E[SCPU] = BCPU/CCPU = 400 secs/200 visits = 2. 0 secs/visit E[Sslowdisk] = Bslowdisk/Cslowdisk = 100 secs/2, 000 visits = 0. 05 sec/visit E[Sfastdisk] = Bfastdisk/Cfastdisk = 600 secs/20, 000 visits =0. 03 sec/visit With faster CPU and two fast disks and rebalancing load across three disks, we get – E[DCPU] = 1 sec/job, E[Dfastdisk 1] = E[Dfastdisk 2] = E[Dslowdisk] = 1. 27 jobs/sec D = 4. 8 secs • Where we have used E[Vslowdisk] 0. 05 = E[Vfastdisk] 0. 03, and E[Vslowdisk] + 2 E[Vfastdisk] = 110 – This gives Dmax = 1. 27 secs and N* = (D+E[Z])/Dmax 16 that remains smaller than N – Hence, throughput and response time are still dominated by Dmax, and the system improves to X 0. 787 jobs/sec and E[R] = 10. 4 secs 15