Lecture 10 Queuing Theory Queueing Analysis l Jobs

Lecture 10: Queuing Theory

Queueing Analysis l Jobs serviced by the system resources l Jobs wait in a queue to use a busy server queue server

Queueing Network Models l Open Network Model Jobs arrive in the system from an external resource and depart the system l Closed Network Model Has a fixed number of jobs that circulate among various queues and servers

Queueing Analysis l s m l l l l l r w q n U a d average arrival rate ( (number of jobs arrive in time T) / T ) average service time average service rate (1/s) traffic intensity ( /m) average response time (waiting time + service time) average waiting time in queue average number of jobs in the system (waiting & being serviced) utilization (busy time / total time) number of arrivals within interval T number of departures within interval T

Operational Analysis Job arrival rate ( ) Average arrival rate Average service rate Computer system = Job service rate (m) (requests/s) s – average service time

Operational Analysis Job arrival rate ( ) Computer system Average arrival rate = Utilization = U= Job service rate ( ) (requests/s) Tb – busy time d – number of departures during the interval s – average service time U=s Utilization Law

Operational Analysis Ex: A single disk system Average time to service each request is 600 ms. What is the maximum possible request rate it can tolerate? Utilization = 100% U = s 1 = (600 x 10 -6) max= 1667 requests/s

Operational Analysis Utilization U=s U= = Traffic intensity >1 arrival rate > service rate number of jobs in the queue increase without bound <1 arrival rate < service rate stable system

Operational Analysis Little’s Law queue server w - average waiting time in queue D – total waiting time of all jobs a – number of arrivals w = D/a D=wa=w T = a/T q - average number of jobs in queue D=q. T q=w Little’s Law

Operational Analysis queue q= w U= s server n= r

Operational Analysis queue server n= r n - average number of jobs in the system (waiting & being serviced) r – average response time of the system (waiting time + service time)

Operational Analysis Ex: Average response time of a web server is 11 ms. Request rate is 450 requests/s What is the average number of requests that are buffered in the system? = 450 r = 11 ms n= r = 450 x 0. 011 = 4. 95 requests

Stochastic Analysis Kendall’s Notation A/S/c A – arrival process most common arrival process is Poisson process: interarrival times follow exponential distribution is memoryless S – service process Poisson service process is the most common c – number of servers

Stochastic Analysis Interarrival time: Time between successive jobs arriving at the queue Ex: M/M/1 Single queue Exponential interarrival time (M) Exponential service time (M) Number of servers (1) queue server

M/M/1 Markov Chain State transition diagram of a birth-death process:

M/M/1 Pn-1 Flow Equations Pn-1 Pi – probability of being in state i Flow from state n-1 to state n: Flow from state n to state n-1: Pn Pn-1 = Pn ? Pn = ( / ) Pn-1 = n P 0

M/M/1 when <1 it converges to 1 -

queue M/M/1 Average number of jobs in the system: E[n] = = = Average response time (r): n = r r = n/ r= = = server

queue server M/M/1 Average number of jobs waiting in the queue (q): E[n] = q= (1 job is being serviced) = = = Average time a job spends waiting in the queue (w): w = average time a job waits in the system – average service time w= =

queue M/M/1 Probability of finding k or more jobs in the system: Pr(K k) = = server

queue M/M/1 Average utilization of the system (U): U = probability of finding one or more jobs in the system = = server

M/M/1 Average number of jobs in the system: E[n] = n. Pn = /(1 - ) Response time: r= = Average number of jobs waiting in the queue: Average waiting time in the queue: q = r 2/(1 -r) w = r/m(1 -r) Probability of finding k or more jobs in the system: Pr(K≥k) = rk

M/M/1 Ex:

M/M/1 Ex: In a web server, the time between requests follow exponential distribution with mean time between requests of 8 ms. Time required by the server to process each request is also exponential distribution with average service time 5 ms. = 1/8 = 0. 125 requests/ms = 1/5 = 0. 2 jobs/ms a. What is the average response time observed? r = 1/( - ) = 13. 33 ms

M/M/1 b. How much faster must the server process requests to halve this average response time? r = 1/( - ) = 1/r + = 1/(13. 33/2) + 0. 125 = 0. 275 jobs/ms (0. 275 -0. 2)/0. 2 37. 5% c. How many queue entries are necessary to buffer requests so that no more than one request per billion is lost? P(K k) 10 -9 P(K k) = k = 0. 625 k 10 -9 k 45

M/M/c queue server M/M/c . . . c – number of servers – job arrival rate n – jobs in the system (both on server & in queue) m – service rate for each server

M/M/c If all servers are busy, mean service rate is: c If some servers are idle, mean service rate is: n (n<c)

M/M/c Probability of finding n jobs in the system: Pn = Total traffic intensity: = (utilization of each server)

M/M/c P 0 =

M/M/c Probability of all servers being busy (c or more jobs in the system): k = Pr(K≥c) = = Average total number of jobs in the system: E[n] = = =

M/M/c Average number of jobs waiting in the queue: q= = Response time: r= = Average time jobs wait in the queue: w= =

M/M/c Ex: How does the response time in M/M/1 queue example change if the number of servers in the system is 4? mean time between requests of 8 ms average service time 5 ms M/M/4 system = 1/8 = 0. 125 requests/ms = 1/5 = 0. 2 jobs/ms

M/M/c = 1/8 = 0. 125 requests/ms = 1/5 = 0. 2 jobs/ms = /(c ) = 0. 125/(4 x 0. 2) = 0. 1563 r= = P 0 = = 0. 5352 = 0. 0040326 r = 5. 01 ms (13. 33 - 5. 01)/13. 33 = 62% improvement

M/M/1 Ex 1: A communication channel is operating at a transmission rate of 1000 bps. To the channel arrive packets according to a Poisson process with rate λ=100 packets per second. The packets have an exponentially distributed length with a mean of v= 5 000 bits. We assume that the channel can be modeled as an M/M/1 system with queuing discipline FCFS (First- Come- First- Served). Calculate a) μ b) ρ c) aver. service time d) aver. number of packets in the system e) aver. time spent in the system f) aver. waiting time in queue g) aver. number of packets in the queue Answer: a) μ =200 e) r =1/100 s b) ρ =1/2 f) w =1/200 c) s =1/200 s g) q =1/2 d) E[n] =1

M/M/1 Ex 2: Jobs arriving at an M/M/1 system according to a Poisson process with an average rate of 8 jobs per second. The Service rate is μ=10 jobs per second. Find a) The offered load b) The probability that the system is idle (no jobs in the system) c) The probability that there are exact 2 jobs in the system. d) Average number of jobs in the system e) Average number of jobs in the queue Answer: a) ρ = 0. 8 e) q= 3. 2 b) P 0 =0. 2 c) P 2 = 0. 128 d) E[n] =4

M/M/c Ex 3: A system can be modeled as an M/M/c queuing system with c=5 servers, arrival rate of λ=8 jobs per second and service rate of μ=2 jobs per second. Find a) The probability that all servers are idle (free). b) The probability that all servers are busy. c) Average number of jobs in the system d) Average waiting time spent in the system by a job e) Average number of jobs in the queue f) Average waiting time spent in the queue by a job Answer: a) P 0 = 0. 013158 b) = 0. 561404 c) E[n] =6. 245614 d) r =0. 780702 e) q = 2. 245614 f) w = 0. 280702

M/M/c wrong Ex 3: A system can be modeled as an M/M/c queuing system with c=5 servers, arrival rate of λ=8 jobs per second and service rate of μ=2 jobs per second. Find a) The probability that all servers are idle (free). b) The probability that all servers are busy. c) Average number of jobs in the system d) Average waiting time spent in the system by a job e) Average number of jobs in the queue f) Average waiting time spent in the queue by a job Answer: a) P 0 = 0. 013158 b) = 0. 554113 c) n=6. 216450 d) r =0. 777056 e) q =2. 216450 f) w=0. 277056