Random Number Generators Why do we need random

Random Number Generators

Why do we need random variables? • random components in simulation → need for a method which generates numbers that are random • examples –interarrival times –service times –demand sizes 040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I 2

terminology • Random Numbers –stochastic variable that meets certain conditions –numbers randomly drawn from some known distribution –It is impossible to “generate” them with the help of a digital computer. • Pseudo-random Numbers –numbers which seem to be randomly drawn from some known distribution –may be generated with the help of a digital computer 040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I 3

techniques for generating random variables –ten-sided die (each throw generates a decimal) –throwing a coin n times • get a binary number between 0 and 2^n-1 –other physical devices –observe substances undergoing atomic decay • e. g. , Caesium-137, Krypton-85, . . . 040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I 4

techniques for generating pseudo-random variables • numbers are generated using a recursive formula –ri is a function of ri-1, ri-2, … –relation is deterministic – no random numbers –if the mathematical relation not known and well chosen it is practically impossible to predict the next number • In most cases we want the generated random numbers to simulate a uniform distribution over (0, 1), that is a U(0, 1)-distribution –there’re many simple techniques to transform uniform (U(a, b)) samples into samples from other well-known distributions 1 040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I 0 1 5

terminology • seed –random number generators are usually initialized with a starting number which is called the “seed” –different seeds generates different streams of pseudorandom numbers –the same seed results in the same stream • cycle length –length of the stream of pseudorandom numbers without repetition 040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I 6

methods for generating pseudo-random variables • all methods usually generate uniformly distributed pseudorandom numbers in the interval [0, 1] –midsquare method (outdated) –congruential method (popular) –shift register (Tausworthe generator) 040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I 7

midsquare method • arithmetic generator –proposed by von Neumann and Metropolis in 1940 s • algorithm –start with an initial number Z 0 with m digits (seed) [m even] –square it and get a number with 2 m digits (add a zero at the beginning if the number has only 2 m-1 digits) –obtain Z 1 by taking the middle m digits –to be in the interval [0, 1) divide the Zi by 10 m –etc…. 040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I 8

midsquare method: example i 0 1 2 3 4 5 6 7 … Zi 7182 5811 7677 9363 6657 3156 9603 2176 … Ui 0. 7182 0. 5811 0. 7677 0. 9363 0. 6657 0. 3156 0. 9603 0. 2176 … (Z 0 = 7182, m = 4) Z i 2 51581124 33767721 58936329 87665769 44315649 9960336 92217609 4734976 … 040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I 9

midsquare method (cont. ) • advantages –fairly simple • disadvantage –tends to degenerate fairly rapidly to zero • example: try Z 0 = 1009 –not random as predictable 040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I 10

Linear Congruential Generator (LCG) • LCGs –first proposed by Lehmer (1951). –produce pseudorandom numbers such that each single number determines its successor by means of a linear function followed by a modular reduction –to be in the interval [0, 1) divide the Zi by m –Z 0 seed –c increment (integer) • Variations are possible a multiplier (integer) m modulus (integer) –combinations of previous numbers instead of using only the last value 040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I 11

a=5 c=3 m = 16 Z 0 = 7 LCG: example i 0 1 2 3 4 5 6 7 … Zi 7 6 1 8 11 10 5 12 … Ui 0. 4375 0. 0625 0. 6875 0. 625 0. 3125 0. 75 … Z 1 = (5*Z 0 + 3) mod 16 = 38 mod 16 = 6 Z 2 = (5*Z 1 + 3) mod 16 = 33 mod 16 = 1 Z 3 = (5*Z 2 + 3) mod 16 = 8 … 040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I 12

properties of LCGs • parameters need to be chosen carefully –nonnegative – 0 < m a<m • disadvantages c<m Z 0 < m –not random –if a and m are properly chosen, the Uis will “look like” they are randomly and uniformly distributed between 0 and 1. –can only take rational values 0, 1/m, 2/m, … (m-1)/m –looping 040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I 13

LCG (cont. ) • looping –whenever Zi takes on a value it has had previously, exactly the same sequence of values is generated, and this cycle repeats itself endlessly. –length of sequence = period of generator –period can be at most m • if so: LCG has full period –LCG has full period iff • the only positive integer that divides both m and c is 1 • if q is a prime number that divides m then q divides a-1 • if 4 divides m, then 4 divides a-1 040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I 14

Linear Feedback Shift Register Generators (LFSR) • developed by Tausworthe (1965) –related to cryptographic methods –operate directly on bits to form random numbers • linear combination of the last q bits –ci constants (equal to 0 or 1, cq = 1) –in most applications only two of the cj coefficients are nonzero –execution can be sped up • modulo 2 equivalent to exclusive-or 040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I 15

LFSR (cont. ) • form a sequence of binary integers W 1, W 2, … –string together l consecutive bits –consider them as numbers in base 2 • transform them into Uis • maximum period 2 q -1 –if l is relatively prime to 2 q-1 LFSR has full period 040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I 16

LFSR : example • r = 3 q=5 b 1 = b 2 = b 3 = b 4 = b 5 = 1 l=4 I 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 15 17 18 bi 1 1 1 0 0 0 1 1 1 0 1 i 1 2 3 Wi (base 2) 1111 1000 1101 Wi (base 10) 15 8 13 Ui 0. 9375 0. 8125 040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I 17

Testing RNGs Empirical Tests 040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I 18

random number generators • methods presented so far… –completely deterministic –Uis appear as if they were U(0, 1) • test their actual quality –how well do the generated Uis resemble values of true IID U(0, 1) • different kinds of tests –empirical tests • based on actual Uis produced by RNG –theoretical tests 040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I 19

Chi-Square test (with all parameters known) • checks whether the Uis appear to be IID U(0, 1) –divide [0, 1] into k subintervals of equal length (k > 100) –generate n random variables: U 1, U 2, . . Un –calculate fj (number of Uis that fall into jth subinterval) –calculate test statistic –for large n χ2 will have an approximate chi-square distribution with k-1 df under the null hypothesis that the Uis are IID U(0, 1) –reject null hypothesis at level ® if 040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I 20

Generating Random Variates

Generating Random Variates • General Approach –Inverse Transformation –Composition –Convolution • Generating Continuous Random Variates –Uniform U(a, b) –Exponential –m-Erlang –Gamma, Weibull –Normal –etc… 040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I 22

Inverse Transformation • generate continuous random variate X –distribution function F (continuous, strictly increasing) • 0 < F(x) < 1 • if x 1 < x 2 then 0 < F(x 1) · F(x 2) < 1 –inverse of F: F-1 • algorithm –generate U ~ U(0, 1) –return X = F-1(U) 040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I 23

Inverse Transformation (cont. ) F(x) 1 U 2 X 2 0 x X 1 • returned value X has desired distribution F 040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I 24

Inverse Transformation (cont. ) • create X according to exponential distribution with mean ¯ • in order to find F-1 set take natural logarithm (base e) 040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I 25

Inverse Transformation for discrete variates • distribution function • probability mass function • algorithm –generate U ~ U(0, 1) –determine smallest possible integer i such that U · F(xi) –return X = xi 040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I 26

Composition • applies then the distribution function F can be expressed as a convex combination of other distribution functions F 1, F 2, . . –we hope to be able to sample from the Fj’s more easily than from the original F –if X has a density it can be written as • algorithm –generate positive random integer J such that P(J = j) = pj –Return X with distribution function FJ 040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I 27

Composition (example) • double-exponential distribution (laplace distribution) –can be rewritten as –where IA(x) is the indicator function of set A • f(x) is a convex combination of –f 1(x) = ex I(-1, 0) and f 2(x) = e-x I[0, 1) –p 1 = p 2 = 0. 5 040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I 28

Convolution • desired random variable X can be expressed as sum of other random variables that are IID and can be generated more readily then X directly • algorithm –generate Y 1, Y 2, … Ym IID each with distribution function G –return X = Y 1 + Y 2 + + Ym • example: m-Erlang random variable X with mean ¯ –sum of m IID exponential random variables with common mean ¯/m 040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I 29

Generate Continuous Variates • Uniform distribution X(a, b) X = a + (b-a)U • Exponential (mean ¯) X = - ¯ ln (1 -U) or X = -¯ ln U • m-Erlang 040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I 30