 # Random Variables Continuous and Discrete Probability Distributions Sharif

• Slides: 66 Random Variables Continuous and Discrete Probability Distributions Sharif University of Technology Random Variables • If belongs to the associated field F, 3/5/2021 Sharif University of Technology 2 Random Variable • Definition – If is an event for every x in R, – Random Variable is a finite single valued function that maps the set of all experimental outcomes into the set of real numbers R 3/5/2021 Sharif University of Technology 3 Random Variable • The specification of a random variable can also imply a redefinition of the sample space. Example • we could define as a random variable “the total number of heads. ” • Experiment: Fair coin tossed 3 times, 3/5/2021 Sharif University of Technology 4 Notation • Notation: – RV will always be denoted with uppercase letters – The realized values of the variable (its range) will be denoted by the corresponding lowercase letter. – Thus, the random variable X can take the value x. 3/5/2021 Sharif University of Technology 5 Borel collection • X : r. v, • B represents semi-infinite intervals of the form • The Borel collection B of such subsets of R is the smallest σ-field of subsets of R that includes all semi-infinite intervals of the above form. • if X is a r. v, then is an event for every x. 3/5/2021 Sharif University of Technology 6 Probability Distribution Function(PDF) (Cumulative Distribution Function ) • is said to the Probability Distribution Function associated with the r. v X. • The role of the subscript X is only to identify the actual r. v. 3/5/2021 Sharif University of Technology 7 Properties of a PDF • if g(x) is a distribution function, then – – If then – for all x. 3/5/2021 Sharif University of Technology 8 Properties of a PDF - Proof • The probability distribution function is: – nonnegative – monotone nondecreasing. • Proof - We have: • If then • implies So, 3/5/2021 Sharif University of Technology 9 Properties of a PDF - Proof • Let • Consider the event • Since • We get (mutually exclusive property) • But and hence 3/5/2021 Sharif University of Technology 10 Properties of a PDF - Proof • Thus • But the right limit of x, and hence • i. e. , is right-continuous, justifying all properties of a distribution function. 3/5/2021 Sharif University of Technology 11 Additional Properties • If for some then This follows, since implies is the null set, and for any will be a subset of the null set. • since • The events and are mutually exclusive and their union represents the event 3/5/2021 Sharif University of Technology 12 Additional Properties • Let and • We have: or • Thus the only discontinuities of a distribution function occur at points where 3/5/2021 Sharif University of Technology 13 Example • X is a RV with p. d. f: Find a and P(1< x <2). Example • X is a RV with cdf: a) Compute f(x) and check its properties. b) Find P(− 1 <x<2) using both the pdf and the cdf. 3/5/2021 Sharif University of Technology 14 Continuous-type & Discrete-type R. V • X is said to be a continuous-type r. v if • If is constant except for a finite number of jump discontinuities (piece-wise constant; step-type), then X is said to be a discrete-type r. v. • If is such a discontinuity point, then 3/5/2021 Sharif University of Technology 15 Example • X is a r. v such that Find Solution • For so that F • For so that • at a point of discontinuity we get 3/5/2021 Sharif University of Technology 16 Example • In tossing a coin, Suppose the r. v X is such that Find Solution • For so that • at a point of discontinuity we get 3/5/2021 Sharif University of Technology 17 Example • A fair coin is tossed twice, and let the r. v X represent the number of heads. Find Solution 3/5/2021 Sharif University of Technology 18 Probability Density Function (p. d. f) (Probability Mass Function - for discrete RVs) Since from the monotone-nondecreasing nature of for all x 3/5/2021 Sharif University of Technology 19 Probability Density Function (p. d. f) • X is a continuous type r. v: will be a continuous function • X is a discrete type r. v: has the general form Fig. 3. 5 • represent the jump-discontinuity points in • represents a collection of positive discrete masses, • It is known as the probability mass function (p. m. f ) in the discrete case. 3/5/2021 Sharif University of Technology 20 Probability Density Function (p. d. f) • We have: • Since we can say • the area under in the interval represents the probability. (a) 3/5/2021 (b) Sharif University of Technology 21 Continuous-type Random Variables X can take any value in the real line within a bounded or unbounded interval. • • Normal (Gaussian) Uniform Exponential Gamma Beta Chi-Square Rayleigh 3/5/2021 • • Nakagami – m distribution Cauchy Laplace Student’s t-distribution with n degrees of freedom • Fisher’s F-distribution • Others: Weibull, Erlang, Lognormal, … Sharif University of Technology 22 Normal Distribution Where, • Notation: mean 3/5/2021 variance Sharif University of Technology 23 Normal Distribution The normal distribution is symmetrical around mean. p. d. f 3/5/2021 PDF Sharif University of Technology 24 Normal Table • The Normal Distribution illustrated in the table. • With mean of 0 and a standard deviation of 1. • In examples and exercises where z is used, it is found using the formula (x- μ) / σ • or (value given - mean) / standard deviation 3/5/2021 Sharif University of Technology 25 Using the Normal Table • The shaded area, A, gives the probability that Z is greater than the given value. Example • Calculate probability of: – 0 < z < 0. 11 – 0 < z < 0. 34 – 0 < z < 0. 53 3/5/2021 Sharif University of Technology 26 Using the Table in Reverse • Suppose you want the value of z such that the probability of exceeding z is 33% or 0. 33 Example • Do the above for: – A = 0. 4443 – A = 0. 3778 3/5/2021 Sharif University of Technology 27 Applications • Approximately normal distributions occur in many situations. • When a large number of small effects acting additively and independently. • Example: Measurement errors are often assumed to be normally distributed 3/5/2021 Sharif University of Technology 28 Example • What is the maximum number of people who can occupy a lift? • Knowing that the total weight of 8 people chosen at random follows a normal distribution with • mean of 550 kg and • standard deviation of 150 kg. • So, • What’s the probability that the total weight of 8 people exceeds 600 kg? 3/5/2021 Sharif University of Technology 29 Uniform Distribution • Notation: p. d. f PDF Application • Sampling from arbitrary distributions. (random number generation) • inverse transform sampling method, which uses the cumulative distribution function (CDF) of the target random variable. 3/5/2021 Sharif University of Technology 30 Exponential distribution • Notation: Here, Fig. 3. 9 3/5/2021 Sharif University of Technology 31 Exponential distribution p. d. f 3/5/2021 PDF Sharif University of Technology 32 Exponential Distribution • Assume the occurences of nonoverlapping intervals are independent, and assume: – q(t): the probability that in a time interval t no event has occurred. – x: the waiting time to the first arrival – Then we have: P(x>t)=q(t) – t 1 and t 2 : two consecutive nonoverlapping intervals, 3/5/2021 Sharif University of Technology 33 Exponential Distribution • Then we have: q(t 1) q(t 2) = q(t 1+t 2) • The only bounded solution is: So the pdf is exponential. If the occurrences of events over nonoverlapping intervals are independent, the corresponding pdf has to be exponential. 3/5/2021 Sharif University of Technology 34 Memoryless Property Of Exponential Distribution • Let . Consider the events 3/5/2021 Sharif University of Technology 35 Application • describing the lengths of the inter-arrival times in a homogeneous Poisson processes. – the time it takes before your next telephone call • situations where certain events occur with a constant probability per unit distance: – the distance between mutations on a DNA strand; • Because of memoryless property: – well-suited to model the constant hazard rate in reliability theory. 3/5/2021 Sharif University of Technology 36 Example • Let X denote the time between log-ons to a server in a large corporate network. • Assuming that X has an exponential distribution with minutes, • What is the probability that a log-on occurs within 30 seconds after starting the server? Solution 3/5/2021 Sharif University of Technology 37 Gamma Distribution • Notation: • For integer 3/5/2021 Sharif University of Technology 38 Gamma Distribution p. d. f 3/5/2021 PDF Sharif University of Technology 39 Beta Distribution • Notation: where, • Beta distribution with a=b=1 is the uniform distribution on (0, 1). 3/5/2021 Sharif University of Technology 40 Beta Distribution • B(i, j) with integer values of i and j is the distribution of the i-th order statistic of a sample of i + j − 1 independent random variables uniformly distributed between 0 and 1. • Used to model events which are constrained to take place within an interval defined by a minimum and maximum value. • Used in PERT, CPM and other project management / control systems to describe the time to completion of a task. p. d. f 3/5/2021 PDF Sharif University of Technology 41 Chi-Square Distribution • Notation: p. d. f Applications • chi-square tests for goodness of fit PDF 3/5/2021 Sharif University of Technology 42 Rayleigh • Notation: PDF • when a two-dimensional vector (e. g. wind velocity) has elements that are • normally distributed, • uncorrelated, • with equal variance p. d. f • The vector’s magnitude (e. g. wind speed) will then have a Rayleigh distribution. 3/5/2021 Sharif University of Technology 43 Nakagami – m Distribution p. d. f Applications • Used to model attenuation of wireless signals traversing multiple paths. PDF 3/5/2021 Sharif University of Technology 44 Cauchy Distribution • Notation: p. d. f • The ratio of two independent standard normal random variables is a standard Cauchy variable • It has no mean, variance or higher moments defined. PDF 3/5/2021 Sharif University of Technology 45 Laplace Distribution • The difference between two iid exponential random variables is governed by a Laplace distribution p. d. f PDF 3/5/2021 Sharif University of Technology 46 Student’s t-Distribution • arises in the problem of estimating the mean of a normally distributed population when the sample size is small. p. d. f 3/5/2021 PDF Sharif University of Technology 47 Fisher’s F-Distribution • A random variate of the F-distribution arises as the ratio of two chisquared variates p. d. f 3/5/2021 PDF Sharif University of Technology 48 Discrete-type Random Variables X can take only a finite (or countably infinite) number of values • • Bernoulli Binomial Poisson Hypergeometric Geometric Negative Binomial Discrete-Uniform Polya’s distribution 3/5/2021 Sharif University of Technology 49 Bernoulli Distribution X takes the values (0, 1) and, 3/5/2021 Sharif University of Technology 50 Binomial Distribution • Notation: The probability of k successes in n experiments with replacement (in ball drawing) p. d. f 3/5/2021 PDF Sharif University of Technology 51 Binomial Distribution Example • Assume 5% of a very large population to be green-eyed. • You pick 100 people randomly. • X : The number of green-eyed people • 3/5/2021 Sharif University of Technology 52 Poisson Distribution • Notation: p. d. f 3/5/2021 PDF Sharif University of Technology 53 Poisson Distribution Law Of Rare Events In a Binomial distribution, as • n approaches ∞ • p approaches 0 • np remains fixed at λ > 0 or at least np approaches λ > 0 The Binomial(n, p) distribution approaches the Poisson distribution with expected value λ. 3/5/2021 Sharif University of Technology 54 Poisson Distribution - Applications • The Poisson distribution applies to various phenomena of discrete nature. • The probability of the phenomenon should be constant in time or space. • Examples: The number of… – spelling mistakes one makes while typing a single page. – phone calls at a call center per minute. – times a web server is accessed per minute. – roadkill (animals killed) found per unit length of road. – pine trees per unit area of mixed forest. – stars in a given volume of space. 3/5/2021 Sharif University of Technology 55 Hypergeometric Distribution • The probability of k successes in n experiments without replacement (ball drawing) Application • The classical application of the hypergeometric distribution is sampling without replacement. 3/5/2021 Sharif University of Technology 56 Geometric Distribution • Notation: the geometric distribution is memoryless. 3/5/2021 Sharif University of Technology 57 Example • From past experience it is known that 3% of accounts in a large accounting population are in error. a) What is the probability that 5 accounts are audited before an account in error is found? b) What is the probability that the first account in error occurs in the first five accounts audited? Solution a) b) 3/5/2021 Sharif University of Technology 58 Negative Binomial Distribution • Notation: • The geometric distribution is a special case of the negative binomial distribution • x independent trials each with two possible outcomes: success/failure. • The probability of success, denoted by p, is the same on every trial. • Continues until r successes are observed 3/5/2021 Sharif University of Technology 59 Example • A basketball player is a 70% free throw shooter. • During the season, what is the probability that he makes his third free throw on his fifth shot? • What is the probability that he makes his first free throw on his fifth shot? 3/5/2021 Sharif University of Technology 60 Discrete-Uniform Distribution Example • In experiment of throwing a die, let X be the random variable denoting what number is thrown. P(X = 1) = 1/6 P(X = 2) = 1/6 etc In fact, P(X = x) = 1/6 for all x between 1 and 6. 3/5/2021 Sharif University of Technology 61 Example: Polya’s distribution Distribution • A box contains a white balls and b black balls. • A ball is drawn at random, and it is replaced along with c balls of the same color. • If X represents the number of white balls drawn in n such draws, find the probability mass function of X. 3/5/2021 Sharif University of Technology 62 Solution : probability of drawing k successive white balls : probability of drawing k white balls , followed by n – k black balls 3/5/2021 Sharif University of Technology 63 Example – cont’d • Polya distribution: probability of getting k white balls in n draws. • if draws are done with replacement, then c = 0 and the above formula simplifies to the binomial distribution 3/5/2021 Sharif University of Technology 64 Example – cont’d • if the draws are conducted without replacement, then c = – 1 and it gives • which represents the hypergeometric distribution. 3/5/2021 Sharif University of Technology 65 Example – cont’d • Finally c = +1 gives (replacements are doubled) • This is Polya’s +1 distribution. 3/5/2021 Sharif University of Technology 66