Jointly distributed Random variables Multivariate distributions Quite often

Jointly distributed Random variables Multivariate distributions

Quite often there will be 2 or more random variables (X, Y, etc) defined for the same random experiment. Example: A bridge hand (13 cards) is selected from a deck of 52 cards. X = the number of spades in the hand. Y = the number of hearts in the hand. In this example we will define: p(x, y) = P[X = x, Y = y]

The function p(x, y) = P[X = x, Y = y] is called the joint probability function of X and Y.

Note: The possible values of X are 0, 1, 2, …, 13 The possible values of Y are also 0, 1, 2, …, 13 and X + Y ≤ 13. The number of ways of choosing the y hearts for the hand The number of ways of choosing the x spades for the hand The number of ways of completing the hand with diamonds and clubs. The total number of ways of choosing the 13 cards for the hand

Table: p(x, y)

Bar graph: p(x, y) y x

Example: A die is rolled n = 5 times X = the number of times a “six” appears. Y = the number of times a “five” appears. Now p(x, y) = P[X = x, Y = y] The possible values of X are 0, 1, 2, 3, 4, 5. The possible values of Y are 0, 1, 2, 3, 4, 5. and X + Y ≤ 5

A typical outcome of rolling a die n = 5 times will be a sequence F 5 FF 6 where F denotes the outcome {1, 2, 3, 4}. The probability of any such sequence will be: where x = the number of sixes in the sequence and y = the number of fives in the sequence

Now p(x, y) = P[X = x, Y = y] Where K = the number of sequences of length 5 containing x sixes and y fives.

Thus p(x, y) = P[X = x, Y = y] if x + y ≤ 5.

Table: p(x, y)

Bar graph: p(x, y) y x

General properties of the joint probability function; p(x, y) = P[X = x, Y = y]

Example: A die is rolled n = 5 times X = the number of times a “six” appears. Y = the number of times a “five” appears. What is the probability that we roll more sixes than fives i. e. what is P[X > Y]?

Table: p(x, y)

Marginal and conditional distributions

Definition: Let X and Y denote two discrete random variables with joint probability function p(x, y) = P[X = x, Y = y] Then p. X(x) = P[X = x] is called the marginal probability function of X. and p. Y(y) = P[Y = y] is called the marginal probability function of Y.

Note: Let y 1, y 2, y 3, … denote the possible values of Y. Thus the marginal probability function of X, p. X(x) is obtained from the joint probability function of X and Y by summing p(x, y) over the possible values of Y.

Also

Example: A die is rolled n = 5 times X = the number of times a “six” appears. Y = the number of times a “five” appears.

Conditional Distributions

Definition: Let X and Y denote two discrete random variables with joint probability function p(x, y) = P[X = x, Y = y] Then p. X |Y(x|y) = P[X = x|Y = y] is called the conditional probability function of X given Y =y and p. Y |X(y|x) = P[Y = y|X = x] is called the conditional probability function of Y given X=x

Note and

• Marginal distributions describe how one variable behaves ignoring the other variable. • Conditional distributions describe how one variable behaves when the other variable is held fixed

Example: A die is rolled n = 5 times X = the number of times a “six” appears. Y = the number of times a “five” appears. y x

The conditional distribution of X given Y = y. p. X |Y(x|y) = P[X = x|Y = y] y x

The conditional distribution of Y given X = x. p. Y |X(y|x) = P[Y = y|X = x] y x

Example A Bernoulli trial (S - p, F – q = 1 – p) is repeated until two successes have occurred. X = trial on which the first success occurs and Y = trial on which the 2 nd success occurs. Find the joint probability function of X, Y. Find the marginal probability function of X and Y. Find the conditional probability functions of Y given X = x and X given Y = y,

Solution A typical outcome would be: x y FFF…FS x-1 y–x-1

p(x, y) - Table y x 1 2 3 4 5 6 7 8 1 0 0 0 0 2 p 2 0 0 0 0 3 4 5 6 p 2 q 2 p 2 q 3 p 2 q 4 0 p 2 q 2 p 2 q 3 p 2 q 4 0 0 0 0 0 0 7 p 2 q 5 p 2 q 5 0 0 8 p 2 q 6 p 2 q 6 0

The marginal distribution of X This is the geometric distribution

The marginal distribution of Y This is the negative binomial distribution with k = 2.

The conditional distribution of X given Y = y This is the geometric distribution with time starting at x.

The conditional distribution of Y given X = x This is the uniform distribution on the values 1, 2, …(y – 1)

Summary Discrete Random Variables

The joint probability function; p(x, y) = P[X = x, Y = y]

Continuous Random Variables

Definition: Two random variable are said to have joint probability density function f(x, y) if

If then defines a surface over the x – y plane

Multiple Integration

A f(x, y) A

If the region A = {(x, y)| a ≤ x ≤ b, c ≤ y ≤ d} is a rectangular region with sides parallel to the coordinate axes: y d c Then A a b x

To evaluate A First evaluate the inner integral Then evaluate the outer integral

y d dy y c a b x = area under surface above the line where y is constant Infinitesimal volume under surface above the line where y is constant

The same quantity can be calculated by integrating first with respect to y, than x. A First evaluate the inner integral Then evaluate the outer integral

y dx d c a x b x = area under surface above the line where x is constant Infinitesimal volume under surface above the line where x is constant

Example: Compute Now

The same quantity can be computed by reversing the order of integration

Integration over non rectangular regions

Suppose the region A is defined as follows A = {(x, y)| a(y) ≤ x ≤ b(y), c ≤ y ≤ d} y d c Then A a(y) b(y) x

If the region A is defined as follows A = {(x, y)| a ≤ x ≤ b, c(x) ≤ y ≤ d(x) } y d(x) c(x) Then A a b x

In general the region A can be partitioned into regions of either type y A 2 A 1 A 3 A A 4 x

Example: Compute the volume under f(x, y) = x 2 y + xy 3 over the region A = {(x, y)| x + y ≤ 1, 0 ≤ x, 0 ≤ y} y (0, 1) x+y=1 (1, 0) x

Integrating first with respect to x than y y (0, 1) (0, y) x+y=1 (1 - y, y) (1, 0) x A

and

Now integrating first with respect to y than x y (0, 1) x+y=1 (x, 1 – x ) (1, 0) (x, 0) A x

Hence

Continuous Random Variables

Definition: Two random variable are said to have joint probability density function f(x, y) if

Definition: Let X and Y denote two random variables with joint probability density function f(x, y) then the marginal density of X is the marginal density of Y is

Definition: Let X and Y denote two random variables with joint probability density function f(x, y) and marginal densities f. X(x), f. Y(y) then the conditional density of Y given X = x conditional density of X given Y = y

The bivariate Normal distribution

Let where This distribution is called the bivariate Normal distribution. The parameters are m 1, m 2 , s 1, s 2 and r.

Surface Plots of the bivariate Normal distribution

Note: is constant when is constant. This is true when x 1, x 2 lie on an ellipse centered at m 1, m 2.

Marginal and Conditional distributions

Marginal distributions for the Bivariate Normal distribution Recall the definition of marginal distributions for continuous random variables: and It can be shown that in the case of the bivariate normal distribution the marginal distribution of xi is Normal with mean mi and standard deviation si.

Proof: The marginal distributions of x 2 is where

Now:

Hence Also and

Finally

and

Summarizing where and

Thus

Thus the marginal distribution of x 2 is Normal with mean m 2 and standard deviation s 2. Similarly the marginal distribution of x 1 is Normal with mean m 1 and standard deviation s 1.

Conditional distributions for the Bivariate Normal distribution Recall the definition of conditional distributions for continuous random variables: and It can be shown that in the case of the bivariate normal distribution the conditional distribution of xi given xj is Normal with: mean standard deviation and

Proof

where and Hence Thus the conditional distribution of x 2 given x 1 is Normal with: and mean standard deviation

Bivariate Normal Distribution with marginal distributions

Bivariate Normal Distribution with conditional distribution

x 2 ( m 1, m 2) Major axis of ellipses Regression to the mean x 1

Example: Suppose that a rectangle is constructed by first choosing its length, X and then choosing its width Y. Its length X is selected form an exponential distribution with mean m = 1/l = 5. Once the length has been chosen its width, Y, is selected from a uniform distribution form 0 to half its length. Find the probability that its area A = XY is less than 4.

Solution:

xy = 4 y = x/2

This part can be evaluated This part may require Numerical evaluation

multivariate distributions k≥ 2

Definition Let X 1, X 2, …, Xk denote k discrete random variables, then p(x 1, x 2, …, xk ) is joint probability function of X 1, X 2, …, Xk if

Definition Let X 1, X 2, …, Xk denote k continuous random variables, then f(x 1, x 2, …, xk ) is joint density function of X 1, X 2, …, Xk if

Example: The Multinomial distribution Suppose that we observe an experiment that has k possible outcomes {O 1, O 2, …, Ok } independently n times. Let p 1, p 2, …, pk denote probabilities of O 1, O 2, …, Ok respectively. Let Xi denote the number of times that outcome Oi occurs in the n repetitions of the experiment. Then the joint probability function of the random variables X 1, X 2, …, Xk is

Note: is the probability of a sequence of length n containing x 1 outcomes O 1 x 2 outcomes O 2 … xk outcomes Ok

is the number of ways of choosing the positions for the x 1 outcomes O 1, x 2 outcomes O 2, …, xk outcomes Ok

is called the Multinomial distribution

Example: Suppose that a treatment for back pain has three possible outcomes: O 1 - Complete cure (no pain) – (30% chance) O 2 - Reduced pain – (50% chance) O 3 - No change – (20% chance) Hence p 1 = 0. 30, p 2 = 0. 50, p 3 = 0. 20. Suppose the treatment is applied to n = 4 patients suffering back pain and let X = the number that result in a complete cure, Y = the number that result in just reduced pain, and Z = the number that result in no change. Find the distribution of X, Y and Z. Compute P[X + Y ≥ Z]

Table: p(x, y, z)

P [X + Y ≥ Z] = 0. 9728

Example: The Multivariate Normal distribution Recall the univariate normal distribution the bivariate normal distribution

The k-variate Normal distribution where

Marginal distributions

Definition Let X 1, X 2, …, Xq+1 …, Xk denote k discrete random variables with joint probability function p(x 1, x 2, …, xq+1 …, xk ) then the marginal joint probability function of X 1, X 2, …, Xq is

Definition Let X 1, X 2, …, Xq+1 …, Xk denote k continuous random variables with joint probability density function f(x 1, x 2, …, xq+1 …, xk ) then the marginal joint probability function of X 1, X 2, …, Xq is

Conditional distributions

Definition Let X 1, X 2, …, Xq+1 …, Xk denote k discrete random variables with joint probability function p(x 1, x 2, …, xq+1 …, xk ) then the conditional joint probability function of X 1, X 2, …, Xq given Xq+1 = xq+1 , …, Xk = xk is

Definition Let X 1, X 2, …, Xq+1 …, Xk denote k continuous random variables with joint probability density function f(x 1, x 2, …, xq+1 …, xk ) then the conditional joint probability function of X 1, X 2, …, Xq given Xq+1 = xq+1 , …, Xk = xk is

Definition – Independence of sets of vectors Let X 1, X 2, …, Xq+1 …, Xk denote k continuous random variables with joint probability density function f(x 1, x 2, …, xq+1 …, xk ) then the variables X 1, X 2, …, Xq are independent of Xq+1, …, Xk if A similar definition for discrete random variables.

Definition – Mutual Independence Let X 1, X 2, …, Xk denote k continuous random variables with joint probability density function f(x 1, x 2, …, xk ) then the variables X 1, X 2, …, Xk are called mutually independent if A similar definition for discrete random variables.

Example Let X, Y, Z denote 3 jointly distributed random variable with joint density function then Find the value of K. Determine the marginal distributions of X, Y and Z. Determine the joint marginal distributions of X, Y X, Z Y, Z

Solution Determining the value of K.

The marginal distribution of X.

The marginal distribution of X, Y.

Find the conditional distribution of: 1. Z given X = x, Y = y, 2. Y given X = x, Z = z, 3. X given Y = y, Z = z, 4. Y , Z given X = x, 5. X , Z given Y = y 6. X , Y given Z = z 7. Y given X = x, 8. X given Y = y 9. X given Z = z 10. Z given X = x, 11. Z given Y = y 12. Y given Z = z

The marginal distribution of X, Y. Thus the conditional distribution of Z given X = x, Y = y is

The marginal distribution of X. Thus the conditional distribution of Y , Z given X = x is