Probability Statistics Lecture 8 1 1 Reminding Probability

Probability &Statistics Lecture 8 1

1. Reminding: Probability Density Function (pdf); Cumulative Distribution Function (cdf). 2. Relation between the Binomial, Poisson and Normal distributions 3. Measures of Central Tendency: Expectation, Variance, higher moments 4. Work in groups, Mathematica 2

Topic 1. Probability distribution function also called “cumulative distribution function” ( CDF) Lect. 7 (equation numbers start with 7 ). 1. <this is transferred from Continuous random variable From the “outside”, random distributions are well described by the probability distribution function (we will use CDF for short) F(x) defined as This formula can also be rewritten in the following very useful form: 3

To see what the distribution functions look like, we return to our examples. a. The uniform distribution (7. 15): Before solving the problems involving integration, please read the following file : Self-Test and About the integrals Using the definition (7. 16) and Mathematica, try to find F(x) for the uniform distribution. Prove that F(x)=0 for x a; (x-a)/(b-a) for a x b; 1 for x>b. Draw the CDF for several a and b. Consider an important special case a=0, b=1. How is it related to the spinner problem? To the balanced die? b. The exponential distribution (7. 17): 4

c. (Gamma)- and chi-square distributions We already introduced the 2 -parameter Gamma distribution discussing the maximum likelihood. Its pdf is Please plot f(x) (7. 18) corresponding CDF for A specific case of Gamma distribution with describes so called chi-square distribution with v degrees of freedom. This distribution is very important in many statistical applications, e. g. For the hypothesis testing. 5

d. The standard and general normal distribution fsnd(x)=(2 )-1/2 exp(-x 2/2) fnd(x)=(2 2)-1/2 exp[-(x- )2/2 2] (7. 19) (7. 20’) Using Mathematica and Eq. (7. 12), find F[x] for the SND, ND. Use NIntegrate[f[t], {t, - , x}] and Plot[…] functions. 2. CDF for discrete random variables For discrete variables the integration is substituted by summation: It is clear from this formula that if X takes only a finite number of values, the distribution function looks like a stairway. 6

F(x) p(x 4) 1 p(x 3) p(x 2) p(x 1) x 1 x 2 x 3 x 4 x 7

Topic 2 Relation between binomial and normal distributions. If n is large and if neither p nor q=1 -p are too close to zero, the binomial distribution can be closely approximated by the normal distribution with = np and =(npq)1/2. Here p is the probability function for the binomial distribution and X is the random variable (the number of successes) 8

Let’s now open Lect 8/Mathematica/Lect 8_Bin. Pois. Normal. nb and run some examples. Topic 3: Measures of Central tendency When a large collection of data is considered, as in a census or in a massive blood test, we are usually interested not in individual numbers but rather in certain descriptive numbers such as the expected value and the variance. 9

Definition of mathematical expectation (a) Descrete random variables A very important concept in probability and statistics is the mathematical expectation or expected value, or briefly the expectation of a random variable. For a discrete random variable X having the possible values x 1, x 2, …xn E(X)= x 1 p(x 1)+ x 2 p(x 2) + …+ xnp(xn)= This can also be rewritten as (8. 3’) where the summation in rhs is taken over all the points of the sample space A special case of (8. 3) is the uniform distribution where the probabilities are equal: 10

Example What is the expectation for the balanced die? Roll one die an Let X be the number that appears. P(X=x) = 1/6, X = 1, 2, 3, 4, 5, 6. EX = (1/6) (1 +2+… + 6) = (1/6)*[6(6+1)/2]= 3. 5. 11

Expectation for a function of the random variable The functions of Random variables are very important in various applications. We will only briefly mention this important concept. A simple example is the kinetic energy mv 2/2 which is a quadratic function of the random variable v, the velocity of a particle. Theorem Let X be a discrete random variable with probability function p(X) and g(X) be a real-valued function of X. Then the expected value of g(X) is given by E[g(X)] = 12

b. Continuous random variable For a continuous random variable X having density function f(x), the expectation of X is defined as The expectation of X is often called the mean of X and is denoted by X, or simply . If g(x) is a function of random variable x, then the expected value can be found as 13

Some Examples (check the results at home) 1. Suppose that a game is played with a single fair die. Assuming that the player wins $20 for X=2 (the face 2 is up), $40 if X=4 and -$30 (loses) if X=6. In addition, he neither wins nor loses if any other face turns up. Find the expected sum of money to be won. 2. The density function of a random variable X is given by f(x)=x/2 for 0<x<2, and 0 otherwise. Find the expected value of X. 3. The same with f (x) = 4 x 3 for 0<x<1, 0 otherwise. 14

Some Examples (answers) 1. Suppose that a game is played with a single fair die. Assuming that the player wins $20 for X=2 (the face 2 is up), $40 if X=4 and -$30 (loses) if X=6. In addition, he neither wins nor loses if any other face turns up. Find the expected sum of money to be won. Solution: X ($) p 20 1/6 40 -30 1/6 0 1/2 15

Example 2 The density function of a random variable X is given by f(x)=x/2 for 0<x<2, and 0 otherwise. Find the expected value of X. Solution Let’s use Eq. 8. 5, , and Mathematica: The answer: 4/3 16

The interpretation of the expected value In statistics, one is frequently concerned with the average values. Average = (Sum of entries) : (number of entries) For any finite experiment, where the number of entries is small or moderate, the average of the outcomes is not predictable. However, we will eventually prove that the average will usually be close to E(X) if we repeat the experiment a large number of times. Let us illustrate these observations using the Lect 8_Average. Vs. Expectation. nb 17

Expectation of a function of random variable (see for details G&S, 6. 1, pp. 229 -230. Note that they use notation m(x) (instead of p(x) ) for the probability function) This discussion can be considered a “proof” of the equation (8. 3’’). Suppose that f(X) is a real value function of the discrete random variable X. How to determine its expected value? Let’s consider the example 6. 6 of G&S, where X describes the outcomes of tossing three coins, and Y , the function of X, is the number of “runs”. The sequence of the identical consecutive outcomes (such as HH, or HHH, or TT) is considered as one run (see 6. 6 for details). Three coin experiment can be described in terms of X and Y by the following table: 18

HHT 2 Notice that this table describes indeed Y as a function of X by mapping all possible outcomes x in the real numbers 1, 2 or 3. The fact that x are the “three letter words” should not be confusing. They can equally be described as numbers. Using, for instance, a convention HTH 3 H->1 and T->2 we will get HHH-> 111, THT->212, etc. HTT To define the expected value of Y we can build a probability function p(Y). It can be done with the help of the same table, by grouping the values of X corresponding a common Y-value and adding their probabilities: p(Y=1) = 2/8= ¼; p(2) = ½; p(3) = ¼. X Y HHH 1 2 THH 2 THT 3 TTH 2 TTT 1 Now we can easily find the expected value of Y: E(Y) = 1*1/4 + 2*1/2 + 3*1/4 = 2. 19

X Y HHH 1 HHT 2 HTH 3 HTT 2 THH 2 THT 3 TTH 2 TTT 1 But what if we did not group the values of X with a common Y-value, but instead simply multiplied each value of Y(X) by the corresponding probability of X. In other words, we multiply each value from the right column by the probability of the corresponding outcome from the left column (=1/8): 1*(1/8) +2*{1/8) +3*(1/8) + 2*(1/8) + 1*(1/8) = 16/8 = 2. This illustrates the following general rule: If X and Y are two random variables, and Y can be written as a function of X, then one can compute the expected value of Y using the distribution function of X. In other words, 20

The sum of two random variables and its expected values Sometimes, the sum of random variables is easy to define. For instance, the definition is straightforward with dies or coins: One simply adds up the corresponding numbers (0 or 1 for the coins, 1 through 6 for the dice). But what if X= die, and Y= coin. What does X+Y mean in this case? In such a case it’s reasonable to define the “joint” random variable Z ={X, Y} whose outcomes are the ordered pairs {x, y} where x = {0, 1}, and Y= {1, 2, 3, 4, 5, 6}. Each outcome, such as {0, 5} or {1, 4}, has now a probability p = 1/12 and all the characteristics of the distribution can be easily obtained (see Example 6. 7 from G&S, p. 230) E(Z=X+Y) = (1/12) ({0, 1}+{0, 2} +… + {0, 6})+ (1/12) ({1, 1}+{1, 2} +… + {1, 6}) = (1/12) {0, 21} + (1/12){6, 21} = {1/2, 3. 5}. 21

Theorem: Let X and Y be random variables with finite expected values. Then, E(X+Y)= E(X) + E(Y) ; E(X-Y)= E(X) - E(Y) (8. 7) and if c any constant, then E(c. X)=c. E(X) (8. 8) The prove is quite simple, and can be found in Grinstead&Snell, p. 231. Let’s discuss (8. 7). We can consider the random variable X+Y as a result of applying function f(x, y) = x + y to a joined random variable (X, Y). Then according to (8. 6) we have: In plain language these conditions mean that the probability that X = xi while Y takes any of allowed values (or regardless the value of Y) is given by the 22 probability function of X, and visa versa.

An Important Comment: In the derivation of (8. 7) it is not assumed that the summands were mutually independent. The fact that the expectations add, whether or not the summands are mutually independent, is sometimes referred to as the First Fundamental Mystery of Probability. 23

b. The variance and standard deviation. While the expectation of a random variable describes its “center”, the variance describes how strongly the distribution is localized near its center. In other words, the variance (or standard deviation) is a measure of the dispersion, or scatter, of the values of the random variable about the mean value. Let’s start with an example of the normal distribution 24

25

Let us now define the variance and have some practice Var(X) = E[(X- )2] (8. 6) The positive square root of variance is called “standard deviation” and is given by Where no confusion can result, the index in standard deviation is dropped. Remember: Var = 2. 26

Discrete distribution Continuous distribution 27

Some theorems on variance Their proofs can be found in G&S, 6. 2 μ Theorem 1. Theorem 2. If c any constant, Var(c. X)= c 2 Var(X). Theorem 3 If X and Y are independent random variable, then Var(X+Y)=Var(X)+Var(Y) Var(X-Y)=Var(X)+Var(Y) Let’s prove Theorem 3. 28

Theorem 3 If X and Y are independent random variable, then Var(X+Y)=Var(X)+Var(Y) Var(X-Y)=Var(X)+Var(Y) Prove We will use the fact that for the independent X and Y, E(XY) = E(X)E(Y). It can be proven using the definition of independence : P(XY) = P(X)P(Y). Let E(X) = a and E(Y) = b. Var(X+Y) = (according to Theorem 1) E[(X+Y)2 ]– (a + b)2 = E(X 2) + E(2 XY) + E(Y 2) – a 2 - 2 ab – b 2= = (E(X 2) - a 2) + (E(Y 2) - b 2) =(using once again Theorem 1) Var(X) + Var(Y). Var(X+Y) = E((X-Y)2) – (a - b)2 = E(X 2) - E(2 XY) + Y(Y 2) – a 2+ 2 ab – b 2= = (E(X 2) - a 2) + (E(Y 2) - b 2) = Var(X) + Var(Y). As promised, we used that E(2 XY) = E(2 X)E(Y) = 2 ab. 29

Working in groups 1 Expectation and Variance for Poisson distribution (open Lect 8/Lect 8_Class. Pract_Expect. Var_for. Students. nb) (1) Finding Expectation and Variance for the Poisson (discrete) and Exponential (continuous) distributions. (2) Find (a) expectation, (b) variance and (c) the standard deviation of the sum obtained in tossing of two fair dice. 30