RANDOM VARIABLES EXPECTATIONS VARIANCES ETC 1 Variable Recall

RANDOM VARIABLES, EXPECTATIONS, VARIANCES ETC. 1

Variable • Recall: • Variable: A characteristic of population or sample that is of interest for us. • Random variable: A function defined on the sample space S that associates a real number with each outcome in S. 2

DISCRETE RANDOM VARIABLES • If the set of all possible values of a r. v. X is a countable set, then X is called discrete r. v. • The function f(x)=P(X=x) for x=x 1, x 2, … that assigns the probability to each value x is called probability density function (p. d. f. ) or probability mass function (p. m. f. ) 3

Example • Discrete Uniform distribution: • Example: throw a fair die. P(X=1)=…=P(X=6)=1/6 4

CONTINUOUS RANDOM VARIABLES • When sample space is uncountable (continuous) • Example: Continuous Uniform(a, b) 5

CUMULATIVE DENSITY FUNCTION (C. D. F. ) • CDF of a r. v. X is defined as F(x)=P(X≤x). • Note that, P(a<X ≤b)=F(b)-F(a). • A function F(x) is a CDF for some r. v. X iff it satisfies F(x) is continuous from right F(x) is non-decreasing. 6

Example • • Consider tossing three fair coins. Let X=number of heads observed. S={TTT, TTH, THT, HTT, THH, HTH, HHT, HHH} P(X=0)=P(X=3)=1/8; P(X=1)=P(X=2)=3/8 x F(x) (-∞, 0) 0 [0, 1) 1/8 [1, 2) 1/2 [2, 3) 7/8 [3, ∞) 1 7

Example • Let 8

JOINT DISTRIBUTIONS • In many applications there are more than one random variables of interest, say X 1, X 2, …, Xk. JOINT DISCRETE DISTRIBUTIONS • The joint probability mass function (joint pmf) of the k-dimensional discrete rv X=(X 1, X 2, …, Xk) is 9

JOINT DISCRETE DISTRIBUTIONS • A function f(x 1, x 2, …, xk) is the joint pmf for some vector valued rv X=(X 1, X 2, …, Xk) iff the following properties are satisfied: f(x 1, x 2, …, xk) 0 for all (x 1, x 2, …, xk) and 10

Example • Tossing two fair dice 36 possible sample points • Let X: sum of the two dice; Y: |difference of the two dice| • For e. g. : – For (3, 3), X=6 and Y=0. – For both (4, 1) and (1, 4), X=5, Y=3. 11

Example • Joint pmf of (x, y) x 2 0 1 y 2 3 3 1/36 4 5 1/36 1/18 6 7 1/36 1/18 5 9 1/36 1/18 4 8 1/18 11 1/36 1/18 10 12 1/36 1/18 1/18 Empty cells are equal to 0. e. g. P(X=7, Y≤ 4)=f(7, 0)+f(7, 1)+f(7, 2)+f(7, 3)+f(7, 4)=0+1/18+0=1/9 12

MARGINAL DISCRETE DISTRIBUTIONS • If the pair (X 1, X 2) of discrete random variables has the joint pmf f(x 1, x 2), then the marginal pmfs of X 1 and X 2 are 13

Example • In the previous example, – – 14

JOINT DISCRETE DISTRIBUTIONS • JOINT CDF: • F(x 1, x 2) is a cdf iff 15

JOINT CONTINUOUS DISTRIBUTIONS • A k-dimensional vector valued rv X=(X 1, X 2, …, Xk) is said to be continuous if there is a function f(x 1, x 2, …, xk), called the joint probability density function (joint pdf), of X, such that the joint cdf can be given as 16

JOINT CONTINUOUS DISTRIBUTIONS • A function f(x 1, x 2, …, xk) is the joint pdf for some vector valued rv X=(X 1, X 2, …, Xk) iff the following properties are satisfied: f(x 1, x 2, …, xk) 0 for all (x 1, x 2, …, xk) and 17

JOINT CONTINUOUS DISTRIBUTIONS • If the pair (X 1, X 2) of discrete random variables has the joint pdf f(x 1, x 2), then the marginal pdfs of X 1 and X 2 are 18

JOINT DISTRIBUTIONS • If X 1, X 2, …, Xk are independent from each other, then the joint pdf can be given as And the joint cdf can be written as 19

CONDITIONAL DISTRIBUTIONS • If X 1 and X 2 are discrete or continuous random variables with joint pdf f(x 1, x 2), then the conditional pdf of X 2 given X 1=x 1 is defined by • For independent rvs, 20

Example Statistical Analysis of Employment Discrimination Data (Example from Dudewicz & Mishra, 1988; data from Dawson, Hankey and Myers, 1982) % promoted (number of employees) Pay grade Affected class others 5 100 (6) 84 (80) 7 88 (8) 87 (195) 9 93 (29) 88 (335) 10 7 (102) 8 (695) 11 7 (15) 11 (185) 12 10 (10) 7 (165) 13 0 (2) 9 (81) 14 0 (1) 7 (41) Affected class might be a minority group or e. g. women 21

Example, cont. • Does this data indicate discrimination against the affected class in promotions in this company? • Let X=(X 1, X 2, X 3) where X 1 is pay grade of an employee; X 2 is an indicator of whether the employee is in the affected class or not; X 3 is an indicator of whether the employee was promoted or not • x 1={5, 7, 9, 10, 11, 12, 13, 14}; x 2={0, 1}; x 3={0, 1} 22

Example, cont. Pay grade Affected class others 10 7 (102) 8 (695) • E. g. , in pay grade 10 of this occupation (X 1=10) there were 102 members of the affected class and 695 members of the other classes. Seven percent of the affected class in pay grade 10 had been promoted, that is (102)(0. 07)=7 individuals out of 102 had been promoted. • Out of 1950 employees, only 173 are in the affected class; this is not atypical in such studies. 23

Example, cont. Pay grade Affected class others 10 7 (102) 8 (695) • E. g. probability of a randomly selected employee being in pay grade 10, being in the affected class, and promoted: P(X 1=10, X 2=1, X 3=1)=7/1950=0. 0036 (Probability function of a discrete 3 dimensional r. v. ) • E. g. probability of a randomly selected employee being in pay grade 10 and promoted: P(X 1=10, X 3=1)= (7+56)/1950=0. 0323 (Note: 8% of 695 > 56) (marginal probability function of X 1 and X 3) 24

Example, cont. • E. g. probability that an employee is in the other class (X 2=0) given that the employee is in pay grade 10 (X 1=10) and was promoted (X 3=1): P(X 2=0| X 1=10, X 3=1)= P(X 1=10, X 2=0, X 3=1)/P(X 1=10, X 3=1) =(56/1950)/(63/1950)=0. 89 (conditional probability) • probability that an employee is in the affected class (X 2=1) given that the employee is in pay grade 10 (X 1=10) and was promoted (X 3=1): P(X 2=1| X 1=10, X 3=1)=(7/1950)/(63/1950)=0. 11 25

Describing the Population • We’re interested in describing the population by computing various parameters. • For instance, we calculate the population mean and population variance. 26

EXPECTED VALUES Let X be a rv with pdf f. X(x) and g(X) be a function of X. Then, the expected value (or the mean or the mathematical expectation) of g(X) providing the sum or the integral exists, i. e. , <E[g(X)]<. 27

EXPECTED VALUES • E[g(X)] is finite if E[| g(X) |] is finite. 28

Population Mean (Expected Value) • Given a discrete random variable X with values xi, that occur with probabilities p(xi), the population mean of X is 29

Population Variance – Let X be a discrete random variable with possible values xi that occur with probabilities p(xi), and let E(xi) =. The variance of X is defined by Unit*Unit 30

EXPECTED VALUE • The expected value or mean value of a continuous random variable X with pdf f(x) is • The variance of a continuous random variable X with pdf f(x) is 31

EXAMPLE • The pmf for the number of defective items in a lot is as follows Find the expected number and the variance of defective items. Results: E(X)=0. 99, Var(X)=0. 8699 32

EXAMPLE • Let X be a random variable. Its pdf is f(x)=2(1 -x), 0< x < 1 Find E(X) and Var(X). 33

Laws of Expected Value • Let X be a rv and a, b, and c be constants. Then, for any two functions g 1(x) and g 2(x) whose expectations exist, 34

Laws of Expected Value and Variance Let X be a rv and c be a constant. Laws of Expected Value § E(c) = c § E(X + c) = E(X) + c § E(c. X) = c. E(X) Laws of Variance § V(c) = 0 § V(X + c) = V(X) § V(c. X) = c 2 V(X) 35

EXPECTED VALUE If X and Y are independent, The covariance of X and Y is defined as 36

EXPECTED VALUE If X and Y are independent, The reverse is usually not correct! It is only correct under normal distribution. If (X, Y)~Normal, then X and Y are independent iff Cov(X, Y)=0 37

EXPECTED VALUE If X 1 and X 2 are independent, 38

CONDITIONAL EXPECTATION AND VARIANCE 39

CONDITIONAL EXPECTATION AND VARIANCE (EVVE rule) Proofs available in Casella & Berger (1990), pgs. 154 & 158 40

Example • An insect lays a large number of eggs, each surviving with probability p. Consider a large number of mothers. X: number of survivors in a litter; Y: number of eggs laid • Assume: • Find: expected number of survivors, i. e. E(X) 41

Example - solution EX=E(E(X|Y)) =E(Yp) =p E(Y) =p E(E(Y|Λ)) =p E(Λ) =pβ 42

SOME MATHEMATICAL EXPECTATIONS • Population Mean: = E(X) • Population Variance: (measure of the deviation from the population mean) • Population Standard Deviation: • Moments: 43

SKEWNESS • Measure of lack of symmetry in the pdf. If the distribution of X is symmetric around its mean , 3=0 Skewness=0 44

KURTOSIS • Measure of the peakedness of the pdf. Describes the shape of the distribution. Kurtosis=3 Normal Kurtosis >3 Leptokurtic (peaked and fat tails) Kurtosis<3 Platykurtic (less peaked and thinner tails) 45

Measures of Central Location • Usually, we focus our attention on two types of measures when describing population characteristics: – Central location – Variability or spread 46

Measures of Central Location • The measure of central location reflects the locations of all the data points. • How? With two data points, the central location But if the third fall data With one data point should in point the middle appears on the left hand-side clearly the central between them (in order of the midrange, it should “pull” location is at the point to reflect the location of the central location to the left. itself. both of them). 47

The Arithmetic Mean • This is the most popular measure of central location Sum of the observations Mean = Number of observations 48

The Arithmetic Mean Sample mean Sample size Population mean Population size 49

The Arithmetic Mean • Example The reported time on the Internet of 10 adults are 0, 7, 12, 5, 33, 14, 8, 0, 9, 22 hours. Find the mean time on the Internet. 0 7 22 11. 0 50

The Arithmetic Mean • Drawback of the mean: It can be influenced by unusual observations, because it uses all the information in the data set. 51

The Median • The Median of a set of observations is the value that falls in the middle when the observations are arranged in order of magnitude. It divides the data in half. Example Comment Suppose only 9 adults were sampled Find the median of the time on the internet for the 10 adults of previous example(exclude, say, the longest time (33)) Odd number of observations Even number of observations 0, 0, 5, 5, 7, 7, 8, 8, , 9, 12, 14, 22, 33 0, 330, 5, 7, 8 8 9, 12, 14, 2252 8. 5

The Mode • The Mode of a set of observations is the value that occurs most frequently. • Set of data may have one mode (or modal class), or two or more modes. The modal class 53

The Mode • Find the mode for the data in the Example. Here are the data again: 0, 7, 12, 5, 33, 14, 8, 0, 9, 22 Solution • All observation except “ 0” occur once. There are two “ 0”s. Thus, the mode is zero. • Is this a good measure of central location? • The value “ 0” does not reside at the center of this set (compare with the mean = 11. 0 and the median = 8. 5). 54

Relationship among Mean, Median, and Mode • If a distribution is from a bell shaped symmetrical one, then the mean, median and mode coincide Mean = Median = Mode • If a distribution is asymmetrical, and skewed to the left or to the right, the three measures differ. A positively skewed distribution (“skewed to the right”) Mode < Median < Mean Mode Mean Median 55

Relationship among Mean, Median, and Mode • If a distribution is non symmetrical, and skewed to the left or to the right, the three measures differ. A positively skewed distribution (“skewed to the right”) Mode A negatively skewed distribution (“skewed to the left”) Mean Median Mean < Median < Mode Median 56

Measures of variability • Measures of central location fail to tell the whole story about the distribution. • A question of interest still remains unanswered: How much are the observations spread out around the mean value? 57

Measures of variability Observe two hypothetical data sets: Small variability The average value provides a good representation of the observations in the data set. This data set is now changing to. . . 58

Measures of Variability Observe two hypothetical data sets: Small variability The average value provides a good representation of the observations in the data set. Larger variability The same average value does not provide as good representation of the observations in the data set as before. 59

The Range – The range of a set of observations is the difference between the largest and smallest observations. – Its major advantage is the ease with which it can be computed. – Its major shortcoming is its failure to provide information on the dispersion of the observations between the two end points. But, how do all the observations spread out? The range cannot assist in answering this question ? Range ? ? Smallest observation Largest observation 60

The Variance l l l This measure reflects the dispersion of all the observations The variance of a population of size N x 1, x 2, …, x. N whose mean is defined as The variance of a sample of n observations x 1, x 2, …, xn whose mean is is defined as 61

Why not use the sum of deviations? Consider two small populations: 9 -10= -1 11 -10= +1 8 -10= -2 A measure of dispersion A B 4 Can the sum of deviations agreesofwith this Be Should a good measure dispersion? The sum of deviations observation. is zero for both populations, 8 9 10 therefore, 11 12 is not …but a The good measurements mean measure of both in B arepopulations more dispersed is 10. . . of dispersion. than those in A. 7 10 13 12 -10= +2 Sum = 0 4 -10 = - 6 16 -10 = +6 7 -10 = -3 16 13 -10 = +3 Sum = 620

The Variance Let us calculate the variance of the two popula Why is the variance defined as After all, the sum of squared the average squared deviation? deviations increases in Why not use the sum of squared magnitude when the variation deviations as a measure of of a data set increases!! variation instead? 63

The Variance Let us calculate. Which the sum deviations for both dataof setsquared has a larger dispersion? Data set B is more dispersed around the mean A B 1 2 3 1 3 5 64

The Variance Sum. A = (1 -2)2 +…+(1 -2)2 +(3 -2)2 +… +(3 -2)2= 10 Sum. B = (1 -3)2 + (5 -3)2 = 8 Sum. A > Sum. B. This is inconsistent with observation that set B is more dispers A B 1 2 3 1 3 5 65

The Variance However, when calculated on “per observation” basis (variance), the data set dispersions are properly ranked. s. A 2 = Sum. A/N = 10/5 = 2 s. B 2 = Sum. B/N = 8/2 = 4 A B 1 2 3 1 3 5 66

The Variance • Example – The following sample consists of the number of jobs six students applied for: 17, 15, 23, 7, 9, 13. Find its mean and variance • Solution 67

The Variance – Shortcut method 68

Standard Deviation • The standard deviation of a set of observations is the square root of the variance. 69

The Coefficient of Variation • The coefficient of variation of a set of measurements is the standard deviation divided by the mean value. • This coefficient provides a proportionate measure of variation. A standard deviation of 10 may be perceived large when the mean value is 100, but only moderately large when the mean value is 500 70

Percentiles • Example from http: //www. ehow. com/how_2310404_calculate-percentiles. html • Your test score, e. g. 70%, tells you how many questions you answered correctly. However, it doesn’t tell how well you did compared to the other people who took the same test. • If the percentile of your score is 75, then you scored higher than 75% of other people who took the test. 71

Sample Percentiles • Percentile – The pth percentile of a set of measurements is the value for which • p percent of the observations are less than that value • 100(1 -p) percent of all the observations are greater than that value. 72

Sample Percentiles • Find the 10 percentile of 6 8 3 6 2 8 1 • Order the data: 1 2 3 6 6 8 8 • 7*(0. 10) = 0. 70; round up to 1 The first observation, 1, is the 10 percentile. 73

• Commonly used percentiles – First (lower) quartile, Q 1 = 25 th percentile – Second (middle) quartile, Q 2 = 50 th percentile – Third quartile, Q 3 = 75 th percentile – Fourth quartile, Q 4 = 100 th percentile – First (lower) decile = 10 th percentile – Ninth (upper) decile = 90 th percentile 74

Quartiles and Variability • Quartiles can provide an idea about the shape of a histogram Q 1 Q 2 Positively skewed histogram Q 3 Q 1 Q 2 Q 3 Negatively skewed histogram 75

Interquartile Range • Large value indicates a large spread of the observations Interquartile range = Q 3 – Q 1 76

Paired Data Sets and the Sample Correlation Coefficient • The covariance and the coefficient of correlation are used to measure the direction and strength of the linear relationship between two variables. – Covariance - is there any pattern to the way two variables move together? – Coefficient of correlation - how strong is the linear relationship between two variables 77

Covariance x ( y) is the population mean of the variable X (Y). N is the population size. x (y) is the sample mean of the variable X (Y). n is the sample size. 78

Covariance • If the two variables move in the same direction, (both increase or both decrease), the covariance is a large positive number. • If the two variables move in opposite directions, (one increases when the other one decreases), the covariance is a large negative number. • If the two variables are unrelated, the covariance will be close to zero. 79

Covariance • Compare the following three sets xi yi (x – x) (y – y) (x – x)(y – y) 2 6 7 13 20 27 -3 1 2 -7 0 7 21 0 14 x=5 y =20 Cov(x, y)=17. 5 xi yi (x – x) (y – y) (x – x)(y – y) 2 6 7 27 20 13 -3 1 2 7 0 -7 -21 0 -14 x=5 y =20 Cov(x, y)=-17. 5 xi yi 2 6 7 20 27 13 Cov(x, y) = -3. 5 x=5 y =20 80

The coefficient of correlation – This coefficient answers the question: How strong is the association between X and Y. 81

The coefficient of correlation +1 Strong positive linear relationship COV(X, Y)>0 r or r = 0 or No linear relationship COV(X, Y)=0 COV(X, Y)<0 -1 Strong negative linear relationship 82

The Coefficient of Correlation • If the two variables are very strongly positively related, the coefficient value is close to +1 (strong positive linear relationship). • If the two variables are very strongly negatively related, the coefficient value is close to -1 (strong negative linear relationship). • No straight line relationship is indicated by a coefficient close to zero. 83

The Coefficient of Correlation 84

Correlation and causation • Recognize the difference between correlation and causation — just because two things occur together, that does not necessarily mean that one causes the other. • For random processes, causation means that if A occurs, that causes a change in the probability that B occurs. 85

Correlation and causation • Existence of a statistical relationship, no matter how strong it is, does not imply a cause-and-effect relationship between X and Y. for ex, let X be size of vocabulary, and Y be writing speed for a group of children. There most probably be a positive relationship but this does not imply that an increase in vocabulary causes an increase in the speed of writing. Other variables such as age, education etc will affect both X and Y. • Even if there is a causal relationship between X and Y, it might be in the opposite direction, i. e. from Y to X. For eg, let X be thermometer reading and let Y be actual temperature. Here Y will affect X. 86