Basic Probability Theory Probability Appears Everywhere Probability Appears

Basic Probability Theory

Probability Appears Everywhere

Probability Appears Everywhere • Happening of unexpected events around us is not uncommon. It therefore makes sense to abandon the idea of a world with certainties and accept a world where we associate likelihood to each event. • Probability is a quantitative measure of likelihood of an event. We all have a fair understanding of : – “Probability of getting a head in a toss of a fair coin is 1/2”. – “Probability of a human getting infected by HIV during blood transfer is 0. 00001” – “Probability that six appears in two consecutive throws of a fair dice is 1/36”. • For each such random experiment, there is a well defined set of all possible events (outcomes) which the experiment can results in.

Sample Space Ω • The possible outcomes of an experiment are called the elementary events. • Sample space : = set of all elementary events. • Event is a set of outcomes – a set of elementary events (a subset of the sample space) • If the subset is a singleton, then the event is an elementary event – Ω = { head, tail } – Ω = { 1, 2, 3, 4, 5, 6 }

Examples 1. The event: {toss of a die is even}. – Denote this event as { 2, 4, 6 }. 2. We flip a coin twice. – Sample space = { HH, HT, TH, TT }. – The subset { HT, TH, TT } is one of the 16 events defined by this sample space, namely “the event that we get at least one T”.

Probability Distribution • A sample space Ω comes with a probability distribution: a mapping P: 2Ω → R such that: • 1. P(A) ≥ 0 for all events A Ω • 2. P(Ω) = 1 • 3. P(A U B) = P(A) + P(B) for any two events A, B Ω that are mutually exclusive: A ∩ B =

Non-intersecting events • A U B = the event in which A or B or both occur • A and B are mutually exclusive if A and B cannot occur at the same time: P(A ∩ B) = • Elementary events are mutually exclusive. • Example: a die cannot be both 4 and 6 at the same time. • Examples: – The probability for the event {2, 4, 6} is 1/6+1/6=½. A – The probability of the die being even or odd is 1 • Therefore, the sum of P(w) for all elementary events w should be 1. B

Random Variable • Given a probability space, we can have random variables: the outcome of a coin toss, for example, is a variable which gets a random value. • A random variable is a function that assigns a real number to each elementary event. (Note that it does not assign a number to a nonelementary event. ) • discrete random variables: random variables whose possible values come from a discrete set

Expectation • The Expectation is the average or mean of a random variable. • The expected value of X is denoted by E(X) = Σ P(X=m) m • For example: The expected value of a toss of a die is 1/6 +2/6+ 3/6+4/6+ 5/6+ 6/6 = 21/6=3. 5 • The expected value of a toss of a fair coin is • ½*0+1/2*1=1/2. • The expected number of heads coin which has P(tail)=1/3, P(head)=2/3 is 2/3. • Let X be the random variable which is the sum of two tosses of a die. What is its expectation?

Markov Inequality • What is the probability that a non-negative random variable is much greater than its expectation? • The Russian mathematician Andrey Markov gave an upper bound for this question. • Let X is any non negative random variable and a > 0, then

Markov Inequality - Proof

Intersection of events A ∩ B • The intersection of events A and B is the event in which both A and B occur. • Example: • A = the event in which the toss of a die is even, P(A)=½. • B = the event in which the toss is at most 3, P(B)=1/2. • P(A ∩ B) = P(Toss is both even and less than 3) = P(Toss=2) = 1/6. A A and B B

Union of events A U B • The union of events A and B is the event in which A or B or both occur. • In words: The probability of {A or B} is found by adding their individual probabilities, then subtracting the probability of both (which has been counted twice). • In symbols: P(AUB)=P(A)+P(B)-P(A and B) A B

Conditional Probability: B|A • How does the probability of an event B change, given that we know that an event A has occurred ? It is the proportion of {A and B} out of A. • P(B|A)= Pr(A and B)/Pr(A). A A and B B

Example • Suppose we know that the toss of the die was odd. What is the probability that it is 3, given that it is odd? A is the event of the toss being Odd. Pr(A)=1/2. B is the event that the toss is 3. P(B)=1/6. • P({3} given {odd})= P( {3} {odd} )/P({odd}) = P({3})/P({odd})=1/3. A A and B B

Independent events: P(A ∩ B) = P(A)P(B). • If P( B|A )=P(B) we say that the event B is independent of A. • In the independent case we have on one hand what we had in the last slide: • P(B|A)=P(A ∩ B) / P(A). • And because of independence: P(B|A)= P(B). • Hence, for independent events: P(A ∩ B) = P(A)P(B).

Example • A Toss of two fair coins. • Pr(0, 0)=Pr(1, 0)=Pr(0, 1)=Pr(1, 1)=1/4. • Let A be the event that the first coin is 0. Pr(A)=Pr(0, 0)+Pr(0, 1)=1/2 • Let B be the event that the second coin is 1. Pr(B)=Pr(0, 1)+Pr(1, 1)=1/2 Pr(A B) =Pr(0, 1)=1/4 Pr(B|A)=Pr(A and B)/Pr(A) =(1/4)/(1/2)=1/2. And indeed, this is equal to Pr(B), and we have Pr(A B)=pr(B)Pr(A) Hence, the events A and B are independent. This is true for any value that the first and second coin can take • The random variables which are the value of the first and second coin toss are independent random variables. • • •

Linearity of Expectation • For any two random variables X and Y • E(X+Y)=E(X)+E(Y) • (whether they are independent or not!) The average behaves linearly. We will not prove it here…

The Complement of an event • The complement of event A is the event {A does not occur}. • Example, the complement of the even {2, 4, 6} (the toss is even) is {1, 3, 5} (the Toss is odd). The probabilities of an event and its complement add to 1: P(A) + P(not A) = 1 • Example: • A fact: 37% of gingis have type O blood. • What is the chance a gingi will not have type O blood? Answer: 100% - 37% = 63% do not have type O blood.

The Complement • Note: • The complement of right wing people is NOT-Right wing as opposed to left wing. not A • There may be another political party. A B