Introduction to probability Stat 134 FAll 2005 Berkeley
Introduction to probability Stat 134 FAll 2005 Berkeley Lectures prepared by: Elchanan Mossel Yelena Shvets Follows Jim Pitman’s book: Probability Sections 1. 4 -1. 5 5/19/2021
Three draws from a magic hat. 5/19/2021
draws‘ 3 from a magic hat: Space. Three of possible draws’ from the Note: Draws are without replacement 5/19/2021
Three draws from a magic hat. What is the chance that we get an on the 2 nd draw, if we got a on the 1 st draw? 5/19/2021
Three draws from a magic hat. ** p p * * p p P P(*I*|R**)=1/2 5/19/2021
Three draws from a magic hat. What is the chance that we get an draw? on the 1 st draw, if we got a on the 2 nd 5/19/2021
Three draws from a magic hat. ** p p * * pp P P(I**|*R*)=1/2 5/19/2021
Counting formula. Under the uniform distribution: P(A|B)= #(AB)/#(B) A w 1 X AB X w 2 w 3 X w 4 X w 5 X w 6 B X X X 5/19/2021
Frequency interpretation In a long sequence of trials, among those which belong to B, the proportion of those that also belong to A should be about P(A|B). 5/19/2021
Let’s make 3 draws from the magic hat many times: Three from a magic hat. #(AB)/#B = 4/7¼ 1/2 5/19/2021
For a uniform measure we have: 5/19/2021
Conditional probability in general: • Let A and B be two events in a general probability space. • The conditional probability of A given B is denoted by P(A | B). • It is given by: P(A | B) = P(A Å B) / P(B) 5/19/2021
Example: Rich & Famous In a certain town 10% of the inhabitants are rich, 5% are famous and 3% are rich and famous. If a town’s person is chosen at random and she is rich what is the probability she is famous? Neither Famous Rich 5/19/2021
Example: Rich & Famous In a certain town 10% of the inhabitants are rich, 5% are famous and 3% are rich and famous. • P(R) = 0. 1 • P(R & F) = 0. 03 • P(F | R) = =0. 03/0. 1 = 0. 3 Neither Famous Rich 5/19/2021
Example: Relative areas • • A point is picked uniformly at random from the big rectangle whose area is 1. Suppose that we are told that the point is in B, what is the chance that it is in A? A B 5/19/2021
Example: Relative areas In other words: Given that the point is in B, what is the conditional probability that it is in A? Area ( AB ) Area ( B ) = 1/2 A B 5/19/2021
Multiplication Rule • For all events A and B such that P(B) 0: • P(AB)=P(B)P(A|B) 5/19/2021
Box then ball Consider the following experiment: we first pick one of the two boxes; , 1 3 5 2 4 and next we pick a ball from the boxed that we picked. What’s the chance of getting a 2? 5/19/2021
Tree diagrams P(Box 1) = 1/2 P(Box 2) = 1/2 1 2 1/3 3 1/3 5 P(2 | Box 2) = 1/2 4 P(4 | Box 2) =1/2 1/3 1 3 5 2 4 1/6 1/6 1/4 P(2) = P(2 Å Box 2) = P(Box 2) P(2|Box 2) = ½*1/2 = 1/4 5/19/2021
Consider the Partition B 1 t B 2 t … t Bn = B 1 B 4 B 3 B 2 A Bn Bn-1 AB 1 t AB 2 t … t ABn = A P(AB 1)+ P(AB 2)+…+P(ABn) = P(A) 5/19/2021
Rule of Average Conditional Probabilities If B 1, …, Bn is a disjoint Partition of then P(A)= P(AB 1) + P(AB 2) +…+ P(ABn) = P(A|B 1)P(B 1) + P(A|B 2)P(B 2)+…+P(A|Bn)P(Bn) 5/19/2021
Box then ball We make the following experiment: we first pick one of the two boxes; , 1 3 5 2 4 and next we pick a ball from the boxed that we picked. What’s the probability getting a number smaller than 3. 5? 5/19/2021
Independence When the probability for A is unaffected by the occurrence of B we say that A and B are independent. In other words, A and B are independent if P(A|B)=P(A|Bc) = p Note that if A and B are independent then: P(A) = P(A|B)P(B) + P(A|Bc)P(Bc) = p P(B) + p P(Bc) =p 5/19/2021
Independence Obvious: If A is independent of B then A is also independent of Bc Question: If A is independent of B, is B independent of A? 5/19/2021
Independence Consider a ball picked uniformly: 1 1 2 Then Color=Red/Green and Number=1 or 2 are Independent: P(Red|#=1)=1/2=P(Red|#=2); P(Green|#=1)=1/2=P(Green|#=2); 5/19/2021
Independence 1 1 2 Also: P(#=2|Green)=1/3=P(#=2|Red); P(#=1|Green)=2/3=P(#=1|Red). 5/19/2021
Independence Consider a ball picked uniformly: 1 2 2 1 1 2 Then Color and Number are not independent: P(#=2|Green)=1/3 ¹ P(#=2|Red)=2/3; 5/19/2021
Independence Points from a figure have coordinates X and Y. If a point is picked at uniformly from a rectangle then the events {X > a} and {Y > b} are independent! X Y 5/19/2021
Independence P(X>a & Y>b) = P(X>a) P(Y>b) = (1 -a)(1 -b) Y 1 ¾ ½ X 0 0 ½ ¾ 1 5/19/2021
Independence Points from a figure have coordinates X and Y. If a point is picked uniformly from a cat shape then {X > a} and {Y > b} are not independent! (at least for some a & b) Y X 5/19/2021
Independence Points from a figure have coordinates X and Y. Are the events {X > a} and {Y>b} independent if a point is picked uniformly at random from a disc? X Y 5/19/2021
Dependence P(X> 2 2 )= B/p = P(X> Y B 2 2 B 1 Area = p ) X 0 0 1 2 2 5/19/2021
Dependence P(X > 2 So the events 2 Are not independent for a = b = 2 2 ) = 0 B 2/p 2 & Y> Y B B 1 X 0 0 1 2 2 5/19/2021
Independence Follow up question: Are there values of a and b for which the event {X > a} and {X > b} are independent? X Y 5/19/2021
Recall: Multiplication Rule P(AB)=P(A|B)P(B) =P(A) P(B) This holds even if A and B are independent ! 5/19/2021
Picking a Box then a Ball If a ball is drawn from a randomly picked box comes out to be red, which box would you guess it came from and what is the chance that you are right? 5/19/2021
1/3 then 1/3 Ball 1/3 Box 2/3 1/2 1/2 3/4 1/4 5/19/2021
By the Rule of Average Conditional Probability P(R Ball) = P(R Ball | W Box) P(W Box) + P(R Ball| Y Box) P(Y Box) + P(R Ball | B Box) P(B Box) = ½ * 1/3 + 2/3*1/3 + ¾*1/3 = 23/36 5/19/2021
P( P( P( | | | )= )= P( , P( P( )= ) ) , P( P( P( ) ) , ) ) = 1/3*1/2 = 1/3*2/3 23/36 1/3*3/4 = 23/36 5/19/2021
Picking a Box then a Ball If a ball is drawn from a randomly picked box comes out to be red, which box would you guess it came from and what is the chance that you are right? A: Guess -> last box. Chances you are right: 9/23. 5/19/2021
Bayes’ Rule For a partition B 1, …, Bn of all possible outcomes, 5/19/2021
Sequence of Events Multiplication rule for 3 Events P(ABC) = P(AB)P(C|AB) = P(A) P(B|A) P(C|AB) Ac P(A) Bc A P(B|A) Cc B P(C|AB) C 5/19/2021
Multiplication rule for n Events P(A 1 A 2 … An) = P(A 1 … An-1)P(An|A 1 … An-1) = P(A 1) P(A 2|A 1) P(A 3|A 1 A 2)… P(An| A 1 … An-1) A 1 c P(A 1) A 1 A 2 c P(A 2|A 1) A nc P(An| A 1 … An-1) A 2… An 5/19/2021
Shesh Backgammon We roll two dice. What is the chance that we will roll out Shesh Besh: for the first time on the n’th roll? 5/19/2021
This is a Besh Geometric Distribution Shesh Backgammon with parameter p=1/36. 5/19/2021
The Geometric distribution In Geom(p) distribution the probability of outcome n is given by p (1 -p)n-1 5/19/2021
The Birthday Problem n students in the class, what is the chance that at least two of them have the same birthday? P(at least 2 have same birthday) = 1 – P(No coinciding birthdays). We arrange the students’ birthdays in some order: B 1, B 2, …, Bn. We need: P(B 2 Ï {B 1} & B 3 Ï {B 1, B 2} & … & Bn Ï {B 1, …, Bn-1}). 5/19/2021
Use multiplication rule to find P(B 2 Ï {B 1} & B 3 Ï {B 1, B 2} & … & Bn Ï {B 1, …, Bn-1}). B 2=B 1 B 32 {B 1, B 2} Bn 2 {B 1, … Bn-1} B 2Ï{B 1} B 3Ï{B 1, B 2} … BnÏ{B 1, … Bn-1} 5/19/2021
The Birthday Problem P(at least 2 have same birthday) = 1 – P(No coinciding birthdays) = This is hard to compute for large n. So we can use an approximation. 5/19/2021
The Birthday Problem log(P(No coinciding birthdays))= 5/19/2021
The Birthday Problem P(No coinciding birthdays) P(At least 2 have same birthday) 5/19/2021
Probabilities in the. Backgammon Birthday Problem. Shesh Besh n 5/19/2021
Independence of n events P(B|A)=P(B|Ac) = P(B); P(C|AB)= P(C|Ac. B) = P(C|Ac. Bc) = P(C|ABc) =P(C) Multiplication rule for three independent events P(ABC) = P(A) P(B) P(C) 5/19/2021
Pair-wise independence does not imply independence I pick one of these people at random. If I tell you that it’s a girl, there is an equal chance that she is a blond or a brunet; she has blue or brown eyes. Similarly for a boy. However, if a tell you that I picked a blond and blue eyed person, it has to be a boy. So sex, eye color and hair color, for this group, are pair-wise independent, but not independent. 5/19/2021
- Slides: 54