Mathematics Statistics Part II Probability and Distribution Theory
Mathematics & Statistics Part II: Probability and Distribution Theory Topic 4 Probability
Topic Goals After completing this topic, you should be able to: n Explain basic probability concepts and definitions n Use a Venn diagram to illustrate simple probabilities n Apply common rules of probability n Compute conditional probabilities n Determine whether events are independent n Use Bayes’ Theorem for conditional probabilities
Why Probability Theory? n n So far we have discussed ways to describe or summarise data. For decision making this is not enough. Need to make inferences from (observed) sample to (unobserved) population. The underlying mathematical theory for this step: probability and distribution theory n n Topics 4, 5, 6 The actual translation from sample to population: Topic 7
Important Terms n n n Random Experiment – a process leading to an uncertain outcome Basic Outcome – a possible outcome of a random experiment Sample Space – the collection S of all possible outcomes of a random experiment. Event – a subset of basic outcomes from the sample space. Event Space – Collection A of all events.
Example: Roll a die Let the Sample Space be the collection of all possible outcomes of rolling one die: S = {1, 2, 3, 4, 5, 6} Let A 1 be the event “Number 1 comes up” Let A 2 be the event “Either number 3 or 5 comes up” Let A 3 be the event “An even number comes up”
Example (cont’d) n All these events are essentially subsets of the sample space S: n n A 1 A 2 A 3 What probability would you assign to these events? n n We need to know how to manipulate sets… …which we now do
Example: Roll a die (continued) S = [1, 2, 3, 4, 5, 6] Complements: Intersections: Unions: A = [2, 4, 6] B = [4, 5, 6]
Example: Roll a die (continued) S = [1, 2, 3, 4, 5, 6] A = [2, 4, 6] n Mutually exclusive: n Collectively exhaustive: B = [4, 5, 6]
Probability n Probability – the chance that an uncertain event will occur (always between 0 and 1) 0 ≤ P(A) ≤ 1 For any event A 1 Certain . 5 0 Impossible
Example: Roll a die (continued) n What probabilties would you assign? n n Essentially: n n A 1 = {1} A 2 = {3, 5} A 3 = {2, 4, 6} P(A) = This is the classical definition of probability Useful if all outcomes equally likely. Only possible definition?
Probability (Mathematical) n Any function P that assigns a value between 0 and 1 to any event A, such that n n P(S) = 1 If A 1, A 2, A 3, … are mutually exclusive, then P(A 1 U A 2 U A 3 U …) = Σi P(Ai) These two properties are the Axioms of Probability. Everything else follows from these. Probability Space – Mathematical model (S, A, P) of random experiment
Example: Roll a die n n n Suppose you suspect that the die is loaded and that P({1}) = 1/2, whereas all other outcomes are equally likely. The only probability that satisfies the two axioms: The choice of probability is yours, theory only tells you how to manipulate them in a consistent way.
Assessing Probability In addition to the classical definition: 2. relative frequency probability n n the (limit of the) proportion of times that an event A occurs in a large number of trials, n. Basis of most statistical inference. 3. subjective probability an individual opinion or belief about the probability of occurrence
Probability Rules n The Complement rule: n The Addition rule: n The probability of the union of two events is
A Probability Table n Probabilities and joint probabilities for two events A and B can be summarised: B A
Addition Rule Example Consider a standard deck of 52 cards, with four suits: ♥♣♦♠ Let A = {card is an Ace} Let B = {card is from a red suit}
Addition Rule Example (continued) (Using the classical definition) P(Red U Ace) = Type Ace Non-Ace Total Color Red Black Total
Conditional Probability n A conditional probability is the probability of one event, given that another event has occurred:
Conditional Probability Example n n Of the cars on a used car lot, 70% have air conditioning (AC) and 40% have a CD player (CD). 20% of the cars have both. What is the probability that a car has a CD player, given that it has AC ? i. e. , we want to find P(CD | AC)
Conditional Probability Example (continued) n Of the cars on a used car lot, 70% have air conditioning (AC) and 40% have a CD player (CD). 20% of the cars have both. CD AC No AC Total No CD Total
Conditional Probability Example (continued) n Given AC, we only consider the top row (70% of the cars). Of these, 20% have a CD player. 20% of 70% is 28. 57%. CD AC No AC Total No CD Total
Multiplication Rule n Multiplication rule for two events A and B: n also
Multiplication Rule Example P(Red ∩ Ace) = Type Ace Non-Ace Total Color Red Black Total
Independence n Two events are independent if: n n Events A and B are independent when the probability of one event is not affected by the other event If A and B are independent, then if P(B)>0 if P(A)>0
Independence Example n Of the cars on a used car lot, 70% have air conditioning (AC) and 40% have a CD player (CD). 20% of the cars have both. CD No CD Total AC No AC Total n Are the events AC and CD independent?
Independence Example (continued) CD AC No AC Total P(AC ∩ CD) = 0. 2 P(AC) = 0. 7 P(CD) = 0. 4 No CD Total
Bivariate Probabilities Outcomes for bivariate events: B 1 B 2 . . . Bk A 1 P(A 1 B 1) P(A 1 B 2) . . . P(A 1 Bk) A 2 P(A 2 B 1) P(A 2 B 2) . . . P(A 2 Bk) . . . . Ah P(Ah B 1) P(Ah B 2) . . . P(Ah Bk)
Joint and Marginal Probabilities n n The probability of a joint event, A ∩ B (in the classical definition): Computing a marginal probability: n Where B 1, B 2, …, Bk are k mutually exclusive and collectively exhaustive events
Marginal Probability Example P(Ace) Type Ace Non-Ace Total Color Red Black Total
Odds n n The odds in favor of a particular event are given by the ratio of the probability of the event divided by the probability of its complement The odds in favor of A are
Odds: Example n Calculate the probability of winning if the odds of winning are 3 to 1:
Overinvolvement Ratio n n The probability of event A 1 conditional on event B 1 divided by the probability of A 1 conditional on activity B 2 is defined as the overinvolvement ratio: An overinvolvement ratio greater than 1 implies that event A 1 increases the conditional odds ration in favor of B 1:
Bayes’ Theorem n where: Ei = ith event of k mutually exclusive and collectively exhaustive events A = new event that might impact P(Ei)
Bayes’ Theorem Example (1) n n n A drilling company has estimated a 40% chance of striking oil for their new well. A detailed test has been scheduled for more information. Historically, 60% of successful wells have had detailed tests, and 20% of unsuccessful wells have had detailed tests. (relative frequency definition) Given that this well has been scheduled for a detailed test, what is the probability that the well will be successful?
Bayes’ Theorem Example (1) (continued) n Let n P( ) = n Define the detailed test event as n Conditional probabilities: n Goal is to find , P( ) = (prior probabilities)
Bayes’ Theorem Example (1) (continued) Apply Bayes’ Theorem: So the revised probability of success (from the original estimate of ), given that this well has been scheduled for a detailed test, is
Bayes’ Theorem Example (2) n 0. 5% of population has certain disease. n Clinical test is 95% accurate. n Your test result is positive. n What is the probability you have the disease?
Bayes’ Theorem Example (2) n Let n Prior odds: n Conditional probs:
Bayes’ Theorem Example (2) n Bayes’ Theorem: n Why does this surprising result occur? (tutorial)
Bayes’ Theorem Example (3) n In a quiz show you can choose 1 of 3 doors. n Behind 1 door there is a prize. n n You choose door and the quiz-master tells you the prize is not behind door. Should you change your choice?
Bayes’ Theorem Example (3) n Let n n Assume quiz-master knows where prize is and does not lie. prior odds:
Bayes’ Theorem Example (3) n Conditional probabilities: n Bayes’ Theorem: n So,
Topic Summary n Defined basic probability concepts n Sample spaces, events, intersection and union of events, mutually exclusive and collectively exhaustive events, complements n Stated the Axioms of Probability n Examined basic probability rules n Complement rule, addition rule, multiplication rule n Defined conditional, joint, and marginal probabilities n Reviewed odds and the overinvolvement ratio n Defined independence n Discussed Bayes’ theorem
- Slides: 43