Basics on Probability Jingrui He 09112007 Coin Flips

Basics on Probability Jingrui He 09/11/2007

Coin Flips o You flip a coin n o Head with probability 0. 5 You flip 100 coins n How many heads would you expect

Coin Flips cont. o You flip a coin n o Head with probability p Binary random variable Bernoulli trial with success probability p You flip k coins n n n How many heads would you expect Number of heads X: discrete random variable Binomial distribution with parameters k and p

Discrete Random Variables o Random variables (RVs) which may take on only a countable number of distinct values n o E. g. the total number of heads X you get if you flip 100 coins X is a RV with arity k if it can take on exactly one value out of n E. g. the possible values that X can take on are 0, 1, 2, …, 100

Probability of Discrete RV o o Probability mass function (pmf): Easy facts about pmf n n if if

Common Distributions o Uniform n X takes values 1, 2, …, N n n o E. g. picking balls of different colors from a box Binomial n X takes values 0, 1, …, n n n E. g. coin flips

Coin Flips of Two Persons o Your friend and you both flip coins n n n Head with probability 0. 5 You flip 50 times; your friend flip 100 times How many heads will both of you get

Joint Distribution o Given two discrete RVs X and Y, their joint distribution is the distribution of X and Y together n E. g. P(You get 21 heads AND you friend get 70 heads) n E. g. o

Conditional Probability o is the probability of given the occurrence of n o , E. g. you get 0 heads, given that your friend gets 61 heads

Law of Total Probability o Given two discrete RVs X and Y, which take values in and , We have

Marginalization Marginal Probability Joint Probability Conditional Probability Marginal Probability

Bayes Rule o X and Y are discrete RVs…

Independent RVs o o Intuition: X and Y are independent means that neither makes it more or less probable that Definition: X and Y are independent iff

More on Independence o o E. g. no matter how many heads you get, your friend will not be affected, and vice versa

Conditionally Independent RVs o o Intuition: X and Y are conditionally independent given Z means that once Z is known, the value of X does not add any additional information about Y Definition: X and Y are conditionally independent given Z iff

More on Conditional Independence

Monty Hall Problem o o You're given the choice of three doors: Behind one door is a car; behind the others, goats. You pick a door, say No. 1 The host, who knows what's behind the doors, opens another door, say No. 3, which has a goat. Do you want to pick door No. 2 instead?

Host reveals Goat A or Host reveals Goat B Host must reveal Goat A

Monty Hall Problem: Bayes Rule o : the car is behind door i, i = 1, 2, 3 o o o : the host opens door j after you pick door i

Monty Hall Problem: Bayes Rule cont. o o o WLOG, i=1, j=3

Monty Hall Problem: Bayes Rule cont. o o

Monty Hall Problem: Bayes Rule cont. o o o You should switch!

Continuous Random Variables o o o What if X is continuous? Probability density function (pdf) instead of probability mass function (pmf) A pdf is any function that describes the probability density in terms of the input variable x.

PDF o Properties of pdf n n n o Actual probability can be obtained by taking the integral of pdf n E. g. the probability of X being between 0 and 1 is

Cumulative Distribution Function o o Discrete RVs n o Continuous RVs n n

Common Distributions o Normal n n E. g. the height of the entire population

Common Distributions cont. o Beta n n n : uniform distribution between 0 and 1 E. g. the conjugate prior for the parameter p in Binomial distribution

Joint Distribution o o Given two continuous RVs X and Y, the joint pdf can be written as

Multivariate Normal o Generalization to higher dimensions of the one-dimensional normal Covariance Matrix o Mean

Moments o o Mean (Expectation): n Discrete RVs: n Continuous RVs: Variance: n Discrete RVs: n Continuous RVs:

Properties of Moments o Mean n o If X and Y are independent, Variance n n If X and Y are independent,

Moments of Common Distributions o Uniform n o Mean ; variance Normal n o ; variance Binomial n o Mean ; variance Beta n Mean ; variance

Probability of Events o X denotes an event that could possibly happen n o E. g. X=“you will fail in this course” P(X) denotes the likelihood that X happens, or X=true n What’s the probability that you will fail in this course? denotes the entire event set o n

The Axioms of Probabilities o 0 <= P(X) <= 1 o , where o o disjoint events Useful rules n n are

Interpreting the Axioms