Dept Computer Science Korea Univ Intelligent Information System

Dept. Computer Science, Korea Univ. Intelligent Information System Lab. Probability & Statistics #1 In-Jeong Chung chung@korea. ac. kr Intelligent Information System lab. Department of Computer Science Korea University 1

Dept. Computer Science, Korea Univ. Intelligent Information System Lab. 1. 1 Basic Concepts n Experiment any process where outcome is not known in advance with certainty ■ Note : probability theory is based on possible outcomes & events that might occur when an experiment is performed n Random experiment ■ Experiment where the outcome of experiment can’t be predicated with certainty n Sample space (Outcome space) ■ Collection of all possible outcomes of an exp. n Event (Set) ■ Subset of sample space 2

Dept. Computer Science, Korea Univ. Intelligent Information System Lab. 1. 1 Basic Concepts ■ e. g. Tossing a die : S = {1, 2, 3, 4, 5, 6} § Random variable X : a real-valued function defined on the sample space S § i. e. X is a function which assigns a real # X(s) for ∀ possible outcome s ∈ S ■ e. g. Tossing a coin : Consider an experiment of tossing a coin with 10 times § S : 210 different sequence of Head & Tail § i. e. S = {<H 10>, <H 9 T>, … , <T 10>} § X(s) = 4, if random variable X is a function which assigns the # of heads obtained on 10 tosses and s = H 2 T 3 H 3

Dept. Computer Science, Korea Univ. Intelligent Information System Lab. 1. 1 Basic Concepts ■ e. g. Consider a point in the xy-plane, and an experiment of selecting a point in the xy-plane § Random variable X(s) : x-coordinate of the chosen point, § Random variable Y(s) : y-coordinate of the chosen point, § Random variable Z(s) : distance from the origin of the plane to the point 4

Dept. Computer Science, Korea Univ. Intelligent Information System Lab. 1. 1 Basic Concepts n Set algebra with the event & sample space ■ A, B : subset of sample space ■ A 1, … , Ak : mutually exclusive events, exhaustive ■ Probability P(A) : P ∈ [0, 1] & P is the change that event A happens or occurs where A ⊂ sample space § Note : P(A) = 1 : the event A happens(occurs) always P(A) = 0 : the event A never happens 5

Dept. Computer Science, Korea Univ. Intelligent Information System Lab. 1. 1 Basic Concepts ■ Note : 6

Dept. Computer Science, Korea Univ. Intelligent Information System Lab. 1. 1 Basic Concepts ■ e. g. Ex 1. 1. 10 Amsterdam train Paris Brussels train § P(A∪B) : the probability that at least one train is on time. § i. e. the fafaculty will meet at least one student in pair 7

Dept. Computer Science, Korea Univ. Intelligent Information System Lab. 1. 2 Methods of Enumeration n Permutation n! : ■ Arrangement of n different objects with a particular order § Permutation of n different objects taken r at a given time n Combination : ■ Choice of objects without consideration ordering of among the objects n Multiplication rule : ■ Consider some sequence of experiments E 1, … , Ek with outcomes n 1, … nk, respectively, then the experiments can occur altogether 8

Dept. Computer Science, Korea Univ. Intelligent Information System Lab. 1. 2 Methods of Enumeration ■ e. g. R B G linearization 3! = 6 § C 1, C 2, … , Ck : k balls with different color § K! ways of linearization ■ e. g. from 26 alphabet characters, ∃ 26 P 3 different 3 -letter code words 9

Dept. Computer Science, Korea Univ. Intelligent Information System Lab. 1. 2 Methods of Enumeration ■ e. g. from 10 persons selection of president, vice president, secretary, a treasurer n Sampling with replacement : ■ Some object is selected from the population & then replaced before the next object is selected n Sampling without replacement : ■ Some object is selected from the population & once it is selected, it will not be replaced any more 10

Dept. Computer Science, Korea Univ. Intelligent Information System Lab. 1. 2 Methods of Enumeration n The total # of possible ordered samples of size r taken from n objects on the sampling with replacement = nr ■ ∵ use the multiplication rule, ■ The # of ordered samples from n objects ■ e. g. when we roll a die 10 times, the # of possible ordered samples = 610 11

Dept. Computer Science, Korea Univ. Intelligent Information System Lab. 1. 2 Methods of Enumeration ■ e. g. 1 2 … 10 10 balls in an urn : ■ Suppose we select 6 balls with replacement, then # total # of possible ordered samples = 106 ■ The total # of possible ordered samples of size r taken from n object in the sampling without replacement ∃ n samples on the 1 st selection ∃ n-1 samples on the 2 nd selection ∃ n-2 samples on the 3 rd selection ： 12

Dept. Computer Science, Korea Univ. Intelligent Information System Lab. 1. 2 Methods of Enumeration ■ e. g. 1 2 … 10 select 6 balls without replacement § Combination of n objects taken r at a time § Permutation : selection of r objects from n objects with a certain ordering § Combination : selection of r objects from n objects without a certain ordering 13

Dept. Computer Science, Korea Univ. Intelligent Information System Lab. 1. 2 Methods of Enumeration ■ e. g. ∃ 20 m/c of which 5 has flaws. When we select 3 m/c from these 20 m/c, what is the probability that the selected 3 m/c are flawless 14

Dept. Computer Science, Korea Univ. Intelligent Information System Lab. 1. 3 Conditional Probability n Conditional probability ■ The conditional probability of event A, given that event B has happened ■ e. g. ∃ 100 m/c such that Elec. Manual New 40 30 70 Used 20 10 30 60 40 100 § We pick a m/c at random, and suppose that it is New. What’s the probability that it is electiric? 15

Dept. Computer Science, Korea Univ. Intelligent Information System Lab. 1. 3 Conditional Probability ■ Solution § Compute P(E|N) ■ e. g. (similar example 1. 3. 7) ∃ 100 m/c such that 20 are defective and 80 are nondefective. We select 2 m/c at random without replacement, what’s the probability that both m/c are defective? ■ Solution § Event A : {the 1 st m/c is defective} § Event B : {the 2 nd m/c is defective} § We’ve to compute P(A∩B) 16

Dept. Computer Science, Korea Univ. Intelligent Information System Lab. 1. 4 Independent Events n Independent event : ■ A, B : independent event iff ■ i. e. A, B : independent event if ■ e. g. ∃ 10, 000 m/c § 10% : defective § 90% : nondefective § 2 m/c will be selected. Probability that both are nondefective? ■ Solution § Event A = {1 st item is nondefective} § Event B = {2 nd item is nondefective} 17

Dept. Computer Science, Korea Univ. Intelligent Information System Lab. 1. 4 Independent Events § Case 1) with replacement : A, B : independent § Case 2) without replacement : ■ e. g. (P. 29 ex. 1. 4. 4) § § § § Event A = {1, 2} Event B = {1, 3} Event C = {1, 4} P(A) = P(B) = P(C) = ½ P(A∩B) = P(A) P(B) = ¼ P(B∩C) = P(B) P(C) = ¼ P(A∩C) = P(A) P(C) = ¼ 1 2 3 4 ∴ A, B, C are independent i. e. pairwise independence 18