Database Management Systems Data Mining Statistics Review Jerry

D A T A B A S E Probability ² Relative frequency approach: The number of times that an event occurs out of the total population of events. ª You have 3 red balls and 7 white balls in a bag. The probability of drawing a white ball on the first try is 70%. ª Your customers are distributed across five cities: 35% in City A, 25% in City B, 20% in City C, 15% in City D, 15% in City E. ² Subjective probability: A belief in the likelihood of an outcome. Often subjective because of lack of full information. Generally modified over time based on acquisition of new information. It is important to separate belief from preference (but difficult), and also important that subjective probability maintain consistency. ª There is a 65% chance that the Federal Reserve board will reduce interest rates at the next meeting. 2

D A T A B A S E Probability: Frequency ² Need a complete count of events ² Permutations: Order does count ² Combinations: Order does not count ² Basic multiplication rule. If a single action has k ways to be performed, and the action is performed n times; the total number of possible outcomes is: k*k*k*…*k ª Flip a coin five times (n=5). A single act has two outcomes (k=2), so there are 25 = 32 possible outcomes. 3

D A T A B A S E Counting: Permutation ² How many ways can objects (or actions) be rearranged? ª You have four cards: A, K, Q, J. How many ways can they be arranged? ª Four items (n) arranged one card at a time (r): 4 * 3 * 2 * 1 K A K, Q, J 4 Q J 3 Q J J Q K J J K K Q Q K 2 1 4

D A T A B A S E Permutation: General ² Ways to rearrange n items taken r at a time: ª n(n-1)(n-2)…(n-r+1) 5

D A T A B A S E Combinations ² Number of ways of selecting items, and order does not count. ª Combinations are smaller than permutations ª You can divide the number of permutations by the number of ways of ª Elect three people from a group of ten. n = 10, r = 3 6

D A T A B A S E Probability Rules: Complement ² Complement (opposite): ² P(E) + P(E’) = 1 ª The probability of an event happening or not happening is one. 7

D A T A B A S E Probability Rules: Mutually Exclusive ² Mutually Exclusive: Only one event of a group can happen. The probability of both occurring is zero. ² P(A B) = 0 ² Then, the probability of one or the other of the events occurring is computed by the sum of the probabilities: ² P(A B) = P(A) + P(B) ² Example, pool balls, numbered 1 through 10 ª ª Event A: Draw a ball number <= 3 Event B: Draw a ball number >= 6 P(A or B) = 3/10 + 5/10 = 8/10 Can also find as complement: 1 – 2/10 = 8/10 ² In general, P(E 1 E 2 … En) = P(Ei) 8

D A T A B A S E Probability Rules: Independence ² Events are independent (pairwise) if they have no influence on each other. ² If events are independent, the probability of both events occurring is found by multiplying their individual probabilities: ª P(A B) = P(A) P(B) ² Example: An urn has 3 red balls and 7 white ones. Draw a ball and then flip a coin. What is the probability you draw a white ball and flip heads? ª P(A B) = 0. 7 * 0. 5 = 0. 35 9

D A T A B A S E Conditional Probability ² The probability that event A will occur given that event B has already happened: P(A | B) ª Example 1: An urn has 3 red balls and 7 white ones. On the first draw you pull out a white ball (event B). If you do not replace that ball in the urn, what is the probability of drawing a red ball next (Event A). Answer: 3/9 Note that these events are not independent. ² In general, the probability of two events occuring: ª P(A B) = P(A) P(B | A) ª Example 2: Draw 2 cards from a 52 -card deck without replacement. What is the probability that both are kings? ª P(King 1) = 4/52 P(King 2 | King 1) = 3/51 ª P(King 2 King 1) = 4/52 * 3/51 10

D A T A B A S E Probability: Joint and Conditional Table Female Married . 42 . 18 . 60 Not Married . 28 . 12 . 40 . 70 . 30 1. 00 P(Female) =. 70 P(Married Female) =. 42 P(Married | Female) = P(M F)/P(F) =. 42/. 70 11

D A T A B A S E Joint Probability: Tree Diagram Manufacturing: Group A: 4 machines 5% defect rate Group B: 6 machines, 10% defect rate Choose a machine, then a product—probability defective? 05 * P(A D) =. 02. = A) | D P( *. 4 P(A D’) =. . 38 P(D’ | A) = )= * A. 95 P( * P(B )= . 6 * P(D 10 * P(B D) =. 06 =. 90 * P(B D’) =. 54 =. ) B | ’ | B) 1. 00 12

D A T A B A S E Joint Probabilities: Table Probability Defective (D) Non-defective (D’) P(A) = 0. 4 0. 05 0. 95 P(B) = 0. 6 0. 10 0. 90 P(A D) = P(A)*P(D|A) = 0. 4(0. 05) =. 2 Production Defective (D) Non-defective (D’) A 0. 02 0. 38 B 0. 06 0. 54 Total 0. 08 0. 92 13

D A T A B A S E Bayes’ Theorem Now, in a sense, work backwards. We sample a part at random and it is defective. What is the probability that it came from machine A? Machine B? P(A | D) = 0. 02/0. 08 = 1/4 P(B | D) = 0. 06/0. 08 = 3/4 In this example, the machine is the state of nature we wish to identify, and defective or not is the information. 14

D A T A B A S E Bayes’ Theorem in General We know: (1) There are n states of nature S 1, S 2, …, Sn (2) An initial (a priori) probability for each state (3) Some type of information I (4) The conditional probabilities: P(I | Si) We can compute the posterior probabilities, given the new information: 15

D A T A B A S E Bayes’ Theorem Example ² ² Chao: Statistics for Management/2 e States of economy: S 1: recession, S 2: stable, S 3: prosperity P(S 1) =. 25, P(S 2) =. 5, P(S 3) =. 25 (in general/a priori) We have forecasts as information. The forecasts are either optimistic (I) or pessimistic (I’) ² The results of the forecasts in the past are as follows: Prior Probability State of Economy Optimistic (I) Pessimistic (I’) P(S 1) =. 25 S 1 0. 9 P(S 2) =. 50 S 2 0. 5 P(S 3) =. 25 S 3 0. 8 0. 2 16

D A T A B A S E Example: Joint Probability Prior Probability State of Economy Optimistic (I) P(I | Si) Pessimistic (I’) P(I’ | Si) P(S 1) =. 25 S 1 0. 9 P(S 2) =. 50 S 2 0. 5 P(S 3) =. 25 S 3 0. 8 0. 2 State Optimistic (I) Pessimistic (I’) S 1 P(S 1 I) = 0. 025 0. 225 S 2 P(S 2 I) = 0. 250 S 3 P(S 2 I) = 0. 200 0. 050 Total P(I) = 0. 475 P(I’) = 0. 525 17

D A T A B A S E Bayes’ Example State Optimistic (I) Pessimistic (I’) S 1 P(S 1 I) = 0. 025 0. 225 S 2 P(S 2 I) = 0. 250 S 3 P(S 2 I) = 0. 200 0. 050 Total P(I) = 0. 475 P(I’) = 0. 525 Probability next year is prosperous (S 3) if the forecast is optimistic (I): P(S 3 | I) = P(S 3 I)/P(I) = 0. 200/0. 475 =. 421 18

D A T A B A S E Bayes: Prior and Posterior Probabilities Probability estimates at the start (a priori) are naïve: P(S 1) = 0. 25 P(S 2) = 0. 50 P(S 3) = 0. 25 Probabilities after the forecast (posterior) reflect the new information: P(S 1 | I) = 0. 053 P(S 1 | I’) = 0. 429 P(S 2 | I) = 0. 526 P(S 2 | I’) = 0. 476 P(S 3 | I) = 0. 421 P(S 3 | I’) = 0. 095 19