Randomized Algorithms Evaluation algorithms for NOR circuits and

  • Slides: 95
Download presentation
Randomized Algorithms Evaluation algorithms for NOR circuits and Min-Max Principle Speaker: Chuang-Chieh Lin Advisor:

Randomized Algorithms Evaluation algorithms for NOR circuits and Min-Max Principle Speaker: Chuang-Chieh Lin Advisor: Professor Maw-Shang Chang National Chung Cheng University March 30, 2006

n References: – Professor Hsueh-I Lu’ s slides for Randomized Algorithms – Randomized algorithms,

n References: – Professor Hsueh-I Lu’ s slides for Randomized Algorithms – Randomized algorithms, Rajeev Motwani and Prabhakar Raghavan 2020/10/2 Computation Theory Lab, CSIE, CCU, Taiwan 2

Outline n n Lower bound on deterministic evaluation algorithms for NOR circuits Yao’s Principle

Outline n n Lower bound on deterministic evaluation algorithms for NOR circuits Yao’s Principle for proving lower bounds 2020/10/2 Computation Theory Lab, CSIE, CCU, Taiwan 3

Part 1 Randomized evaluation for a logic function 2020/10/2 Computation Theory Lab, CSIE, CCU,

Part 1 Randomized evaluation for a logic function 2020/10/2 Computation Theory Lab, CSIE, CCU, Taiwan 4

Some logic gates NOT gate (inverter) A If A = 0 then B =

Some logic gates NOT gate (inverter) A If A = 0 then B = 1 If A = 1 then B = 0 B OR gate A B 2020/10/2 C = A + B If A = 1 OR B = 1, then C = 1 otherwise C = 0 Computation Theory Lab, CSIE, CCU, Taiwan 5

NOR gates NOR gate 2020/10/2 X Y Z X Y Computation Theory Lab, CSIE,

NOR gates NOR gate 2020/10/2 X Y Z X Y Computation Theory Lab, CSIE, CCU, Taiwan Z 6

A one-level complete NOR circuit X 1 X 2 2020/10/2 Y Computation Theory Lab,

A one-level complete NOR circuit X 1 X 2 2020/10/2 Y Computation Theory Lab, CSIE, CCU, Taiwan 7

A two-level complete NOR circuit X 1 X 2 Y X 3 X 4

A two-level complete NOR circuit X 1 X 2 Y X 3 X 4 2020/10/2 Computation Theory Lab, CSIE, CCU, Taiwan 8

A three-level complete NOR circuit X 1 X 2 X 3 X 4 Y

A three-level complete NOR circuit X 1 X 2 X 3 X 4 Y X 5 X 6 X 7 X 8 2020/10/2 Computation Theory Lab, CSIE, CCU, Taiwan 9

A Circuit evaluation problem n n Let Fk (x 1, x 2, … ,

A Circuit evaluation problem n n Let Fk (x 1, x 2, … , x 2 k) denote the value of the level-k complete NOR circuit on input x 1, x 2, … , x 2 k. The NOR circuit evaluation problem: – Input: x 1, x 2, … , x 2 k – Output: Fk (x 1, x 2, … , x 2 k) 2020/10/2 Computation Theory Lab, CSIE, CCU, Taiwan 10

Take a closer look at NOR x 1 x 2 n y = F

Take a closer look at NOR x 1 x 2 n y = F 1(x 1, x 2) To determine y = F 1(x 1, x 2), there are two cases: – Case 1: y = 0. At least one of x 1 and x 2 is 1, so we only have to read one of x 1 and x 2 if we are lucky. – Case 2: y = 1. Both of x 1 and x 2 are 0, so we have to read both x 1 and x 2 to ensure y = 1. 2020/10/2 Computation Theory Lab, CSIE, CCU, Taiwan 11

Question: Can a correct evaluation algorithm avoid reading all inputs? X 1 X 2

Question: Can a correct evaluation algorithm avoid reading all inputs? X 1 X 2 X 3 X 4 X 5 X 6 Y X 7 X 8 2020/10/2 Computation Theory Lab, CSIE, CCU, Taiwan 12

Perhaps not for 1 F(x 1, x 2) x 1 x 2 n y

Perhaps not for 1 F(x 1, x 2) x 1 x 2 n y = F 1(x 1, x 2) To determine y = F 1(x 1, x 2), there are two cases: – Case 1: y = 0. At least one of x 1 and x 2 is 1, so we only have to read one of x 1 and x 2 if we are lucky. – Case 2: y = 1. Both of x 1 and x 2 are 0, so we have to read both x 1 and x 2 to ensure y = 1. 2020/10/2 Computation Theory Lab, CSIE, CCU, Taiwan 13

How about F 2(x 1, x 2, x 3, x 4)? X 1 X

How about F 2(x 1, x 2, x 3, x 4)? X 1 X 2 Y X 3 X 4 2020/10/2 When y = 1, both input of the rightmost NOR gate has to be 0. Therefore, for the two gates at the left, the algorithm, if lucky enough, doesn’t have to read all of x 1, x 2, x 3, and x 4. Computation Theory Lab, CSIE, CCU, Taiwan 14

Wannachange your answer? 2020/10/2 Computation Theory Lab, CSIE, CCU, Taiwan 15

Wannachange your answer? 2020/10/2 Computation Theory Lab, CSIE, CCU, Taiwan 15

The answer is: n n It’s impossible for any deterministic algorithm to correctly evaluate

The answer is: n n It’s impossible for any deterministic algorithm to correctly evaluate Fk in sublinear (i. e. , o(2 k)) time for any input x 1, x 2, …, x 2 k. It’s possible for a randomized algorithm to correctly evaluate Fk in expected sublinear (i. e. , o(2 k)) time for any input x 1, x 2, …, x 2 k. 2020/10/2 Computation Theory Lab, CSIE, CCU, Taiwan 16

Theorem 1 n For any deterministic algorithm A that always correctly evaluate Fk for

Theorem 1 n For any deterministic algorithm A that always correctly evaluate Fk for any k 1, there exists an input instance x 1, x 2, …, x 2 k such that A has to read all 2 k numbers to determine Fk (x 1, x 2, … , x 2 k). 2020/10/2 Computation Theory Lab, CSIE, CCU, Taiwan 17

Proof strategy n Construct two sets of inputs that differ only at the last

Proof strategy n Construct two sets of inputs that differ only at the last value read by algorithm A. n Show that the value of Fk (x 1, x 2, …, x 2 k) does depend on the very last bit. 2020/10/2 Computation Theory Lab, CSIE, CCU, Taiwan 18

Observations n Since A is a deterministic algorithm, it reads the input x 1,

Observations n Since A is a deterministic algorithm, it reads the input x 1, x 2, …, x 2 k in a fixed order for each k 1. n Each NOR gate in the circuit of Fk has two legs. The leg that contains the last leaf to be read by algorithm A is called its back leg. The other leg is called its front leg. 2020/10/2 Computation Theory Lab, CSIE, CCU, Taiwan 19

Mark the front leg in red 1 3 6 2 X 1 X 2

Mark the front leg in red 1 3 6 2 X 1 X 2 X 3 X 4 Y 5 8 7 4 2020/10/2 X 5 X 6 X 7 X 8 Computation Theory Lab, CSIE, CCU, Taiwan 20

Let all front legs to be 0, then each back leg has to be

Let all front legs to be 0, then each back leg has to be the complement of its parent. 1 3 6 2 0 X 1 X 2 0 X 3 0 X 4 0 Y 5 0 X 5 X 6 8 7 4 X 7 0 X 8 2020/10/2 0 Computation Theory Lab, CSIE, CCU, Taiwan 21

The two possible values of y determine two sets of inputs that differ only

The two possible values of y determine two sets of inputs that differ only at the last variable read by algorithm A. 1 3 6 2 0 X 1 1 X 2 1 X 3 0 X 4 0 1 0 Y 5 0 X 5 0 8 0 X 6 1 7 1 X 0 4 0 X 8 2020/10/2 7 1 Computation Theory Lab, CSIE, CCU, Taiwan 22

The two possible values of y determine two sets of inputs that differ only

The two possible values of y determine two sets of inputs that differ only at the last variable read by algorithm A. 1 3 6 2 0 X 1 1 X 2 1 X 3 0 X 4 0 1 0 Y 5 0 X 5 1 8 1 X 6 0 7 1 X 0 4 0 X 8 2020/10/2 7 0 Computation Theory Lab, CSIE, CCU, Taiwan 23

Therefore, anydeterministic algorithm takes (2 k) time to correctly evaluate Fk in the worstcase!

Therefore, anydeterministic algorithm takes (2 k) time to correctly evaluate Fk in the worstcase! 2020/10/2 Computation Theory Lab, CSIE, CCU, Taiwan 24

Randomization can be very helpful for this problem! 2020/10/2 Computation Theory Lab, CSIE, CCU,

Randomization can be very helpful for this problem! 2020/10/2 Computation Theory Lab, CSIE, CCU, Taiwan 25

Algorithm. Rand. Eval Boolean function Rand. Eval (x 1, … , x 2 t

Algorithm. Rand. Eval Boolean function Rand. Eval (x 1, … , x 2 t ) { if (2 t == 1) return x 1; throw a fair coin; if (the head appears) { if (Rand. Eval (x 1, … , xt ) == 1) return 0; else return (!Rand. Eval (xt+1, … , x 2 t )); } else { if (Rand. Eval (xt+1, … , x 2 t) == 1) return 0; else return (!Rand. Eval (x 1, … , xt )); } } 2020/10/2 Computation Theory Lab, CSIE, CCU, Taiwan 26

Theorem 2 For each integer 1, algorithm Rand. Eval correctly evaluates F 2 in

Theorem 2 For each integer 1, algorithm Rand. Eval correctly evaluates F 2 in expected O(3 ) time. 2020/10/2 Computation Theory Lab, CSIE, CCU, Taiwan 27

Proving theorem by induction on n Basis: two levels. Try for yourselves to show

Proving theorem by induction on n Basis: two levels. Try for yourselves to show that the expected number of variables to be read by Rand. Eval at most 3. X 1 X 2 Y X 3 X 4 2020/10/2 Computation Theory Lab, CSIE, CCU, Taiwan 28

X 1 X 2 1 Y 0 X 3 X 4 Expected legs read

X 1 X 2 1 Y 0 X 3 X 4 Expected legs read by Rand. Eval is at most 1. 5. X 1 X 2 0 Y X 3 X 4 1 0 Expected legs read by Rand. Eval is 2. 2020/10/2 Computation Theory Lab, CSIE, CCU, Taiwan 29

X 1 X 2 1 Y X 3 X 4 0 Expected number of

X 1 X 2 1 Y X 3 X 4 0 Expected number of read variables ≤ 1. 5 2 = 3 Expected legs read by Rand. Eval is 2. X 1 X 2 0 Y X 3 X 4 0 1 Expected number of read variables ≤ 2 1. 5 = 3 Expected legs read by Rand. Eval is at most 1. 5. 2020/10/2 Computation Theory Lab, CSIE, CCU, Taiwan 30

The induction step: +1 Case 1: The output is 0. At least one leg

The induction step: +1 Case 1: The output is 0. At least one leg has to be 1. So, the expected number of legs to be evaluated rightmost level is at most 1. 5, each takes O(2 3 ) time by the inductive hypothesis. The expected time is thus O(3 +1). 1 0 2020/10/2 Computation Theory Lab, CSIE, CCU, Taiwan 31

The induction step: + 1 (contd. ) Case 2: The output is 1. Both

The induction step: + 1 (contd. ) Case 2: The output is 1. Both legs have to be 0. So, the expected number of legs to be evaluated at the second-to -rightmost level is 3, each takes O(3 ) time by the inductive hypothesis. The expected time is thus O(3 +1). 0 1 0 2020/10/2 Computation Theory Lab, CSIE, CCU, Taiwan 32

Theorem 2 For each integer 1, algorithm Rand. Eval correctly evaluates F 2 in

Theorem 2 For each integer 1, algorithm Rand. Eval correctly evaluates F 2 in expected O(3 ) time. Corollary 3 The expected running time of Rand. Eval on evaluating Flg n is O(3 log 4 n) = O(n 0. 793). 2020/10/2 Computation Theory Lab, CSIE, CCU, Taiwan 33

Revisit the Theorem 1 n For any deterministic algorithm A that always correctly evaluate

Revisit the Theorem 1 n For any deterministic algorithm A that always correctly evaluate Fk for any k 1, there exists an input instance x 1, x 2, …, x 2 k such that A has to read all 2 k numbers to determine Fk (x 1, x 2, … , x 2 k). 2020/10/2 Computation Theory Lab, CSIE, CCU, Taiwan 34

Another proof strategy n n Without loss of generality, we may assume that a

Another proof strategy n n Without loss of generality, we may assume that a deterministic algorithm works in a depth-first manner Prove by induction on k that any deterministic depth-first algorithm has two “trouble” instances, one for output 0 and the other for output 1. 2020/10/2 Computation Theory Lab, CSIE, CCU, Taiwan 35

Depth-first evaluation n Definition: does not read any leaf beneath the second leg until

Depth-first evaluation n Definition: does not read any leaf beneath the second leg until the value of the first leg is determined For example, Rand. Eval is an depth-first evaluation algorithm. Observation: Without loss of generality, we can focus only depth-first evaluation algorithms. Why? 2020/10/2 Computation Theory Lab, CSIE, CCU, Taiwan 36

The reason n Lemma. Let D’ be a deterministic algorithm that correctly evaluates any

The reason n Lemma. Let D’ be a deterministic algorithm that correctly evaluates any NOR circuit. Then, there is a deterministic depth-first evaluation algorithm D such that – The outputs of D and D’ coincide on any input. – The number of leaves read by D is always no more than that by D’. n n Proof: a simple exercise. So, it suffices to focus on depth-first algorithms. 2020/10/2 Computation Theory Lab, CSIE, CCU, Taiwan 37

For example, 1 X 2 X 3 1 X 4 0 1 0 Y

For example, 1 X 2 X 3 1 X 4 0 1 0 Y 0 X 5 0 X 6 0 1 0 X 7 0 X 8 2020/10/2 Computation Theory Lab, CSIE, CCU, Taiwan 38

1 X 2 X 3 1 X 4 0 1 0 Y 0 X

1 X 2 X 3 1 X 4 0 1 0 Y 0 X 5 0 X 6 0 1 0 X 7 0 X 8 2020/10/2 Computation Theory Lab, CSIE, CCU, Taiwan 39

Another example, X 1 X 2 X 3 X 4 Y 1 X 5

Another example, X 1 X 2 X 3 X 4 Y 1 X 5 X 6 0 X 7 1 X 8 0 2020/10/2 0 1 Computation Theory Lab, CSIE, CCU, Taiwan 40

X 1 X 2 X 3 X 4 Y 1 X 5 X 6

X 1 X 2 X 3 X 4 Y 1 X 5 X 6 0 X 7 1 X 8 0 2020/10/2 0 1 Computation Theory Lab, CSIE, CCU, Taiwan 41

Proof by induction on k n CLAIM: The following statement holds for any positive

Proof by induction on k n CLAIM: The following statement holds for any positive integer k and any deterministic depth-first evaluation algorithm A. – There is an input Xk, 1 (respectively, Xk, 0) such that A has to read all numbers in the input to correctly compute Fk(Xk, 1) = 1 (respectively, Fk(Xk, 0) = 0). 2020/10/2 Computation Theory Lab, CSIE, CCU, Taiwan 42

Basis: k =1. Xk, 1 0 0 1 Xk, 0 0 1 0 if

Basis: k =1. Xk, 1 0 0 1 Xk, 0 0 1 0 if A reads the first leg first Xk, 0 1 0 0 if A reads the second leg first 2020/10/2 Computation Theory Lab, CSIE, CCU, Taiwan 43

Induction step: k k+1. Xk+1, 1 Xk, 0 0 1 Xk, 0 2020/10/2 0

Induction step: k k+1. Xk+1, 1 Xk, 0 0 1 Xk, 0 2020/10/2 0 Computation Theory Lab, CSIE, CCU, Taiwan 44

Induction step: k k+1. (contd. ) Xk+1, 0 Xk, 0 First leg to be

Induction step: k k+1. (contd. ) Xk+1, 0 Xk, 0 First leg to be evaluated by A. 0 0 Xk, 1 2020/10/2 1 Computation Theory Lab, CSIE, CCU, Taiwan 45

Induction step: k k+1. (contd. ) Xk+1, 0 Xk, 1 1 0 Xk, 0

Induction step: k k+1. (contd. ) Xk+1, 0 Xk, 1 1 0 Xk, 0 2020/10/2 0 First leg to be evaluated by A. Computation Theory Lab, CSIE, CCU, Taiwan 46

Therefore, anydeterministic algorithm takes (2 k) time to correctly evaluate Fk in the worstcase!

Therefore, anydeterministic algorithm takes (2 k) time to correctly evaluate Fk in the worstcase! 2020/10/2 Computation Theory Lab, CSIE, CCU, Taiwan 47

Comments (by Prof. Lu) n Throwing coins does the magic: – For any input,

Comments (by Prof. Lu) n Throwing coins does the magic: – For any input, including Xk, 0 and Xk, 1, Rand. Eval runs in expected O(n 0. 793) time. – By Snir [TCS’ 85]. n Snir’s algorithm Rand. Eval is an optimal Las Vegas algorithm for evaluating NOR circuits. – That is, any Las Vegas algorithm requires (n 0. 793) time to evaluate NOR circuits. – Proved by Saks and Wigderson [FOCS’ 86]. 2020/10/2 Computation Theory Lab, CSIE, CCU, Taiwan 48

Part 2 Yao’s Principle for Proving Lower Bounds [FOCS’ 77] 2020/10/2 Computation Theory Lab,

Part 2 Yao’s Principle for Proving Lower Bounds [FOCS’ 77] 2020/10/2 Computation Theory Lab, CSIE, CCU, Taiwan 49

Yao’s. Inequality n Let be a problem. Let D consist of all deterministic algorithms

Yao’s. Inequality n Let be a problem. Let D consist of all deterministic algorithms for solving . Let I consist of all possible input instances for . Let Ip be denote the input instance of with probability distribution p. Then the following inequality holds for any probability distribution p and any Las Vegas randomized algorithm R. min expected time of A running on Ip A D max expected time of R running on I I I 2020/10/2 Computation Theory Lab, CSIE, CCU, Taiwan 50

An alternative viewpoint n Let I be an input instance for the problem .

An alternative viewpoint n Let I be an input instance for the problem . Let D consist of all deterministic algorithms that correctly solve on I. n Then each Las Vegas algorithm for is actually a probabilistic distribution over D. 2020/10/2 Computation Theory Lab, CSIE, CCU, Taiwan 51

For example, n Such an instance has 8 deterministic depth-first evaluation algorithms. u x

For example, n Such an instance has 8 deterministic depth-first evaluation algorithms. u x 1 x 2 v u x 3 x 4 2020/10/2 w y v w Computation Theory Lab, CSIE, CCU, Taiwan 52

Algorithm. Rand. Eval Boolean function Rand. Eval (x 1, … , x 2 t

Algorithm. Rand. Eval Boolean function Rand. Eval (x 1, … , x 2 t ) { if (2 t == 1) return x 1; throw a fair coin; if (the head appears) { if (Rand. Eval (x 1, … , xt ) == 1) return 0; else return (!Rand. Eval (xt+1, … , x 2 t )); } else { if (Rand. Eval (xt+1, … , x 2 t) == 1) return 0; else return (!Rand. Eval (x 1, … , xt )); } } 2020/10/2 Computation Theory Lab, CSIE, CCU, Taiwan 53

The alternative view n Rand. Eval is the uniform distribution over the eight deterministic

The alternative view n Rand. Eval is the uniform distribution over the eight deterministic depth-first evaluation algorithms. 2020/10/2 Computation Theory Lab, CSIE, CCU, Taiwan 54

n We shall explain Yao’s Inequality through game-theoretic techniques as follows. 2020/10/2 Computation Theory

n We shall explain Yao’s Inequality through game-theoretic techniques as follows. 2020/10/2 Computation Theory Lab, CSIE, CCU, Taiwan 55

2020/10/2 Computation Theory Lab, CSIE, CCU, Taiwan 56

2020/10/2 Computation Theory Lab, CSIE, CCU, Taiwan 56

Strategic Interactions n n n Players: Strategies: Payoffs: Reynolds and Philip Morris { Advertise

Strategic Interactions n n n Players: Strategies: Payoffs: Reynolds and Philip Morris { Advertise , Do Not Advertise } Companies’ Profits n Each firm earns $5 million from its customers Advertising costs a firm $2 million Advertising captures $3 million from competitor n How to represent this game? n n 2020/10/2 Computation Theory Lab, CSIE, CCU, Taiwan 57

Stategic. Normal Form PLAYERS Reynolds No Ad Philip Morris No Ad Ad 5 ,

Stategic. Normal Form PLAYERS Reynolds No Ad Philip Morris No Ad Ad 5 , 5 2 , 6 6 , 2 STRATEGIES 2020/10/2 3 , 3 PAYOFFS Computation Theory Lab, CSIE, CCU, Taiwan 58

Nash Equilibrium Reynolds n Best reply for Reynolds: n n n No Ad Ad

Nash Equilibrium Reynolds n Best reply for Reynolds: n n n No Ad Ad Philip Morris No Ad Ad 5 , 5 2 , 6 6 , 2 3 , 3 If Philip Morris advertises: If Philip Morris does not advertise: advertise Regardless of what you think Philip Morris will do Advertise! 2020/10/2 Computation Theory Lab, CSIE, CCU, Taiwan 59

Nash Equilibrium j n i n 2020/10/2 When the row player uses the i-th

Nash Equilibrium j n i n 2020/10/2 When the row player uses the i-th strategy, the best strategy for the column player is strategy j. When the column player uses the j-th strategy, the best strategy for the row player is strategy i. Computation Theory Lab, CSIE, CCU, Taiwan 60

Another example: Prisoner’s Dilemma 2020/10/2 Computation Theory Lab, CSIE, CCU, Taiwan 61

Another example: Prisoner’s Dilemma 2020/10/2 Computation Theory Lab, CSIE, CCU, Taiwan 61

The scenario n In the Prisoner’s Dilemma, A and B are picked up by

The scenario n In the Prisoner’s Dilemma, A and B are picked up by the police and interrogated in separate cells without the chance to communicate with each other. 2020/10/2 Computation Theory Lab, CSIE, CCU, Taiwan 62

Both are told: · If you both confess, you will both get in prison.

Both are told: · If you both confess, you will both get in prison. 4 years · If neither of you confesses, the police will be able to pin part of the crime on you, and you’ll both get 2 years. · If one of you confesses but the other doesn’t, the confessor will make a deal with the police and will go free while the other one goes to jail for 5 years. 2020/10/2 Computation Theory Lab, CSIE, CCU, Taiwan 63

Payoff Table B does not confess A confesses 2020/10/2 B confesses 2, 2 5,

Payoff Table B does not confess A confesses 2020/10/2 B confesses 2, 2 5, 0 0, 5 4, 4 Computation Theory Lab, CSIE, CCU, Taiwan 64

Question: Does each game have a unique equilibrium? 2020/10/2 Computation Theory Lab, CSIE, CCU,

Question: Does each game have a unique equilibrium? 2020/10/2 Computation Theory Lab, CSIE, CCU, Taiwan 65

Matching Pennies 2020/10/2 Head Tail Head 1, 1 1, 1 Tail 1, 1 Computation

Matching Pennies 2020/10/2 Head Tail Head 1, 1 1, 1 Tail 1, 1 Computation Theory Lab, CSIE, CCU, Taiwan No Nash equilibrium 66

Go to movie together, but action or romance? Action Romance Action 2, 1 0,

Go to movie together, but action or romance? Action Romance Action 2, 1 0, 0 Romance 0, 0 1, 2 Two Nash equilibriums occurs!! 2020/10/2 Computation Theory Lab, CSIE, CCU, Taiwan 67

Focus 2 -person 0 -sum games – the sum of payoffs of two players

Focus 2 -person 0 -sum games – the sum of payoffs of two players is zero in each cell of the table 2020/10/2 Computation Theory Lab, CSIE, CCU, Taiwan 68

2 -person 0 -sum games c n It suffices to list the payoffs for

2 -person 0 -sum games c n It suffices to list the payoffs for the rowplayer. r 2020/10/2 M (r; c) Computation Theory Lab, CSIE, CCU, Taiwan 69

An observation Try to prove it yourself as a simple exercise. LHS = the

An observation Try to prove it yourself as a simple exercise. LHS = the maximum payoff that the row-player can guarantee for himself. 2020/10/2 RHS = the minimum loss that the column-player can guarantee for himself. Computation Theory Lab, CSIE, CCU, Taiwan 70

: greater than : minimum of the row : maximum of the column Proof

: greater than : minimum of the row : maximum of the column Proof by contradiction: Min-max Max-min 2020/10/2 Computation Theory Lab, CSIE, CCU, Taiwan 71

For example, 2020/10/2 剪刀 石頭 布 剪刀 0 1 1 石頭 1 0 1

For example, 2020/10/2 剪刀 石頭 布 剪刀 0 1 1 石頭 1 0 1 布 1 1 0 Computation Theory Lab, CSIE, CCU, Taiwan 72

Another example 剪刀 饅頭 布 剪刀 0 1 2 饅頭 1 0 1 布

Another example 剪刀 饅頭 布 剪刀 0 1 2 饅頭 1 0 1 布 2 1 0 Saddle point 2020/10/2 Computation Theory Lab, CSIE, CCU, Taiwan 73

Not every 2 -person 0 -sum game has a saddle point n However, with

Not every 2 -person 0 -sum game has a saddle point n However, with respect to “mixed strategies”, mixed strategies von Neumann showed that each 2 -person 0 sum game has a saddle point. 2020/10/2 Computation Theory Lab, CSIE, CCU, Taiwan 74

Pure strategies versus mixed strategies n A mixed strategy is a probability distribution over

Pure strategies versus mixed strategies n A mixed strategy is a probability distribution over the set of all pure strategies. Let p be a mixed strategy for the row player and q be a mixed strategy for the column player. That is, pr is the probability for the row player to use his rth strategy, and qc is the probability for the column player to use his c-th strategy. Then the expected payoff with respect to p and q is 2020/10/2 Computation Theory Lab, CSIE, CCU, Taiwan 75

von Neumann’s Minimax. Theorem n For any two-person 0 -sum game, we have n

von Neumann’s Minimax. Theorem n For any two-person 0 -sum game, we have n That is, each two-person 0 -sum game has a saddle point with respect to mixed strategies. 2020/10/2 Computation Theory Lab, CSIE, CCU, Taiwan 76

Loomis’ Theorem n For any two-person 0 -sum game, we have where ei means

Loomis’ Theorem n For any two-person 0 -sum game, we have where ei means running the i-th strategy with probability 1. n To see this theorem, just observe that when p is known, the column player has an optimal strategy that is a pure strategy. A similar observation holds for the row player, too. 2020/10/2 Computation Theory Lab, CSIE, CCU, Taiwan 77

Yao’sinterpretation n n The row player = the maximizer = the adversary responsible for

Yao’sinterpretation n n The row player = the maximizer = the adversary responsible for designing malicious inputs. The column player = the minimizer = the algorithm designer responsible for designing efficient algorithms. 2020/10/2 Computation Theory Lab, CSIE, CCU, Taiwan 78

Pure strategies n For the column player (minimizer) – Each pure strategy corresponds to

Pure strategies n For the column player (minimizer) – Each pure strategy corresponds to a deterministic algorithm. n For the row player (maximizer) – Each pure strategy corresponds to a particular input instance. 2020/10/2 Computation Theory Lab, CSIE, CCU, Taiwan 79

It is related to the non-deterministic complexity of the problem It stands for the

It is related to the non-deterministic complexity of the problem It stands for the worst-case running time of any deterministic algorithm for the problem. 2020/10/2 Computation Theory Lab, CSIE, CCU, Taiwan 80

Mixedstrategies n For the column player (minimizer) – Each mixed strategy corresponds to a

Mixedstrategies n For the column player (minimizer) – Each mixed strategy corresponds to a randomized algorithm. n For the row player (maximizer) – Each mixed strategy corresponds to a probability distribution over all the input instances. 2020/10/2 Computation Theory Lab, CSIE, CCU, Taiwan 81

Yao’sinterpretation for Loomis’ Theorem n Let T(I, A) denote the time required for algorithm

Yao’sinterpretation for Loomis’ Theorem n Let T(I, A) denote the time required for algorithm A to run on input I. Then by Loomis’ theorem, we have n Therefore, the following inequality holds for any probbility distribution p and q: 2020/10/2 Computation Theory Lab, CSIE, CCU, Taiwan 82

Yao’s. Inequality n Let be a problem. Let D consist of all deterministic algorithms

Yao’s. Inequality n Let be a problem. Let D consist of all deterministic algorithms for solving . Let I consist of all possible input instances for . Let Ip be denote the input instance of with probability distribution p. Then the following inequality holds for any probability distribution p and any Las Vegas randomized algorithm R. min expected time of A running on Ip A D max expected time of R running on I I I 2020/10/2 Computation Theory Lab, CSIE, CCU, Taiwan 83

A comment n The two different topics “probabilistic analysis for deterministic algorithms” and “randomized

A comment n The two different topics “probabilistic analysis for deterministic algorithms” and “randomized algorithms” interact by Yao’s Principle. 2020/10/2 Computation Theory Lab, CSIE, CCU, Taiwan 84

How to use. Yao’s. Lemma? n Task 1: – Design a probability distribution p

How to use. Yao’s. Lemma? n Task 1: – Design a probability distribution p for the input instance. n Task 2: – Obtain a lower bound on the expected running for any deterministic algorithm running on Ip. 2020/10/2 Computation Theory Lab, CSIE, CCU, Taiwan 85

牛刀小試 on Yao’s. Principle A lower bound (n 0. 694) on the expected running

牛刀小試 on Yao’s. Principle A lower bound (n 0. 694) on the expected running time of any Las Vegas algorithms for evaluating NOR circuits 2020/10/2 Computation Theory Lab, CSIE, CCU, Taiwan 86

Task 1: designing. Ip x 1 x 2 x 3 x 4 y x

Task 1: designing. Ip x 1 x 2 x 3 x 4 y x 5 x 6 x 7 x 8 2020/10/2 Computation Theory Lab, CSIE, CCU, Taiwan 87

An Ip Let each leaf of the NOR circuit be independently assigned value 1

An Ip Let each leaf of the NOR circuit be independently assigned value 1 with probability and value 0 with probability 1 – p. 2020/10/2 Computation Theory Lab, CSIE, CCU, Taiwan 88

Interestingly, The output of each NOR gate also has value 1 with probability (As

Interestingly, The output of each NOR gate also has value 1 with probability (As a matter of fact, the above equation is how we obtain the value of p in the first place. ) 2020/10/2 Computation Theory Lab, CSIE, CCU, Taiwan 89

Depth-first evaluation n Recall that we can focus on deterministic depth-first evaluation algorithms. Let

Depth-first evaluation n Recall that we can focus on deterministic depth-first evaluation algorithms. Let A be an arbitrary algorithm of this kind. Let W(k) be the time required for A to evaluate the circuit on Ip with 2 k numbers. So, W(k) = W(k – 1) + (1 – p) W(k – 1), implying W(k) = ((2 – p)k) = (n 0. 694). 2020/10/2 Computation Theory Lab, CSIE, CCU, Taiwan 90

A comment n Saks and Wigderson proved that Rand. Eval is optimal by designing

A comment n Saks and Wigderson proved that Rand. Eval is optimal by designing a much complicated Ip. – The values of the inputs are chosen at random but not independently. 2020/10/2 Computation Theory Lab, CSIE, CCU, Taiwan 91

向左走 向右走 n The two different topics “probabilistic analysis for deterministic algorithms” and “randomized

向左走 向右走 n The two different topics “probabilistic analysis for deterministic algorithms” and “randomized algorithms” meet at Yao’s Principle. 2020/10/2 Computation Theory Lab, CSIE, CCU, Taiwan 92

Exercises n Use Yao’s Minmax Principle to prove a lower bound on the expected

Exercises n Use Yao’s Minmax Principle to prove a lower bound on the expected running time of any Las Vegas algorithm for sorting n numbers. n Determine the value VR of the following 2 2 matrix game and give optimal mixed strategies for the two players. 2020/10/2 Computation Theory Lab, CSIE, CCU, Taiwan 93

n Consider a Monte Carlo algorithm A for a problem whose expected running time

n Consider a Monte Carlo algorithm A for a problem whose expected running time is at most T(n) on any instance of size n and that produces a correct solution with probability (n). Suppose further that given a solution to , we can verify its correctness in time t(n). Show to obtain a Las Vegas algorithm that always gives a correct answer to and runs in expected running time at most (T(n) + t(n)) / (n). – Hint: Consider the geometric random variable. 2020/10/2 Computation Theory Lab, CSIE, CCU, Taiwan 94

Thank you.

Thank you.