Algorithms with Random Walks 2 SAT by Random
Algorithms with Random Walks • 2 SAT by Random Walks • 3 SAT by Random Walks 1
The k-SAT problem Input: m clauses C 1 ; C 2 ; : : : ; Cm with j. Cj j · k on n variables 1; 2; : : : ; n. Output: Is there a truth assignment that satis¯es all m clauses C 1 ; C 2 ; : : : ; Cm ? 2
Examples Input: fx 1 ; : x 3 g; f: x 2 ; x 3 g; f: x 1 ; : x 2 g; fx 2 ; x 3 g. Output: Yes. (x 1 = x 3 = true and x 2 = false. ) Input: fx 1 ; : x 2 g; fx 2 ; : x 3 g; fx 3 ; : x 1 g; f: x 1 ; : x 3 g. Output: Yes. (x 1 = x 2 = x 3 = false). Input: fx 1 ; : x 2 g; fx 1 ; x 2 g; fx 3 ; : x 1 g; f: x 1 ; : x 3 g. Output: No. 3
Facts • NP-complete – 3 -SAT • P – 2 -SAT • Solvable in linear time [Aspvall-Plass-Tarjan, Information Processing Letters, 1979] 4
A Simple Monte Carlo Algorithm for 2 SAT Papadimitriou, FOCS 1991 5
Papadimitriou’s algorithm Initially, let xi = true for each i = 1; 2; : : : ; n; Repeat the following steps for 2 n 2 iterations f If the input set of clauses is currently satis¯ed then Output the current truth assignment and halt; else f Let Cj be a currently unsatis¯ed clause; Pick one of the two literals in Cj uniformly at random; Flip the truth value of the chosen literal; g g Output that the input set of clauses is unsatis¯able. 6
We will see that … • If the input set of clauses is satisfiable, Papadimitriou’s algorithm correctly outputs a satisfying truth assignment with probability at least ½. • If the input set of clauses is unsatisfiable, Papadimitriou’s algorithm always outputs the correct answer. 7
Random Walks 凌波微步 8
Random Walks on Graphs • A random walk on a graph G, starting from vertex s, is the following random process – Let u = s; – Repeat for t steps: • Choose a neighbor v of u uniformly at random; • Let u = v; 10
Illustration 11
A key quantity F H(u; v): hitting time from u to v. The expected number of steps for a random walk starting from u to visit v for the ¯rst time. 12
Illustration Hitting time from u to v is 6. Hitting time from x to y is 10. y v u x 13
Random walks on Ln+1 0 1 2 3 n The expected number of iterations for 0 Papadimitriou s algorithm to ¯nd a satisfying truth assignment, if any, is at most H(n; 0) on Ln+1. 14
Why? ¤ Assume that x is an arbitrary but ¯xed truth assignment that satis¯es the input set of clauses. Let ¢ denote the number of variables whose values are ¤ di®erent from those of x. F At any moment, we have ¢ 2 f 0; 1; 2; : : : ; ng. Moreover, when ¢ = 0, the current assignment ¤ is x. F Each iteration of the loop either increases or decreases the value of ¢ by exactly one. Morover, the probability of decreasing ¢ by one is at least 0: 5. 15
Why? If ¢ > 0, then the current truth assignment is dif¤ ¤ ferent from x. Since x satis¯es every single clause (including Cj ), at least one literal of Cj has di®erent ¤ value from x. Therefore, °ipping the value of one randomly chosen literal of Cj decreases the value of ¢ by one with probability at least 0: 5. 16
Random walks on Ln+1 0 1 2 3 n The expected number of iterations for Papadimitrious algorithm to ¯nd a satisfying truth assignment, if any, is at most H(n; 0) on Ln+1. 17
Why? • Reason 1 – When measuring H(n, 0) on Ln+1, the probability of going left is exactly 0. 5. – In Papadimitriou’s algorithm, the probability of going left is at least 0. 5. • Reason 2 – When measuring H(n, 0) on Ln+1, the random walk has to visit node 0. – In Papadimitriou’s algorithm, the algorithm starts from a node much closer to node 0 and may stops at a node representing another satisfying truth assignment. 18
We will show that • H(n, 0) on Ln+1 is n 2. • It follows from Markov’s Inequality that if the input set of clauses is satisfiable, then Pr[#iteration ≥ 2 n 2] ≤ 0. 5. 1 ¢ ¸ · : Pr[X k E[X]] k 19
An intriguing theorem Hitting time H of random walks in graphs v. s. Effective resistance R in an electrical network 20
Electrical Networks 101 • Kirchoff’s Law: – The total current flowing into any node is equal to the total current flowing out of it. • Ohm’s Law: – The potential difference across any wire is equal to the current along it times its resistence. 21
Kirchoff’s Principle of Superposition 22
Ohm’s Law • V=I*R 23
Effective resistance 等效電阻 • Serial: – R = R 1 + R 2 R 1 • Parallel: – R = R 1 * R 2 / (R 1 + R 2). R 1 R 2 24
Theorem [Chandra et al. STOC 1989] • For any two nodes u and v of a simple medge graph G, H(u, v) + H(v, u) = 2 m. R(u, v), where R(u, v) is the effective resistance in G between nodes u and v with respect to the setting that each edge of G is equipped with one unit of resistance. 25
Illustration 1 A u G 1 A v The e®ective resistance R(u; v) is the corresponding potential di®erence between u and v with one unit of current inserting into u and one unit of current drawing out of v. 26
So, H(n, 0) = n 2 on Ln+1 • H(n, 0) + H(0, n) = 2 m * R(n, 0) = 2 n 2. • Clearly, H(n, 0) = H(0, n). Therefore, we have H(n, 0) = n 2. 0 n 27
A question • Adding edges to the underling graph always decreases H(u, v)? 28
How about H(u, v) on LP 2 n? H(u; v) + H(v; u) = £(n 3 ). H(v; u) = n 2. Therefore, H(u; v) = £(n 3 ). n-node complete graph u v n nodes 29
Let us prove theorem! • For any two nodes u and v of a simple medge graph G, H(u, v) + H(v, u) = 2 m*R(u, v), where R(u, v) is the effective resistance in G between nodes u and v with respect to the setting that each edge of G is equipped with one unit of resistance. 30
Scenario 1 deg(x) x 2 m v deg(v) G y z deg(y) deg(z) Drawing 2 m units of current out of v. For each node u, inserting deg(u) units of current into u. 31
Claim 1 • In Scenario 1, V(u, v) = H(u, v) holds for each node u of G, where V(u, v) is the resulting potential difference between u and v. • Strategy of the proof: We show that V and H satisfy the same set of equations. 32
The equations H(u; v) = 1 + X (u; x)2 G V (u; v) = 1 + X (u; x)2 G H(x; v) ; deg(u) (1) V (x; v) : deg(u) (2) 33
Equation (1) • Straightforward by definition of H(). 34
Equation (2)By Kirchoff deg(u) = By Ohm X (current from u to x) (u; x)2 G = X V (u; x)2 G = X (u; x)2 G V (u; v) ¡ V (x; v) = deg(u) ¢ V (u; v) ¡ X V (x; v): (u; x)2 G 35
Scenario 2 deg(x) x G y z 2 m deg(y) deg(z) u deg(u) Inserting 2 m units of current into u. For each node v, drawing deg(v) units of current out of v. 36
Claim 2 • In Scenario 2, V’(u, v) = H(v, u) holds for each node v of G, where V’(u, v) is the resulting potential difference between u and v. • Proof: By its symmetry to Scenario 1. 37
Scenario 3 = Scenarios 1+2 2 m G v 2 m u The potential di®erence between u and v is 2 m ¤ R(u; v). 38
By Claims 1 and 2 • H(u, v) + H(v, u) = V(u, v) + V’(u, v). • It is not hard to see that V(u, v) + V’(u, v) is the potential difference 2 m * R(u, v) between u and v in Scenario 3. • Therefore, H(u, v) + H(v, u) = 2 m * R(u, v). The theorem is proved! 39
Theorem • For any two nodes u and v of a simple medge graph G, H(u, v) + H(v, u) = 2 m. R(u, v), where R(u, v) is the effective resistance in G between nodes u and v with respect to the setting that each edge of G is equipped with one unit of resistance. 40
A randomized algorithm for 3 SAT • Start with an arbitrary truth assignment. • Repeat up to m times, terminating if all clauses are satisfied. (a) Choose any clause that is not satisfied. (b) Randomly choose one of its literals and change the value of the variable. • If a valid truth assignment has been found, return it. • Otherwise, return that the formula is unsatisfiable. 41
Analysis: • Assume the formula is satisfiable and let S be a satisfying assignment. • Let Ai be the assignment after i steps. • Let Xi be the number of variables in Ai that match S. • For j=1, . . . , n-1, Pr[Xi+1 = j+1 | Xi = j ] 1/3; Why? Pr[Xi+1 = j -1 | Xi = j ] 2/3. 42
Analysis: • We can obtain an upper bound on the number of steps until Xi = n by analyzing a Markov chain Y 0, Y 1, . . such that Y 0=X 0 and Pr[Yi+1 = 1 | Yi = 0 ] = 1, Pr[Yi+1 = j+1 | Yi = j ] = 1/3, Why? Pr[Yi+1 = j -1 | Yi = j ] = 2/3. • Let hj be the expected number of steps to reach n when starting from j. Then hn = 0; hj = 2 hj-1/3 + hj+1/3 + 1, j=1. . , n-1; h 0 = h 1 + 1. 43
Analysis: • hj = 2 hj-1/3 + hj+1/3 + 1, j=1. . , n-1; 2(hj – hj-1 )/3 = (hj+1 - hj )/3 + 1. • Let Zi = hi – hi-1, then 2 Zi = Zi+1 + 3 and Z 1 =-1. • We can solve this recurrence and obtain Zi = -2 i+1 + 3, for i= 2, . . . , n. • By telescoping, we have h 0 = 2 n+2 -3 n – 4. 44
Analysis: • h 0 = 2 n+2 -3 n – 4 mean that the randomized algorithm takes O(2 n) on average to find a satisfying assignment! ---Not good at all! • Observe that (1) If we uniformly find an initial truth assignment, then it match S with expectation n/2. (2) Once the algorithm starts, it is more likely move toward 0 than n. Thus we try many initial assignments and for each we run a small number of steps. 45
A modified 3 SAT algorithm • Repeat up to m times, terminating if all clauses are satisfied: (1) Start with an arbitrary truth assignment. (2) Repeat up to 3 n times, terminating if a satisfying assignment found: (a) Choose any clause that is not satisfied. (b) Randomly choose one of its literals and change the value of the variable. • If a valid truth assignment has been found, return it. • Otherwise, return that the formula is unsatisfiable. 46
Analysis: • Let q be the probability that the modified algorithm reaches S or some other satisfying assignment in 3 n steps. • Let qj be a lower bound on the probability that the modified algorithm reaches S within j+2 k ≤ 3 n steps when starts with a truth assignment that disagree with S on j variables. ¡ ¢ 2 )k ( 1 )j+k. ( Note that qj ¸ maxk=0; : : : ; j j+2 k 3 3 k ¡ ¢ 2 )j ( 1 )2 j , when k = j. ( In particular, qj ¸ 3 j 3 3 j 47
Analysis: [Stirlings formula]: p p m · m! · 2 2¼m( m )m : ) For m > 0, 2¼m( m e e Hence when j > p 0, ¡ 3 j ¢ (3 j)! ¸ p 3 (27=4)j = pc (27=4)j : = j j!(2 j)! 8 ¼j j ¡ 3 j ¢ ¸ 2 )j ( 1 )2 j ( qj 3 3 j ¸ pc (27=4)j ( 2 )j ( 1 )2 j 3 3 j ¸ pc (1=2)j. j Note that q 0 =1. 48
Analysis: Pn ¸ q qj Pr[has j mismatches with S: ] 0 j= ¡ ¢ P ¸ 1 + n n n pc (1=2)j (1=2) j =1 j 2 n j ¡ ¢ P ¡j ¸ pc (1=2)n n n j n (1=2) (1) = = j =0 j n pc (1=2)n (3=2)n n pc (3=4)n : n • Assuming satisfiable, the number of random assignments used before finding a satisfying one is a geometric random variable with parameter q. • The expected number of assignments tried is 1/q and each runs at most 3 n steps. • Total expected steps: O(n 3/2(4/3)n). 49
- Slides: 49