Classical and quantum algorithms for Boolean satisfiability Ashley

Classical and quantum algorithms for Boolean satisfiability Ashley Montanaro

Talk structure Intro to Boolean satisfiability (SAT) n Classical algorithms n Quantum algorithms n Query complexity lower-bound results n

What is SAT? n The problem of finding an assignment to a set of variables that satisfies a given Boolean logical expression E n For example: E = (a v b) ^ (a v ¬b) ^ (¬a v c) ^ (¬c v b) ¨ n But if we change the last clause, thus: E = (a v b) ^ (a v ¬b) ^ (¬a v c) ^ (¬c v ¬a) ¨ n has satisfying assignment a = b = c = TRUE this formula is not satisfiable There are obviously 2 n possible assignments to the n variables, so exhaustive search takes time O(2 n)

Why is SAT important? n It’s NP-complete ¨ if we can solve SAT quickly, we can solve anything in NP quickly (Cook’s theorem, 1971) n Many and varied applications in itself: ¨ theorem proving ¨ hardware design ¨ machine vision ¨. . . n In fact, any problem where there exist constraints that have to be satisfied!

Some restricted versions of SAT n We generally consider the case where the expression E is in CNF, i. e. is made up of clauses of ORs linked by ANDs: ¨ (a v ¬b v. . . ) ^ (¬c v d v. . . ) ^. . . n Thus it’s hard to find a satisfying assignment, but easy to find an unsatisfying one; DNF is the opposite n Other variants: Horn-SAT [clauses with all but 1 negation] ¨ MAX-SAT [find the maximum number of satisfied clauses] ¨ NAESAT [all literals in a clause not allowed to be TRUE] ¨. . . ¨

k-SAT n If the maximum number of variables in each clause is k, we call the problem k-SAT n 1 -SAT is simple: E = a ^ ¬b ^. . . ¨ and n can be solved in time O(n) 2 -SAT is also straightforward ¨ can be solved in time O(n 2) using a simple random walk algorithm n 3 -SAT is NP-complete ¨ eek!

Classical algorithms for SAT Davis-Putnam n Depth-first search n Random walk algorithms n Greedy local search n. . . many others. . . n

The Davis-Putnam algorithm (1960) n Uses the fact that clauses like ¨ n (a v b v c) and (a v b v ¬c) can be “simplified” to ¨ (a v b) n This simplification process is called resolution n Algorithm: keep on resolving until you find a contradiction, otherwise output “satisfiable” n Impractical for real-world instances (exponential memory usage normally required)

DPLL algorithm n Davis, Logemann, Loveland (1962) n Basic idea: depth-first search with backtracking on the tree of possible assignments n This idea is common to many modern SAT algorithms n Still exponential time in worst case, but lower memory usage

Example: solving (a v b) ^ (a v ¬c) ^ (¬a v b) ^ (a v c) a 0 1 b 0 b 1 c 0 c 1 c c 0 1 0 1 û û û ü ü

Example: solving (a v b) ^ (a v ¬c) ^ (¬a v b) ^ (a v c) a 0 1 b 0 b 1 0 c û 1 c û 0 1 û û ü ü

Random walk algorithms n Schöning developed (1999) a simple randomised algorithm for 3 -SAT: ¨ start with a random assignment to all variables ¨ find which clauses are not satisfied by the assignment ¨ flip one of the variables which features in that clause ¨ repeat until satisfying assignment found (or 3 n steps have elapsed) n This simple algorithm has worst-case time complexity of O(1. 34 n) ¨ and it’s (almost) the best known algorithm for 3 -SAT

Example: solving (a v b) ^ (a v ¬c) ^ (¬a v b) ^ (a v c) 110 111 010 011 100 101 000 001

Example: solving (a v b) ^ (a v ¬c) ^ (¬a v b) ^ (a v c) 110 111 010 011 100 101 000 001

How does it work? n It’s almost a simple random walk on the hypercube whose vertices are labelled by the assignments n Apart from the crucial step: ¨ “flip one of the variables which features in that clause” n This turns it into a walk on a directed graph with the same topology n We can use theory of Markov chains to determine its probability of success, and hence its expected running time

The directed graph of (a v b) ^ (a v ¬c) ^ (¬a v b) ^ (a v c) 110 111 010 011 100 101 000 001

Turning the random walk into a quantum walk n Is it possible to convert Schöning’s algorithm into a quantum walk in a straightforward way? n No! The algorithm performs a walk on a directed graph with sinks (the satisfying assignments) n It turns out that quantum walks cannot be defined easily on such graphs n If we remove the “directedness”, we end up with simple unstructured search

Greedy local search (GSAT) n Selman, Levesque, Mitchell (1992) n Similar to random walk, but only accept changes that improve the number of satisfied clauses ¨ (but sometimes accept changes that don’t, to avoid local minima) n Worse than the simple random walk in a worstcase scenario ¨ finds it too easy to get stuck in local minima

Classical upper bounds for k-SAT k Time Who 3 O(1. 324 n) Iwama, Tamaki (2003) 4 O(1. 474 n) Iwama, Tamaki (2003) 5 O(1. 6 n) Schöning (1999) n O(1. 2388 m) Hirsch (2000) (m is the number of clauses; note that the algorithms for cases k=3, 4, 5 are randomised)

Quantum algorithms for SAT Unstructured search n Multi-level unstructured search n Hogg’s algorithm n Adiabatic evolution n

Unstructured search n Don’t use any knowledge of the problem’s structure; just pass in an assignment and ask “does this satisfy the expression? ” n Well-known that you can find a satisfying assignment in O(1. 42 n) tests of satisfiability using Grover’s algorithm n The other quantum algorithms given here don’t do much better. . .

Multi-level unstructured search n Idea: perform a Grover search on a subset of the variables, then nest another search within the subspace of those variables that satisfies the expression ¨ ¨ n Results in an average case O(1. 27 n) query complexity for 3 -SAT ¨ ¨ n for 3 -SAT, optimal “nesting level” is ~2/3 of the variables can think of it as a natural quantum analogue of the DPLL algorithm worse than the square root of the best classical algorithm could this be because expressions are very sensitive to the values of all the variables they contain? Due to Cerf, Grover & Williams 1. quant-ph/9806078

Multi-level search example n Let’s solve (a v b) ^ (a v ¬c) ^ (¬a v b) ^ (a v c) n First, search in the space of (a, b); ie. find the satisfying assignments to (a v b) ^ (¬a v b) n This will give us a superposition |0 a 1 b> + |1 a 1 b> n Now search for a satisfying assignment to the original expression in this space ¨ ending up with a (correct) superposition |1 a 1 b 0 c>+|1 a 1 b 1 c>

Hogg’s algorithm 1 n Works in a similar way to Grover’s algorithm ¨ in fact, Grover’s algorithm is a special case of it n Starts with a superposition over all assignments, then combines phase rotations Pt (based on the number of conflicts in a given assignment) with “mixing” matrices Mt : n |jend> = Mn. Pn. . . M 1 P 1|+> n These matrices are heuristically parametrised, and change over the course of the algorithm, becoming closer to the identity 1. quant-ph/0104048

Hogg’s algorithm (2) n Phase matrix (problem-dependent): Pii = eiπ K c(i) where K changes throughout the run and c(i) is the number of conflicts in assignment i ¨ compare Grover phase oracle Pii = -(-1 f(i) ) ¨ n Mixing matrix (problem-independent): M = Hxn * T * Hxn [ Tii = eiπ L w(i) ] where L changes throughout the run and w(i) is the Hamming weight of the binary string i ¨ compare Grover diffusion Tii = -(-1(δi 1)) ¨ n n Values Mab in mixing matrix are only dependent on distance(a, b) Values Pii in phase matrix are only dependent on number of conflicts in assignment i

Performance of Hogg’s algorithm n 1 -SAT can be solved in 1 step with this algorithm the number of conflicts in a 1 -SAT assignment is the same as its distance from the solution ¨ so we can choose our mixing matrix cleverly to destroy those assignments with >0 conflicts ¨ n For k-SAT, the number of conflicts provides a rapidly worsening estimate of the distance; we have to use heuristics to try to adjust the estimate n No rigorous worst-case analysis done, but simulation on (small) hard random instances of 3 -SAT suggests an average case query complexity of O(1. 05 n)

Adiabatic evolution n Uses the quantum adiabatic theorem n Idea: start in the ground state of a known Hamiltonian, and continuously evolve to the unknown ground state of a “solution” Hamiltonian n The solution Hamiltonian is set up so its lowest energy eigenstate is the state with no conflicts (ie. the solution) n No rigorous analysis of its power has been made, but it’s known that problem instances exist that take exponential time (e. g. van Dam et al 1) ¨ n these rely on a very large local minimum, and a hard-to-find global minimum Due to Farhi et al 2 1. quant-ph/0206003 quant-ph/0001106 2.

Lower bounds for these algorithms n Proving lower bounds on time complexity is a bit tricky n One way we can do it for quantum algorithms is to consider query complexity n All of the algorithms mentioned here use oracles – black boxes which give us the answer to a question n If we can put a bound on the minimum number of calls to these oracles, this gives us an idea of the time complexity of the algorithms

Oracle models in SAT (1) n n These quantum algorithms use (implicitly or otherwise) the following oracles: “Black box” ¨ Grover’s x algorithm, multi-level Grover search f(x, E) 1 if x satisfies E 0 if x doesn’t

Oracle models in SAT (2) n “Conflict counting” ¨ Hogg’s x algorithm, adiabatic algorithm f(x, E) The number of clauses in E that x doesn’t satisfy

Oracle models in SAT (3) n Another obvious oracle is “clause satisfaction” ¨ not x used by any algorithms so far. . . f(x, c, E) 1 if x satisfies clause #c of E 0 if x doesn’t

Lower bounds for oracle models n We consider bounds in the number of calls to these oracles – aka query complexity n Adversary method used: consider multiple instances of the problem – i. e. multiple oracles – that are somehow “close” but different ¨ show a limit on the amount any two instances can be distinguished with one oracle call ¨ work out how many oracle calls are needed to distinguish them all ¨ n Several different formulations of the method ¨ all known formulations have been shown to be equivalent 1 1. quant-ph/0409116

Geometric adversary method n Summed over a set of N oracles, consider the largest possible overlap |x. G> of an input |x> with the “good” states – ie. ones for which the oracle returns 1 ¨ intuitively, the “best” value of |x> to input for any instance of the problem will produce the largest overlap n Can show that T 2 ≥ N / ∑ || |x. G>||2 ¨ proof omitted

Lower bounds for oracle models (2) n Unstructured search is well-known to have a lower bound of W(2 n/2) queries n This implies that the multi-level search should have the same worst-case lower bound, as it uses the same oracle n To put a bound on the other oracle models, we pick instances of SAT such that they essentially reduce down to unstructured search; i. e. so that the more powerful oracles are no help to us

Lower bound for the “conflict counting” oracle n We consider a set of 2 n instances of SAT, each of which has a single and different satisfying assignment n Each instance has n clauses, varying in length from 1 to n variables n Set the clauses up so none of them “overlap” – i. e. cause conflicts with more than one assignment n The number of conflicts will then be 1 for every assignment, bar the satisfying assignment: the oracle becomes no more powerful than unstructured search n So we can show the minimum query complexity is W(2 n/2)

Example expression used a ^ (¬a v b) ^ (¬a v ¬b v c) 000 001 010 011 100 101 110 111 Each assignment satisfies all but one clause

Lower bound for the “clause satisfaction” oracle n A similar approach. But this time, we need more clauses n Consider a set of 2 n expressions which have different, unique satisfying assignments. Each expression has 2 n clauses, and each clause of each expression contains all n variables n Can then show a bound of W(2 n/2) queries, extensible to W(sqrt(m)), where m is the number of clauses n Considerably weaker! We need exponential input size to show an exponential lower bound

Example expression used (a v b v c) ^ (a v b v ¬c) ^ (a v ¬ b v ¬ c) ^ (¬ a v ¬ b v c) 000 001 010 011 100 101 110 111

Do these results extend to k-SAT? n No! van Dam et al 1 have shown that, for 3 -SAT, an algorithm using the conflict counting oracle can recover the input in O(n 3) calls to the oracle n I’ve extended this to k-SAT to show that the input can be recovered in O(nk) calls n Idea behind this: once you know about the number of conflicts in all the assignments of Hamming weight k or less, you can work out the number of conflicts for all other assignments without needing to call the oracle again 1. quant-ph/0206003

Conclusion n SAT has been known for 50 years, but classical algorithms to solve it are still improving n Quantum algorithms haven’t beaten the performance of classical ones by much – if at all n Thinking about the oracle models we use – implicitly or otherwise – gives us clues to how we should develop quantum algorithms n It looks like no algorithm can solve SAT quickly without “looking inside” the clauses n It’s also clear that we can’t prove any lower bounds for k. SAT using these restricted oracle methods