Chapter 7 Introduction to Computational Complexity Reductions and

Chapter 7 Introduction to Computational Complexity: Reductions and Completeness Complexity Classes

Time and Space versus Energy and Mass In actual (real) machines, what is the physical relationship between: 1. Speed (rate of execution) and energy consumption? 2. Memory (amount of storage) and mass required? What are the (ultimate) physical limits to these? 1. Thermodynamics (heat) 2. Gravitation (black holes) 12/6/2020 Theory of Computation: Chapter 7 2

Space complexity (and tape compression) Definition: A language is (computable) in space S(n) if it is accepted by a machine which never requires more than S(|w|) cells on any input w (ignore any read-only input tape or write-only output tape). Lemma: If L is in space S(n), then L is in space c. S(n) for any positive c. Proof (sketch): Let M be an S(n) space-bounded machine accepting L. Represent the contents of two adjacent cells of M’s work tape by a single symbol (square the size of the work tape alphabet). Using an enhanced finite control, keep track of individual moves within each pair of cells. This uses only half the space, ½S(n). Continue for any c > 0. • Functions are the same up to a constant factor of proportionality. 12/6/2020 Theory of Computation: Chapter 7 3

Time complexity (and speed up) Definition: L is (computable) in time T(n) if it is accepted by a machine which never requires more than T(|w|) steps on any input w (insist T is super-linear, i. e. T(n) ≥ n, so that we can examine the entire input). Lemma: If L is in time T(n), then L is in time c. T(n) for any positive c. Proof (sketch): Let M be a T(n) time-bounded machine accepting L. Compress the contents of two cells of the input tape to the work tape in one pass (using a larger alphabet, as before). When entering this cell pair from the right or left, anticipate in an enhanced finite control the finitely many behaviors of M on the pair, and which direction it exits. • Functions are the same up to a constant factor of proportionality. 12/6/2020 Theory of Computation: Chapter 7 4

Growth rates as measures of complexity Input size: 10⁰ 10³ 10⁶ 10⁹ 10¹² 10¹⁵ 10¹⁸ 10²¹ • Logarithmic • Linear • Polynomial • Exponential 12/6/2020 Theory of Computation: Chapter 7 5

(Non)-deterministic hierarchies • Assume a two-tape model (an input tape with a separate work tape). • Let (N)SPACE(S(n)) = {L : L is in (non)-deterministic space S(n)}. • Let (N)TIME(T(n)) = {L : L is in (non)-deterministic time T(n)}. • Theorem: If S₁(n) ≥ log n and lim S₁(n)∕S₂(n) = 0 as n → ∞, then SPACE(S₁(n)) ⊊ SPACE(S₂(n)) and NSPACE(S₁(n)) ⊊ NSPACE(S₂(n)) • Theorem: If T₁(n) ≥ n and lim T₁(n) log T₁(n)∕T₂(n) = 0 as n → ∞, then TIME(T₁(n)) ⊊ TIME(T₂(n)) and NTIME(T₁(n)) ⊊ NTIME(T₂(n)) Proofs: use diagonalization 12/6/2020 Theory of Computation: Chapter 7 6

Automata theoretic separation results What are the time and space complexities of the regular languages? REG = SPACE(1) ⊊ TIME(n) What are the complexities of the context-free languages? CFL ⊊ NSPACE(n); TIME(n³) How do these compare with each other, as well as phrase-structure? 12/6/2020 Theory of Computation: Chapter 7 7

Relationships between time and space classes Fact: Time(f(n)) ⊆ NTime(f(n)) ⊆ Space(f(n)) Proof (sketch): The first containment is trivial. Each branch in the computation tree has length at most f(n), so we can reuse space f(n) to depth-first simulate each possible run because of the bound f(n). 12/6/2020 Fact: Space(f(n)) ⊆ NSpace(f(n)) ⊆ Time(cf(n)) Proof (sketch): The first containment is trivial. The size of the configuration graph will be exponential in the space bound f(n). Search for a path from the start vertex to a halting configuration. This single source reachability problem is linear in the number of nodes. Theory of Computation: Chapter 7 8

Complexity Classes Resource bounded classes closed under: • Imperative: composition (pipelines) and nesting (subroutines) • Functional: non-recursive calls (sum) and recursive calls (product) Linear time or space satisfies the first closure type but not the second. • P = polynomial time = TIME(nk) for any k • L = logarithmic space = SPACE(logk n) for any k Introduce nondeterminism: NP = NTIME(nk) and NL = NSPACE(logk n) 12/6/2020 Theory of Computation: Chapter 7 9

Containments SPACE(1) = NSPACE(1) [finite automaton] strict separation from space hierarchy L NL follows from space time containment P NP follows from time space containment NSPACE(nk) ⊆ SPACE(n 2 k) [Savitch] 12/6/2020 Theory of Computation: Chapter 7 constant space logarithmic space non-deterministic " polynomial time non-deterministic " polynomial space 10

Savitch’s theorem Theorem: NSPACE(S(n)) ⊆ DSPACE(S²(n)) for any S(n) ≥ log n. Idea: Search the configuration graph G by path doubling. For simplicity, let |G| = n. Determine if there is a path of length ≤ n from start to final. For all u, v, ∈ G, notice that: u ⊦≤ 2ⁱ v ⇔ ∃w u ⊦≤ 2ⁱ⁻¹ w ⊦≤ 2ⁱ⁻¹ v ! Basis: for i = 0, u ⊦≤ 2⁰ v ⇔ u = v or u ⊦ v. Induction: for i > 0, test all w by reusing space, and see the stack of activation records uses O(S²(n)): entries on stack have size S(n); height of stack ≤ log c. S(n) = O(S(n)). 12/6/2020 Theory of Computation: Chapter 7 11

Different forms of nondeterminism (logspace) • Existential nondeterminism: some path leads to acceptance (NL) • Symmetric nondeterminism: moves are always symmetric (SL) • Universal nondeterminism: every path leads to acceptance (co-NL) Completeness phenomenon: there is a hardest problem in each class • SL = L reachability in an undirected graph • co-NL = NL reachability in a directed graph Know: NL ⊆ log² space Open: L = NL? • Does reachability require more memory on directed graphs? 12/6/2020 Theory of Computation: Chapter 7 12

Reductions Definition: We write L₁ ≤ L₂ if there is a way to solve x ∈ L₁ by making ‘calls’ to L₂ as an oracle, and say L₁ is reducible to L₂. The usual way is to find an “easy” function f so that one call suffices: w ∈ L₁ ⇔ f(w) ∈ L₂. Idea: L₁ is no harder than L₂. So that L₁ ≤ L₂ ∈ C ⇒ L₁ ∈ C. Caveat: The reduction f must be of smaller complexity than the class C. 12/6/2020 Theory of Computation: Chapter 7 13

Examples Idea: Solve one problem using the solution to another problem. Undirected reachability: Is there a path from s to t in G (simple graph)? Connected component: Return the smallest vertex c(v) connected to v. • Show these have the same complexity: linear time and log space 12/6/2020 Theory of Computation: Chapter 7 14

Another example • Directed reachability: Is there a directed path from s to t in digraph G? • Distance: What is the distance from s to t in an undirected graph G'? Claim: Directed reachability is reducible to distance in simple graphs. Reduction: Let G = �V, E�. Construct the simple graph G' from V × V with edges between adjacent copies �u, i�— �v, i + 1�for 1 ≤ i < n = |V| if u = v or u → v in G. Observe that �s, n�and �t, n�are equidistant from �s, 1�just in case there is a directed path from s to t. • The other direction? 12/6/2020 Theory of Computation: Chapter 7 15

Completeness Definition: A language L is hard for a complexity class C if L' ≤ L for every L' in C. If in addition L ∈ C, then L is complete for C. • Directed reachability is complete for NL (i. e. NL-complete). Reason: Nondeterministic acceptance is equivalent to finding a directed path in the configuration graph of a log-space machine. • If directed reachability were be to in L, then L = NL. Similarly, undirected reachability is complete for L (using SL = L). Reason: Acceptance is equivalent finding an undirected path in the symmetric configuration graph of the log-space machine. 12/6/2020 Theory of Computation: Chapter 7 16

NP-completeness Examples of NP-complete problems include: • Hamiltonian circuit (cf. Euler circuit from discrete math) If any NP-complete problem was in P, then P = NP. 12/6/2020 Theory of Computation: Chapter 7 17