Communication and Complexity Arun Velagapalli Computer Science Engineering

Communication and Complexity Arun Velagapalli Computer Science Engineering Mississippi State University

Outline Background Motivation Model to Study Communication Complexity Theorems – 9. 10 , 9. 11 , 9. 12

Background We have seen in the previous presentation hierarchy of randomized complexity classes Models of Parallel computation

Motivation Models of parallel Computation Ignore the costs of synchronization and interprocess communication Include them directly in their Time and Space complexity measures.

For Certain problems Running time and Space for the processes is irrelevant. Example: Voting problems Cost? Real Cost derives from the number and size of messages exchanged by the processes. Some measure of Communication Complexity is needed

The notion of communication complexity (CC) was introduced by Yao in 1979. [1] Model to study Communication Complexity

Model Consider a function f(x, y) Two Machines TM 1 and TM 2 TM 1 is given x as input TM 2 is given y as input Assumption: length(x) = length(y) TM 1 , TM 2 communicate by exchanging alternating messages.

TM 1 computes the next message to send based upon its share of input + record of all the messages that it has received so far , vice versa. How many bits must be exchanged in order to allow TM 1 or TM 2 to output f(x, y) ? The upper bound on complexity of any function under this model is |x|.

TM 1 can send all of x to TM 2 and let TM 2 do the computing. Example: if x, y are integers , sum of x and y is determined by TM 2 when TM 1 sends x. Here model requires single message from TM 1 to TM 2.

Fix x and y communication complexity of f(x, y ) is denoted by c(f(x, y )) minimum number of bits that must be exchanged in order for one of the machines to compute f, allowing messages of arbitrary length. Let ‘n’ be the length of x and y. we define the communication complexity of f for inputs of size 2 n, c(f 2 n ) as min of c(f(x, y) over all partitions of input into two strings x and y , of equal length. Restrict to functions f that represent ‘ Decision Problems’

Theorem 9. 10 Let t(n) be a function with 1< t(n) ≤ n for all n and denote b Co. MM(t(n)) the set of decision problems f obeying c(f 2 n) ≤ t(n) for all n. Then Co. MM(t(n)) is a proper superset of Co. MM(t(n)-1). proof: nonconstructive and relies on counting arguments to establish that a randomly chosen language has probability of requiring ‘n’ bits of communication≠ 0 => there are languages in Co. MM(n) – Co. MM(n-1). argument is extended from n to t(n). Define the nondeterministic communication complexity

if there exists an algorithm that recognizes “yes” instances of size 2 n using no more than t(n) bits of communication Cost to solve a decision problem nondeterministically – t(n) Nondeterminism gives rise to exponential gaps. We need to show that gap is no larger than exponential.

Theorem 9. 11 Nco. MM(t(n)) C Co. MM(2 t(n)). All that TM 2 needs to know in order to solve the problem is TM 1 output to any possible sequence of communications. TM 1 can produce output within the stated bounds. TM 1 enumerates all possible sequences of messages of total length <= t(n), binary alphabet {0, 1}. As alphabet is binary 2 t(n) sequences are possible. ith bit encodes its answer to ith sequence of messages. => TM 1 communicates with a single message of length 2 t(n) , which is needed by TM 2.

Theorem 9. 12 There is a problem in NCo. MM(α log n) that requires Ω(n) bits of communication in any deterministic solution. Lemma 9. 1 Let the function f(x, y), where x and y are binary strings of the same length, be the logical inner product of the two strings. That is, writing x = x 1 x 2 ………. xn and y = y 1 y 2 ……. yn we have f(x, y) = Vn i=1(xi ∧ yi). then the communication complexity of decision problem “Is f(x, y) false ? ” is exactly n. proof of Theorem is followed by proof of lemma.

References: [1] Communication complexity http: //en. wikipedia. org/wiki/Communication_complexity Theory of Computation Bernard Moret, University of New Mexico.

9. 5 Interactive Proofs and Probabilistic Proof Checking Avinash R. Yerramreddy

Definition 9. 11: An Interactive Proof System is composed of a checker, which runs in probabilistic polynomial time, and a prover, which can use unbounded resources. We write such a system (P, C). A problem ∏ admits an interactive proof if there exists a checker C a constant ε>0 such that there exists a prover P* such that the interactive proof system (P*, C) accepts every “yes” instances of ∏ with probability at least ½ + ε; and for any prover P, the interactive proof system (P, C) rejects every “no” instances of ∏ with probability at least ½ + ε;

Definition 9. 12: hkj The class IP(f) consists of all problems that admit an interactive proof where, for an instance x, the parties exchange at most f(|x|) messages. In particular, IP is the class of decision problems that admit an interactive proof involving at most a polynomial number of messages, IP=IP(n. O(1)).

How is an interactive proof system developed? • Solved by using one sided proof system of Graph Nonisomorphism Given two graphs G 1 and G 2, are they nonisomorphic? • The checker chooses at random the index i Є{1, 2} and sends to the • prover a random permutation H of Gi The prover tests H against G 1 and G 2 and sends back to the checker the index of the graph to which H is isomorphic • The checker compares its generalised index i with the index sent by prover; if they agree, the checker accepts the instance, otherwise it rejects it.

-> G 1 and G 2 are nonisomorphic -> when both are isomorphic the prover finds that H is isomorphic to both, not knowing the random bit used by the checker, the prover must answer at random, with a probability of ½ returning the value used by the checker.

Theorem 9. 13: IP space equals PSPACE. -> so far we have used relativization. -> however there exists an oracle A under which the relativized version of this theorem fails, that is, under this we have IPA ≠ PSPACEA

So to prove this theorem we use the Arithmetization of Boolean formulae which means the encoding of Boolean formulae into the low-degree polynomials over the integers. We derive the polynomial function pf from f by setting up for each Boolean variable ui a corresponding variable yi by applying the following three rules: 1). The Boolean literal ui corresponds to the polynomial pui = 1 -yi and the Boolean literal ūi to the polynomial pūi =yi. 2). The Boolean clause c={ûi 1, ûi 2, ûi 3} corresponds to the polynomial pc=1 -p ûi 1 p ûi 2 p ûi 3 3). The Boolean formula over n variables f=c 1^c 2^c 3^…. . cm corresponds to the polynomial pf(y 1, y 2…. yn)=pc 1, pc 2, …pcm.

Lemma 9. 2: Let p(y 1, y 2, …yn) be a polynomial with integer coefficient and denote by test(p) the problem of deciding, for given values a 1, a 2, … an, and b, whether p(a 1, a 2, …. an) equals b. If test(p) belongs to IP, then So does each of test(andyi(p)), test(oryi(p)), and test(reduceyi(p)). We can encode an instance of the PSPACE –complete problem say Q 1 x 1 Q 2 x 2……Qnxnf(x 1, x 2, …. xn), as follows: 1. Produce the polynomial pf corresponding to the formula f. 2. For each i=1, …. n in turn apply reducex. 3. For each i=1, …. n in turn, if the quantifier Qn+1 -i is universal, then apply andxn+1 -i else apply orxn+1 -i. No polynomial produced in this process ever has a degree greater than that of f – after the second step, in fact, no polynomial ever has a degree greater than two. Using this low-degree polynomial, we can proceed to devise an interactive proof protocol.

How much power does a benevolent IP prover need? The restrictions on the prover can be in both ways: -> the benevolent prover may lose some of its ability to convince the checker of the validity of the true statement, but the malevolent provers also become less able to fool the checker into accepting the false statement.

9. 5. 2 Zero-Knowledge Proofs: In cryptographic applications there are occasions when one party wants to convince the other party of the correctness of some statement without divulging any real information. For example, you might want to convince someone that a graph is three colorable without, however revealing the three colors. -> for this consider the following experiment. -> both the prover and the checker have a description of the graph with colors by values from {1, 2, 3}. -> the prover sends to the checker n locked “boxes”, one for each vertex of the graph. Each box contains the color assigned to its corresponding vertex. -> the checker cannot look into the box without a key; thus the checker selects at random one edge of the graph say {u, v} and asks the prover for the keys to the boxes u and v. -> the prover sends the keys to the checker; the checker opens the boxes.

If the colors are distinct and both in the set {1, 2, 3}, the checker accepts the claim of the prover; otherwise it rejects the claim. -> if the graph is not three-colorable, any filling of the n boxes has at least one illegal color, which the checker discovers with probability |E|-1. With enough rounds the probability of error can be reduced to any desired constant. After k rounds the probability of error is bounded by (1 - |E|-1)k. So with this experiment we can define a zero-knowledge proof system; Definition 9. 13: A prover stategy, P, is perfect zero-knowledge for problem ∏ if, for every probabilistic polynomial-time checking strategy, C, there exists a probabilistic polynomial-time algorithm A such that, for every instance x of ∏, the output of the prover-checker interaction, (P, C)(x), equals that of the algorithm, A(x).

Thus we can infer from the definition that the checker can compute with the help of the prover that it could not compute on its own. The zeroknowledge proof is also called the computational zero-knowledge. One-way function: It is a function that is P-easy to compute but at least NP-hard to invert. Such functions are conjectured to exist, but their existence has not been proved. Theorem 9. 14: Assuming the existence of one-way functions, any problem in PSPACE has a computational zero-knowledge interactive proof. Graph Three –Colorability can be implemented with one-way functions. This also holds for PSPACE and thus also for IP, thus has the ability to produce zero-knowledge proofs for any problem.

9. 5. 3 Probabilistically Checkable Proofs: Most mathematical proofs, are static entries: they are intended as a single communication to the reader by the writer. This limited interaction is to the advantage of the checker: the prover gets fewer opportunities to prove to the checker. This leads to our definition of NP: the prover runs in non- deterministic time and the checker in deterministic polynomial time. Definition 9. 15: A problem ∏ admits a probabilistically checkable proof if there exists a probabilistic algorithm C that runs in polynomial time and can query specific bits of a proof string ∏ and a constant ε>0 such that for every “yes” instances of x of ∏ there exists a proof string ∏ such that C accepts x with probability at least ½ + ε; and for every “no” instances of x of ∏ and any proof string ∏, C rejects x with probability at least ½ + ε;

Definition 9. 15: The class PCP(r, q) consists of all problems that admits a probabilistically checkable proof where, for instance x, the checker C uses at most r(|x|) random bits and queries at most q(|x|) bits from the proof string. Theorem 9. 15: NP equlas PCP(O(log n), O(1)). Every decision problem in NP has a proof that can be checked probabilistically by looking only at a constant number of bits of the proof string, using a logarithmic number of random bits. In many ways, this theorem known as PCP theorem is a major step in the study of the complexity theory after Cook’s and karp’s results. -> this ties together randomized approaches, proof systems, and nondeterminism.

References: 1). Theory of Computation: BERNARD M. MORET

Thank You!

9. 6 Complexity and Constructive Mathematics Nischal Vadari 32

Existential definitions and constructive proofs Series-Parallel Graphs Minor of a Graph 3 -D Knotless Embedding Immersion Ordering Decision Problems and Constructive Complexity class. 33

A problem belongs to a complexity class if : there exists an algorithm to solve it it exhibits the desired characteristics This existence is proved constructively, as we place the problem in a class by designing the necessary algorithm. Consequence of difference found between existential theories and constructive proofs This gives rise to a situation where it is found that certain problems are in P, yet the polynomial-time algorithm to solve them cannot be found. Kuratowski’s Theorem “Every non-planar graph contains a homeomorphic copy of either the complete graph on 5 vertices K 5 or the complete bipartite graph on 6 vertices K 3, 3. 34

Definition 9. 16 An undirected graph G with a distinguished source vertex, s and a distinguished (and separate) sink vertex, t, is a series-parallel graph, written (G, s, t), if it is one of • • the complete graph on the two vertices s and t (i. e. , a single edge); the series composition of two series-parallel graphs (G 1, s 1, t 1) and (G 2, s 2, t 2) where the composition is obtained by taking the union of the two graphs, merging t 1 with s 2, and designating s 1 to be the source of the result and t 2 its sink; or 35

the parallel composition of two series-parallel graphs (G 1, s 1, t 1) and (G 2, s 2, t 2) where the composition is obtained by taking the union of the two graphs, merging s 1 with s 2 and t 1 with t 2, and designating the merged s 1/s 2 vertex to be the source of the result and merged t 1/t 2 its sink A simple argument shows that an undirected graph with two distinct vertices designated as source and sink is series-parallel iff it does not contain a homeomorphic copy of the graph s t 36

Definition 9. 17 A graph H is a minor of a graph G if H can be obtained from G by a sequence of (edge or vertex) deletions and edge contractions. The relation “is a minor of” is seen to be reflexive and transitive. Hence, if H is a minor of G , it can be written as H ≤minor G. Any minor of a planar graph is itself planar which makes planar graphs closed under this relation. So, Kuratowski’s theorem can be restated as “ A graph G is nonplanar iff one of the following holds: K 5 ≤minor G (or) K 3, 3 ≤minor G. ” {K 5, K 3, 3} is the finite set of specific graphs. In the case of planar graphs, it is referred to as obstruction set 37

Robertson and Seymour’s Results: Theorem 9. 16 Ø Families of graphs closed under minor ordering have finite obstruction sets. Given graphs G and H, each with O(n) vertices, deciding whether H is a minor of G can be done in O(n 3) time. The first result was conjectured by Wagner as a generalization of Kuratowski’s theorem and was proved by Robertson and Seymour. It is regarded as one of the greatest and most complex achievements in mathematics. The power of the two results is summarized in corollary 9. 1 38

Corollary 9. 1 Membership in any family of graphs closed under minor ordering can be decided in O(n 3) time. Proof: Family of graphs is closed under minor ordering. Suppose the obstruction set is { O 1, O 2 , …, Ok } , it is finite, then a graph G belongs to the family iff it does not contain as a minor any of the Oi s. Oi ≤minor G can be tested in cubic time, after a constant k (depending on the family) tests, we can decide membership. 39

The Robertson- Seymour theorem states that only a finite obstruction set exists for each family of graphs closed under minor ordering. It does not tell how to find this set. It is an existential tool for establishing membership in P. Prominent example of the power of this tool is Three-Dimensional Knotless Embedding. A knot is defined as two interlinked cycles. Embedding without knots can be possible by removing vertices or edges of the graph or contracting an edge. A family of graphs that can be embedded without knots in 3 -D space is closed under minor ordering. 40

Given a fixed embedding, it could not be checked for the presence of knots in polynomial time. After the Robertson-Seymour theorem, it could be proved that the problem existed in P, but the fact that it was even recursive could not be proved. The Robertson-Seymour theorem is inherently non-constructive. 41

Theorem 9. 17 There is no algorithm that, given a family of graphs closed under minor ordering, would output the obstruction set for the family. Proof: Let {Φi } be an acceptable programming system, {Gi} be an enumeration of all graphs such that if Gi is minor of Gj, then Gi is enumerated before Gj. The auxiliary partial function is defined as: Θ(x) = μi [step( x, x, i)≠ 0] We define the function f(x, t) = 0 1 step(x, x, t) =0 or (Θ(x)<1 and G Θ(x) !<minor. Gt) otherwise 42

For each x, Sx = {Gt | f( x, t)=1} is closed under minor ordering. If x belongs to K, then {GΘ(x) } is an obstruction set for Sx. If x does not belong to K, then the obstruction set for Sx is empty. This is a reduction from K to the problem of deciding if the obstruction set for the family of graphs closed under minor ordering is empty. This proves theorem. 43

Nash-Williams conjecture: “Families closed under immersion ordering have finite obstruction sets. ” Immersion ordering: This is similar to minor ordering, but differs in the aspect that minor ordering uses edge contraction whereas immersion ordering uses edge lifting. Corollary 9. 2 Membership in a family of graphs closed under immersion ordering can be decided in polynomial time. We can identify P with the class of problems that have finite obstruction sets with polynomial ordering tests. 44

Theorem 9. 18 A problem Π is in P iff there exists a partial ordering ≤Π On the instances of Π such that: (i) given two instances I 1 and I 2, testing I 1 ≤Π I 2 can be done in polynomial time; and (ii) the set of “yes” instances of Π is closed under ≤Π and has a finite obstruction set. This theorem defines partial computations of the polynomial-time machine on the “yes” instances that is used to define the partial order. The “if” part of theorem is trivial and the “only if” part is harder, which can be proved not too hard by defining the partial order and matching obstruction set. 45

Definition 9. 18 • • • A decision problem is a pair Π = (I, M), where I is a set of instances and M is a checker. The checker M defines a relation between yes instances and the evidence for them. This definition sidesteps existence problems and Takes us close to proof systems. Constructive problems comprise of two steps: Generating suitable evidence Checking the answer with the help of the evidence. Complexity of constructive problems is the complexity of their search and checking components. 46

Definition 9. 19 A constructive complexity class is a pair of classical complexity classes (C 1 , C 2) , where C 1 denotes the resource bounds within which the evidence generator must run and C 2 the bounds for the checker. Resource bounds are defined with respect to the classical statement of the problem i. e. , with respect to the size of the domain elements. Definition 9. 20 A problem (I, M) belongs to a class (C 1 , C 2) iff the relation defined by M on I is both C 1 – searchable and C 2 – checkable. 47

References Moret B. M. , Theory of Computation, pp 396 -403

Thank You! 49