Straight Line Code xi a xi xj xk

Straight Line Code xi = a xi = xj + xk xi = xj xk No loop No index as variables No if statement Arithmetic circuit DAG wij : weight of edges from vi to vj

Example x 1 4 x 1 = 4 x 2 = 3 x 3 = x 1 + x 2 x 4 = x 2 + x 3 x 5 = x 2 * x 4 x 6 = x 3 * x 4 x 7 = x 4 * x 6 Straight line code x 2 3 x 3 + + x 4 x 6 * * x 5 x 7 * Arithmetic Circuit

Multi Variate Polynomial v 1 : v 2 : v 3 : v 4 : v 5 : v 6 : v 7 : x 1 x 2 x 1 + 2 x 2 x 1 x 2 + 2 x 22 x 12 + 3 x 2 + 2 x 22 x 1 + … degree 1 1 2. . . Parallel Algorithm O(log. N log d. N) d: degree of circuit (degree of polynomial) N: Size or number of nodes in the circuit d may be large (exponential of N) useful only if d is polynomial of N

Degree of poly O(N) Application Gaussian Elimination straight line program with O( N 3) nodes O ( log 2 N) time Construction of heap Sorting - Straight line program with O(N 2) nodes O ( log 2 N) time [bubble sort, insertion sort]

leaf node: its value is evaluated completely Eval - Add: all inputs are leves Eval - Mult: all inputs are leaves one input is leaf Skip - Add: not all inputs are leaves vi wik vi vj vj wjk => + vk Action of Eval -Add vk vk = vi wik + vj wjk Eval - Mult vi vj 1 vk * vi 1 => vj vk Case (a) Both vi and vj are leaves vj vi 1 vk vi wjk = value(vi) 1 * vj => vk + Case (b) vi is a leaf, and vj are not leaf (exactly one input is a leaf node)

Skip Add w 2 w 1 w 2 w 3 w 4 v + w 4 + w 3*w 5 + w 1 w 3 z x w 1*w 5 w 2*w 5 + Inputs of the first addition node x are promoted to become inputs to the second addition node y. Inputs to x are maintained if x is an input to another node z. z w 4*w 5

Skip-Add Multiple edges from vi to vj may be inserted. New weight from vi to vj if vi, vj are addition node i original wij must be removed(vi, vj both + nodes) vj : addition node vi : not k vj is not an addition nde j if vk, vj are addition node otherwise Skip Add: W <- W - W+ + WW+ O(n 3) PEs, O(logn) time

x 1 = 6 x 2 = 2 x 3 = x 1 + x 2 x 4 = x 2 + x 3 x 5 = x 3 • x 4 x 6 = x 3 + x 5 x 7 = x 4 • x 5 Straight-line Code node v 1 v 2 v 3 v 4 v 5 v 6 v 7 v 1 6 2 v 3 + + v 4 v 5 • • v 7 + v 6 Arithmetic Circuit polynomial x 1 x 2 x 1 + 2 x 2 x 12+3 x 1 x 2+2 x 22+x 1+x 2 x 13+5 x 12 x 2+8 x 1 x 22+4 x 23 degree 1 1 2 2 3

v 1 6 v 3 8 v 1 6 2 v 2 1 v 3 8 1 1 2 v 2 + v 4 1 v 5 • 1 1 1 2 v 3 8 • v 7 after Eval-Mult v 1 6 v 3 8 v 5 80 1 8 • v 7 after Eval-Add 1 1 + v 4 8 v 5 1 + 1 8 + v 6 • v 7 v 1 6 2 v 3 8 10 v 4 v 5 80 1 + v 6 8 after Skip-Add 2 v 2 10 v 4 1 1 2 v 2 8 1 1 + v 6 after Eval-Add v 1 6 8 + v 3 + v 4 1 • v 7 + v 6 v 5 v 1 6 + v 6 v 5 80 8 800 after Eval-Mult 10 v 4 v 7 + v 6 8 800 after Skip-Add v 7

a Complexity Define height leaf node = 2 * node : h(x) = h(a) + h(b) + node: max (a+1, m) a = max (h 1, . . . , hi) v 1, . . . , vi : + nodes inputs to x m = max (hi+1 , . . . , hk) vi+1, . . , vk : * nodes inputs to x b * x v 1 + vi x vi+1 + * * + degree and height: almost same except + node degree: max(d 1, d 2), height: max (h 1, h 2)+1 Theorem: O(logd. N) phases are sufficient. Proof is based on the following two claims. Claim 1: The of an N-node circuit of degree d (N+1)d Claim 2: After log 3/2 h phases, every node has been reduced to a leaf node. vk

Proof on the complexity of straight line program Lemma: The height of an arithmetic circuit of degree d is at most (Z+2)d, where Z is the number of addition nodes in the circuit that have addition nodes as inputs. Proof: By induction on the number of nodes N in the circuit. Basis (N=1) trivially true. Inductive hypothesis: Assume that the claim is true for N. Inductive step: Let us prove for N+1. Let v be a node with degree d and max height h with inputs v 1, . . . , vk, having degree d 1, . . . dk, and heights h 1, . . . , hk. Three cases. case 1: v is “*” node. Then d=d 1 +. . . + dk, and h=h 1 +. . . + hk, hi (Z+2)di by inductive hypothesis. Thus, h (Z+2)d. case 2: v is “+” node, and max of h 1 , . . . , hk is not attained by “+” node. hi = max {hi} max{(Z+2)di} (Z+2)d. case 3: v is “+” node, and max of h 1 , . . . , hk is attained by “+” node. hi = 1 + max {hi} 1 + max{(Z+1)di} 1 + (Z+1)d (Z+2)d. ( hi (Z+1)di follows from the fact that v itself is a type of Z, and must not be part of di subcircuit, and inductive hypothesis. ) Thus, the lemma holds. Corollary: The height of N node circuit of degree d is at most (N+1)d. Proof: Since Z N-1, it follows.

Proof on the complexity (cont’) Lemma: Given any circuit and node v, let h : the height of v at the beginning of a phase, h’ : the height of v at the end of the phase. If h 3, then h’ (2/3)h. Moreover, if v is *” node that gets converted into an addition node during the phase, then h’ (2/3)h - 1/3. Proof: Induction on the size of circuit. Suppose that v is *node at the beginning of phase. Case 1. v becomes a leaf during the phase. h’=2, * node h 4. Thus, h’ h/2. Case 2. v remains *. Nor v 1, v 2 are leaf before Eval-Mult. Thus, h 1, h 2 > 3 h 1’ (2/3)h 1, h 2’ (2/3)h 2, by inductive hypo. Thus, h’ h 1’ + h 2’ (2/3) (h 1 +h 2 ) (2/3)h v 2 v 1 * v Case 3. v becomes “+” node during the phase. One node was leaf, the other is not. Let v 1 be the input to v before the phase. After the phase, either v 1 stays as the only input, or v 1 participate skip-add during the phase. If v 1 remains input of v: h = h 1+2, h 1’ (2/3)h 1, thus, h’ = h 1’+1 (2/3)h - 1. If v 1 participate skip-add during the phase, then any new input u to v after the phase should be the input to v 1 before the phase. Note that the height of u is less than the height of v 1, so the same argument as the above holds.