PRAM Algorithms Sums Prefix Sums by Doubling List

PRAM Algorithms Sums Prefix Sums by Doubling List Ranking 1

Definition: Prefix Sums o Given a set of n values a 1, a 2, …, an and an associative operation @, the Prefix Sums problem is to compute the n quantities a 1, a 1@a 2@a 3, . . . , a 1@. . . @an Example: {2, 7, 9, 4} {2, 9, 18, 22} 2

Doubling o A processing technique in which accesses or actions are governed by increasing powers or 2 o That is, processing proceeds by 1, 2, 4, 8, 16, etc. , doubling on each iteration 3

Prefix Sum by Doubling o Overview n n 1. 2. 3. 4. Each a(i) is is added to to a(i+1) a(i+2) a(i+4) a(i+8) n ETC…. . o At any time if an index exceeds n, the operation is supressed 4

Prefix Sums by Doubling Example 4 9 5 2 10 6 4 13 14 7 12 16 4 13 18 20 T 1 = O(n) * Operation supressed 12 8* 18* 20* 26* 23* 30* 36* 30 36 48 56 Tp = O(log n) 5

Prefix Sums by Doubling Example 0# 1 2 3 4 5 6 7* 0# 0, 1# 1, 2 2, 3 3, 4 4, 5 5, 6* 6, 7* 0# 0, 1, 2, 3, 2, 3, 4, 5, 6, 3# 4 5 6 7 0 0, 1, 2, 0, 1, 2, 3 3, 4, 5, 6 6, 7 T 1 = O(n) # contains final sum Tp = O(log n) * operation suppressed 6

Time Complexity o O(Log N) o At each step, the number of sums that are complete doubles n 1, 2, 4, 8, … n Thus, the number of steps is log n 7

Total Operations – Work/Cost o For N data values n Step 1 = N -1 additions {-20} o 1 PC is suppressed n Step 2 = N – 2 additions {-21} o 2 PCs are suppressed n Step 3 = N – 4 additions {-22} o 4 PCs are suppressed n Etc. 8

Consider case of N = 8 o Step 1 = N-1 = 7 o Step 2 = N-2 = 6 o Step 3 = N-4 = 4 n TOTAL = 17 o Generalize n (N-1) + (N-2) + (N-4) = 3 N -7 o Sequential = N-1 9

Generalize the Work Sum(i=0 to (log n) -1: N – 2 i = (N-1) + (N-2) + (N-4) +…+ (N-2 log n -1 ) =N*log N – (1+2+4+2 log n -1 ) = ? ? ? 10

Total Work for Prefix Sums (1+2+4+2 log n -1 ) = 2 log n - 1 Size = N * Log N – (2 log n – 1) = N * Log N - 2 log n + 1 T 1 = N 11

Prefix Sums - Comparisons Algorithm Work - Cost Depth Sequential N-1 Doubling N Log N - 2 log n + 1 Log N N-1 Upper/Lower N/2 Log N Odd/Even 2 Log N - 2 2 N – Log N - 2 12

Parallel Strategies - PRAM o Broadcast, Fan-out, Expand n O(log n) o Reduction, Combination, Fan-in n O(log n) o These are basically opposites 13

PRAM Algorithm Instructions - Spawn - For all o Step 1 of all PRAM algorithms is to activate P processors (Broadcast) n One processor starts activation n Activation takes O(Log P) time o Instruction: spawn (processor names) n E. G. spawn (P 0, P 1, . . , Pn) o For all <PC list> do {stmts} endfor n E. G. For all Pi, 0<=i<=n-1, do{…} 14

Sum of elements – EREW PRAM Given: n elements in A[0 … n-1] Var: A & j are global, i is local spawn (P 0, P 1, P 2, . . Pn/2 -1)// P = n/2 For all Pi, 0 <= i <= (n/2 -1) for j = 0 to log n - 1 do if (i mod 2 j = 0) & (2 i + 2 j < n) A[2 i] = A[2 i] + A[2 i + 2 j] 15

Trace for P 0 & P 1 P 0, i = 0 for all operations j = 0 to 2 (i=0 mod 20) & (2*0 + 20 < n) – yes A[0] = A[0] + A[1] P 1, i = 1 for all operations j = 0 to 2 (i=1 mod 20) & (2*1 + 20 < n) – yes A[2] = A[2] + A[3] 16

Prefix Sum - Doubling CREW PRAM Given: n elements in A[0 … n-1] Var: A & j are global, i is local spawn (P 1, P 2, . . Pn-1) // note # of PC For all Pi, 1 <= i <= n -1) for j = 0 to log n - 1 do if (i - 2 j >= 0) A[i] = A[i] + A[i - 2 j] 17

List Packing An Application of Prefix Sums o Consider an array of upper and lower case letters. Delete the lower case letters and compact the upper case to the low-order end of the array. 18

List Packing - Demonstration 1 2 3 4 5 6 7 8 a G H i n W b N G H W N 19

List Packing Implementation via Prefix Sums o Assign 1 to items to be packed and 0 to items to be deleted. o Perform the prefix sums on the 1/0 o If upper case, store in location = sum, otherwise do nothing o Last sum provides number of packed items 20

List Packing Implementation via Prefix Sums 1 2 3 4 5 6 7 8 a G H i n W b N 0 1 1 0 0 1 0 1 2 2 2 3 3 4 G H W N 21 S’pose: don’t know how 0/1 assigned. Can I still pack?

Linked List Ranking o Given a linked list, stored in an array, compute the distance of each element from the end (either end) of the list. o Problem is similar to prefix sums, using all 1’s to sum. o Called Pointer Jumping (not doubling) when using pointers. o Don’t destroy original list! 22

Linked List Ranking - demo A Q F C D T B P 8 7 6 5 4 3 2 1 23

rank next data List Ranking - Demonstration * 0 1 2 3 4 5 6 7 A D Q B C F P T 2 7 5 6 1 4 Nil 3 8 4 7 2 5 6 1 3 24

List Ranking Pointer Jumping - similar to Doubling 0 0 0 ► 0 0 ► 0 ► ► 0 nil ► 0 0 0 nil ► ►► nil ► 3 0 0 ► 2 0 0 nil ►► ► 1 0 ► init 0 Note: Must copy pointers to preserve the original linked list. 25

List Ranking Code // Copy next and set 1’s for all elements For all Pί , 0 ≤ ί ≤ n-1 , pardo N(ί) = next (ί) Rank (ί) = 1 // Doubling on the linked list For j = 1 to log n do if N(ί) ≠ N (N(ί)) then Rank (ί) = Rank(ί) + Rank (N(ί)) N(ί) = N (N(ί)) 26

What are we really doing? ? ? o If your location and your next do not point to the same location then n Add rank of next to yourself n Change your next to next(next) o Each step doubles distance of your next pointer o Progression of solution n n Initially – 1 rank correct Step 1 – 2 ranks correct Step 2 – 4 ranks correct Step 3 – 8 ranks correct O(log n) steps 27

Work Analysis o Number of Steps: Tp = O(Log N) o Number of Processors: N o Work = O(N log N) o T 1 = O(N) o Work Optimal? ? 28

Applications of List Ranking o o o Expression Tree Evaluation Parentheses Matching Tree Traversals Ear–Decomposition of Graphs Euler tour of trees - - - many others 29