Parallel Algorithms Parallel Models u Hypercube u Butterfly
Parallel Algorithms
Parallel Models u Hypercube u Butterfly u Fully Connected u Other Networks u Shared Memory v. s. Distributed Memory u SIMD v. s. MIMD
The PRAM Model u Parallel Random Access Machine u All processors act in lock-step u Number of processors is not limited u All processors have local memory u One global memory accessible to all processors u Processors must read and write global memory
A Pram Algorithm u Every Processor knows its own index (usually indicated by variable i) u Vector Sum: Read M[i] Into x; Read M[i+n] Into y; x : = x + y; Write x into M[i];
Binary Fan-In Read M[i] into Largest; Write M[i] into M[i+n]; Delta : = 1; For k : = 1 to élg nù Read M[i+Delta] into x; Largest : = Maximum(x, Largest); Write Largest into M[i]; Delta : = Delta * 2; End For
Parallel Addition Read M[i] into Total; Write 0 into M[i+n]; Delta : = 1; For k : = 1 to élg nù Read M[i+Delta] into x; Total : = x + Total; Write Total into M[i]; Delta : = Delta * 2; End For
Pointer Jumping Read M[i] Into Total; For k : = 1 to élg nù Read Next[i] into Ptr If Ptr ¹ 0 Then Read M[Ptr] Into x; Total : = Total + x; Write Total into M[i]; Read Next[Ptr] Into New. Ptr Write New. Ptr into Next[i] End If End For
Initialization of Next[i] If i = n Then Write 0 Into Next[i]; Else Write i+1 Into Next[i]; End If
Calculate Node Depth I If there is a Left Child 1 To “ 1” of Left Child -1 0 From “-1” of Left Child
Calculate Node Depth 2 If there is no left child 1 -1 0
Calculate Node Depth 3 If there is a Right Child 1 -1 0 To “ 1” of Right Child From “-1” of Right Child
Calculate Node Depth 4 If there is no right child 1 -1 0
Concurrent Reads & Writes u EREW - Exclusive Read, Exclusive Write u CREW - Common Read, Exclusive Write u CRCW - Common Read, Common Write – All common writes must write the same thing – Highest Priority Processor wins contest u CREW is more powerful than EREW u CRCW is more powerful than CREW
Finding Max u Square Array of Processors Indexed by i, j Write True into R[i]; Read M[i] into x; Read M[j] into y; If x < y Then Write False Into R[i]; Else If y < x Then Write False Into R[j]; End If
CRCW V. S. CREW u CRCW Max runs in constant time u CREW Max runs in lg n time u CRCW cannot be any better than lg p faster than EREW
EREW V. S. CREW u Finding Roots by Shortcutting Pointers u CREW Runs in lg lg n Time u EREW Runs in lg n Time
Optimal Parallel Algorithms u NC -- The class of algorithms that run in Q(logmn) time using Q(nk) processors u General Boolean Functions Cannot be Computed any Faster than Q(lg n) u Q(lg n) is optimal for computing the sum of n integers
- Slides: 17