PARALLEL PROCESSING Parallel processing Parallel Processing is a

PARALLEL PROCESSING

Parallel processing Parallel Processing is a technique that are used to provide simultaneous data-processing tasks for the purpose of increasing the computational speed of a computer system.

Parallel processing can be implemented by means of two ways § Pipelining Processing § Vector Processing

Pipelining is a technique of decomposing a sequential process into suboperations, with each subprocess being executed in a special dedicated segment that operates concurrently with all other segment

Example To perform the combined multiply and add operations with a stream of number. Ai * Bi + Ci for i=1, …, 5 Each sub operation is to be implemented in a segment within a pipeline. Each segment has one or two registers and combinational circuit.

Pipeline Process Ai Bi R 1 Ci R 2 MULTIPLIER R 3 R 4 ADDER R 5

R 1 Ai , R 2 R 3 Ci R 1 * R 2, R 4 R 5 product R 3 + R 4 Bi Ci Input Ai and Bi Multiply and input Add Ci to

Pipeline Process Ai A 2 A 3 A 1 Bi B 2 Ci B 1 R 1 C 1 R 2 MULTIPLIER C 1 A 1*B 1 R 3 R 4 ADDER A 1*B 1 +C 1 R 5

Clock Pulse number Segment 1 R 2 Segment 2 R 3 Segment 3 R 4 R 5 1 A 1 B 1 2 A 2 B 2 A 1*B 1 C 1 3 A 3 B 3 A 2*B 2 C 2 A 1 * B 1 + C 1 4 A 4 B 4 A 3*B 3 C 3 A 2 * B 2 + C 2 5 A 5 B 5 A 4*B 4 C 4 A 3 * B 3 + C 3 A 5*B 5 C 5 A 4 * B 4 + C 4 6 7 A 5 * B 5 + C 5

VECTOR PROCESSING �Many scientific problems require arithmetic operations on large arrays of numbers. These numbers are usually formulated as vectors and matrices of floating-point numbers. �A vector is an ordered set of a one-dimensional array of data items. �A vector V of length n is represented as a row vector by V =[V 1 V 2 V 3…. . Vn]

§ Consider the following Fortran Do loop Do 20 I=1, 100 20 c(I)=B(I)+A(I) § A computer capable of vector processing eliminates the overhead associated with the time it takes to fetch and execute the instructions in the program loop. § It allows operation to be specified with a single vector instruction of the form C(1: 100)=A(1: 100)+B(1: 100)

The vector instruction includes � The initial address of the operand � The length of the vectors � The operation to be performed Instruction format for vector processor: Operation code Base Address source 1 Base Address source 2 Base Address destination Vector Length

MATRIX MULTIPLICATION �Matrix multiplication is one of the most computational intensive operations performed in computer with vector processors. �The multiplication of two n*n matrices consist of n² inner products or n³ multiply –add operations. • An n*m matrix of number has n rows and m columns

Consider the multiplication of two 3 * 3 matrix a 11 a 21 a 31 a 12 a 22 a 32 a 13 a 23 a 33 b 11 b 21 b 31 b 12 b 22 b 32 b 13 b 23 c 11 c 21 c 31 c 12 c 22 c 32 c 13 c 23 c 33 The product matrix C is a 3× 3 matrix whose element are related to elements of A and B c ij=∑aik×bkj i=1, j=1 c 11=a 11 b 11+a 12 b 21+a 13 b 31

�In general, the inner product consists of the sum of K product terms of the form Inner product calculation on a pipeline vector processor: Source A Source B Multiplier pipeline Adder Pipeline

§ Multiplier pipeline and adder pipeline have four segments each. § All segment registers in the multiplier and adder are intialized to 0. § Ai and Bi pairs are brought in and multiplied at a rate of one pair per cycle. § After the first four cycles, the products begin to be added to the output of the adder. § At the end of the eighth cycle, the first four products A 1 B 1 through A 4 B 4 are in the four adder segments. § A 5 B 5 through A 8 B 8 are in the four multiplier segments.

§ Source A supplies the input A 1, A 2, …. § Source B supplies the input B 1, B 2, …. § All segment registers in the multiplier and adder are intialized to 0. § Ai and Bi pairs are brought in and multiplied at a rate of one pair per cycle. § After the first four cycles, the products begin to be added to the output of the adder. § At the end of the eighth cycle, the first four products A 1 B 1 through A 4 B 4 are in the four adder segments. § A 5 B 5 through A 8 B 8 are in the four multiplier segments.

�At the beginning of the ninth cycle, the output of the adder is A 1 B 1 and the output of the multiplier is A 5 B 5. �Thus the ninth cycle starts the addition A 1 B 1 + A 5 B 5 in the adder pipeline. And so on. �The tenth cycle starts the addition A 2 B 2 + A 6 B 6 and so on.