Basics Concepts of Parallel Programming What is parallel

Basics Concepts of Parallel Programming

What is parallel computing • It’s a form of computation where many calculations can be carried out simultaneously. • In parallel computing we can divide large programs into smaller one and perform computation simultaneously.

Serial computation 11/28/2020 3

Parallel Computing 11/28/2020 4

Do we need Parallel Computing? • Save time and/or money • Solve larger problems • Provide concurrency 11/28/2020 5

Uses for Parallel Computing • Physics - applied, nuclear, particle, condensed matter, high pressure, fusion, photonics • Bioscience, Biotechnology, Genetics • Chemistry, Molecular Sciences • Geology, Seismology • Mechanical Engineering - from prosthetics to spacecraft • Electrical Engineering, Circuit Design, Microelectronics • Computer Science, Mathematics 11/28/2020 6

Threads “ A Thread is single stream of control in the flow of a program” Example 1: Consider the following code which computes product of 2 dense matrice of size n x n. for(row=0; row<n; row++) for(column=0; column<n; column++) c[row][column]=dot_product(get_row(a, row), get_col(b, col));

Ø The for loop in this code fragment has n 2 Iteration, each which can be executed independently. Such independent sequence of instructions is referred as thread. Ø In above example there are n 2 threads one for each iteration of for loop. Since each thread can be executed independently of others, they can be scheduled concurrently on multi processors. Example 2: here we are using function called create_thread to provide mechanism for specifying c function as thread for(row=0; row<n; row++) for(column=0; column<n; column++) c[row][column]=create_thread(dot_product(get_r ow(a, row), get_col(b, col)));

Why Threads • Thread – Basic Unit of execution – Light weight process – Process can have multiple thread • Why thread – To improve the CPU Utilization Note: all threads share same shared memory. 11/28/2020 9

Open. MP (Open Multi-Processing) • An Application Program Interface (API) that may be used to explicitly direct multi-threaded, shared memory parallelism. • Open. MP is an API that can be used with FORTRAN, C and C++ for programming shared address space machines. • Open. MP directives provides support for concurrency, synchronization and data handling. • The Open. MP directives in C and C++ are based on the #pragma compiler directives. 11/28/2020 10

• Open. MP programs execute serially until they encounter the parallel directive. This directive is responsible of creating group of threads. • The main thread that encounters the parallel directive becomes the Master of this group of thread and the thread is assigned with ID 0 within this group

Open. MP Programming Model • Fork - Join Model: – Open. MP uses the fork-join model of parallel execution: – All Open. MP programs begin as a single process: the master thread 11/28/2020 12

Open. MP Directives • Format: #pragma omp directive-name [clause, . . . ] newline Required for all Open. MP C/C++ directives. A valid Open. MP directive. Must appear after the pragma and before any clauses. Optional. Clauses can be in any order, and repeated as necessary unless otherwise restricted. Required. Precedes the structured block which is enclosed by this directive. • Ex: – #pragma omp parallel default(shared) private(beta, pi) 11/28/2020 13

Open. MP Directives • General Rules – Case sensitive – Directives follow conventions of the C/C++ standards for compiler directives – Only one directive-name may be specified per directive – Each directive applies to at most one succeeding statement, which must be a structured block. – Long directive lines can be "continued" on succeeding lines by escaping the newline character with a backslash ("") at the end of a directive line. 11/28/2020 14

PARALLEL Region Construct • A parallel region is a block of code that will be executed by multiple threads • When a thread reaches a PARALLEL directive – It creates a team of threads – Becomes the master of the team. – Master thread ID is 0. • Starting from the beginning of the parallel region, – the code is duplicated – all threads will execute that code • If any thread terminates within a parallel region, all threads in the team will terminate 11/28/2020 15

How Many Threads? • The number of threads in a parallel region is determined by the following factors – Setting of the NUM_THREADS clause – Use of the omp_set_num_threads() library function – Setting of the OMP_NUM_THREADS environment variable – Implementation default - usually the number of CPUs on a node 11/28/2020 16

#pragma omp parallel • It’s a parallel directive that creates groupp of threads. • Using this we can compute operations parallelly.

tid=omp_get_thread_num() • Used to give unique ID for each thread created.

Parallel program to add 2 single dimensional arrays

program • • #include<stdio. h> #include<omp. h> int main() { int a[10], b[10], c[10], i, n; printf("nenter the number of elements"); scanf("%d", &n); • • • printf("n. Enter the element of 1 st array"); for(i=0; i<n; i++) scanf("%d", &a[i]); • • • printf("enter the elements of 2 nd array"); for(i=0; i<n; i++) scanf("%d", &b[i]);

• printf("nthe contents of array An"); • for(i=0; i<n; i++) • { • printf("na[%d]=%d", i, a[i]); • } • • • printf("nthe contents of array Bn"); for(i=0; i<n; i++) { printf("nb[%d]=%d", i, b[i]); }

• • #pragma omp parallel for shared(c) for(i=0; i<n; i++) { #pragma omp critical (c_lock) c[i]=a[i]+b[i]; printf("nc[%d]=%d, thread id=%dn", i, c[i], omp_get_thread_num()); } • • printf("nthe new array is===="); for(i=0; i<n; i++) { printf("%d " , c[i]); } getch(); }

• Prime numbers

• • • • #include<stdio. h> #include<omp. h> void main() { int a[1000], n, i, k; //n for number of prime numbers printf("enter the value of n: t"); scanf("%d", &n); for(i=2; i<=n; i++) a[i]=1; //intitialize each location to 1 assuming all are not prime #pragma omp parallel { omp_get_thread_num();

• for(k=2; k<=sqrt(n); k++) //generate all prime numbers from k to n • { • if(a[k]!=0) • { • #pragma omp for • // eliminate multiples of k • for(i=k*k; i<=n; i+=k) • a[i]=0; • } • }

• printf("prime numbers are n"); • for(i=2; i<=n; i++) • { • if(a[i]) printf("%dn", i); • } •