1 Introduction to Parallel Processing Classical Use of

1 - Introduction to Parallel Processing

Classical Use of Parallelism • Simulating scientific problems – Weather forecasting – Car crash simulation, • Computer aided design – Drug design , … • Graphical processing – Film editing or special effect software Programs must be partitioned to several independent parts, and each part is assigned to a separate processor 2

Parallelism in Today’s Hardware • Emerging multicore processors • powerful Graphical Processing Units (GPUs) • Require a new era of parallel programming to: – utilize available hardware – Increase application performance • Software restructuring is a critical need for today’s applications. 3

Basic Concepts • The application is partitioned into multiple tasks. • The size of tasks (in terms of the number of instructions) is called granularity. • The tasks are assigned to processes or threads. • The assignment of tasks is called scheduling. • The assignment of processes or threads onto the physical processors or cores, is called mapping. • Parallel programs need synchronization of threads. 4

Memory Organization • Shared memory : thread – Information exchange between threads is done by shared variables written by one thread and read by another. – Programming environment: Open. MP • Distributed memory: process – Information exchange is done by sending data from one processor to another processor via an interconnection network – Programming environment: Message Passing Interface (MPI) 5

parallel execution time • consists of the time for the computation on processors or cores and the time for data exchange or synchronization. • the time elapsed between the start of the application on the first processor and the end of the application on all processors. 6

Course Content 1 - Hardware of parallel computer systems o. Multicore systems (MPSo. Cs) Architecture o. Low power design concepts o. Interconnection networks 2 - Parallel Programming Models o. Coordination o. Synchronization o. Communication 3 - Performance metrics of parallel programs. 15