ITCS 45145 Parallel Computing Spring 2014 TuesdayThursday 5

Home Page For course notes, assignments, announcements, etc. : http: //cs. uncc. edu/~abw/ITCS 4145

Outline These slides constitute the course outline and include the course syllabus and course

Parallel Computing The use of multiple processors or computers to solve problems at a

Computer platforms for parallel computing 1. Multiple interconnected computers - Cluster Computing, group of

UNCC cluster used for course ssh to user gateway Login cci-gridgw. uncc. edu coitgrid

Course Prerequisites UNC-C Catalog prerequisites: ITCS 2214 (Data Structures) and ITCS 3181/3182 (Computer Architecture).

Course Text Parallel Programming: Techniques and Application Using Networked Workstations and Parallel Computers, 2

Course Contents Parallel computers: architectural types, shared memory, message passing, interconnection networks, potential for

Course Contents (Continued) Message passing: MPI message passing APIs, send, receive, collective operations. Running

Course Contents (Continued) Programming with shared memory programming: Specifying parallelism, sharing data, critical sections,

Course Contents (Continued) Algorithms and applications: Selection from: • Sorting algorithms • Searching algorithms

Assessment • Class quizzes (2) • Assignments (5) • Final exam 25% 50% 25%

Course Grade Undergraduate Grade Scale A 90 - 100 B 80 - 89. 99

Code of Student Academic Integrity All submitted assignments must be your own work. Copied

Due dates for assignments Will be posted on home page with assignment. Assignments can

Attendance is expected. Attendance will be recorded. If you miss classes, it can have

Mid-Term Unsatisfactory Grades • UNC-Charlotte requires midterm unsatisfactory grades to be submitted by the

Expectations Read all materials provided. Read slides before class. Take additional notes during class

Instructor details UNC-Charlotte Barry Wilkinson Department of Computer Science University of North Carolina, Charlotte

Course Teaching Assistant Mrunal Nargunde mnargund@uncc. edu Contact TA on matters such as user

Slides: 22

Download presentation

Home Page For course notes, assignments, announcements, etc. : http: //cs. uncc. edu/~abw/ITCS 4145 S 14/ Password protected: Username: pablo Password: rp 19 zb 39 outline. 2

Outline These slides constitute the course outline and include the course syllabus and course policies. outline. 3

Parallel Computing The use of multiple processors or computers to solve problems at a greater computational speed than using a single computer/processor. Basic idea is simple – using N computers/ processors collectively on a problem should lead to a faster solution. Can tackle problems that could not be solved in a reasonable time otherwise. Also solve problems with higher precision and/or with more memory requirements. outline. 4

Computer platforms for parallel computing 1. Multiple interconnected computers - Cluster Computing, group of interconnected computers typically using Ethernet switch and physically all in one room. 2. A single computer system with multiple internal processors or cores - Usually shares a common main memory 3. Computer system with attached graphic processing unit (GPU) - GPUs have large number of execution cores, now used for high performance computing as well as original graphics application Combination of above most likely. outline. 5

UNCC cluster used for course ssh to user gateway Login cci-gridgw. uncc. edu coitgrid 01 coitgrid 02 coit-grid 01 -4: Each two Xeon 3. 4 Ghz processors, 8 GB main memory coitgrid 03 coitgrid 04 coitgrid 06 ccigrid 07 ccigrid 08 C 2050 GPU K 20 GPU (448 core) (2496 core) coit-grid 06: GPU server, with NVIDIA 2050 GPU switch cci-grid 05: Four quadcore 2. 93 Ghz Xeon processors 64 GB main memory 1. 2 TB disk (RAID 5) ccigrid 05 ccigrid 09 cci-grid 08: GPU server, E 5 -1650 3. 2 GHz cci-grid 07: 6 -core Xeon GPU server, X 5560 2. 8 GHz processor with quad-core Xeon NVIDIA K 20 GPU, 32 GB processor with main memory NVIDIA 2050 GPU, 12 GB main memory cci-grid 09: Dual AMD Opteron 6376 2. 3 GHz 16 -core each, with 16 GB main memory RAID 1 disks, redundant power supplies, VM, web server, user gateway login, SSH, NAT, DHCP, DNS, and LDAP, etc. All user’s home directories on cci -grid 05 (NFS) Has all three types of platform. Note: Some names have been changed from coit- to cci-

Course Prerequisites UNC-C Catalog prerequisites: ITCS 2214 (Data Structures) and ITCS 3181/3182 (Computer Architecture). ITCS 3181/2 not strictly necessary so long as you know what a computer consists of (processors, memory and I/O) and how it operates. What is needed -- basic skills in C and Java. This course is a programming course. outline. 7

Course Text Parallel Programming: Techniques and Application Using Networked Workstations and Parallel Computers, 2 nd edition, by B. Wilkinson and M. Allen, Prentice Hall Inc. , 2005, ISBN 0 -13 -140563 -2. On-line notes are provided that are derived from the course text – with some updated materials from 2005 outline. 8

Course Contents Parallel computers: architectural types, shared memory, message passing, interconnection networks, potential for increased speed. Parallel patterns for structured parallel programming, motives, advantages, introduction to various patterns, tools. • Seeds pattern programming framework to create distributed computations. outline. 9

Course Contents (Continued) Message passing: MPI message passing APIs, send, receive, collective operations. Running MPI programs on a cluster, implement patterns with MPI. Using compiler directives for pattern programming, Paraguin compiler. More computation patterns and their application, divide and conquer, pipeline, iterative synchronous all-to-all, stencil. outline. 10

Course Contents (Continued) Programming with shared memory programming: Specifying parallelism, sharing data, critical sections, threads, Open. MP. Running threaded/ Open. MP programs on multi-core system, hybrid MPI/Open. MP programs. CPU-GPU systems: data parallel pattern, GPU architecture, programming in CUDA, issues for achieving high performance. outline. 11

Course Contents (Continued) Algorithms and applications: Selection from: • Sorting algorithms • Searching algorithms • Numerical algorithms • Image processing algorithms outline. 12

Assessment • Class quizzes (2) • Assignments (5) • Final exam 25% 50% 25% The assessment and percentages may be modified. Read small print There will be additional work in the assignments for graduate students, which will be extra credit for undergraduates. outline. 13

Course Grade Undergraduate Grade Scale A 90 - 100 B 80 - 89. 99 C 70 - 79. 99 D 60 - 69. 99 F 0 - 59. 99 Grade Scale A 90 - 100 B 80 - 89. 99 C 70 - 79. 99 U 0 - 69. 99 Some curving likely, depending upon course averages. A student with an average score should expect an average grade. outline. 14

Code of Student Academic Integrity All submitted assignments must be your own work. Copied work or work done by more than one person (unless specifically instructed) will not be accepted - at the very minimum, zero credit - and may be subject to disciplinary action. The Code of Student Academic Integrity applies to both assignments and quizzes/tests. outline. 15

Due dates for assignments Will be posted on home page with assignment. Assignments can be turned in late but with a penalty. Late Penalty < 24 hours 10% > 24 hours and < 48 hours 20% > 48 hours and < 72 hours 30% > 72 hours 100% outline. 16

Attendance is expected. Attendance will be recorded. If you miss classes, it can have a deleterious effect on your grade: After 4 missed classes without good reason: 2 marks off overall accumulated marks (out of 100) for each additional class missed. If you must miss class, see the instructors beforehand to avoid losing marks. Attendance will be checked especially for border-line between final course grades. Good attendance will encourage moving up grades. outline. 17

Mid-Term Unsatisfactory Grades • UNC-Charlotte requires midterm unsatisfactory grades to be submitted by the end of the 8 th week of class. • Mid term grades – Undergraduates: Satisfactory or D or F – Graduates: Satisfactory or C or U will be based upon all work graded by week 7, which probably will be two assignments and one class test. outline. 18

Expectations Read all materials provided. Read slides before class. Take additional notes during class Ask questions during class and participate in class discussion. outline. 19

Instructor details UNC-Charlotte Barry Wilkinson Department of Computer Science University of North Carolina, Charlotte Home page: http: //www. cs. uncc. edu/~abw Email: abw@uncc. edu Office Hours Tuesday/Thursday: 2: 30 pm to 4: 30 pm outline. 20

Course Teaching Assistant Mrunal Nargunde mnargund@uncc. edu Contact TA on matters such as user accounts, UNC-C cluster not functioning, and software issues. If a system issue also copy email to me at abw@uncc. edu Contact me for matters such as posted course materials and assignment write-up, or general assistance.

Questions outline. 22