Parallel Programming Fall 2012 TuesdayThursday 11 00 am

Parallel Programming Fall 2012 Tuesday/Thursday 11: 00 am - 12: 15 pm Instructors Dr. Barry Wilkinson Dr. Clayton Ferner University of North Carolina Charlotte University of North Carolina Wilmington © 2012 B. Wilkinson/Clayton Ferner. Modification date: Aug 23, 2012 outline. 1

Outline These slides constitute the course outline and include the course syllabus and course policies. outline. 2

Parallel Computing The use of multiple processors or computers to solve problems at a greater computational speed than using a single computer/processor. Basic idea is simple – using n computers/ processors collectively on a problem should lead to a faster solution. Can tackle problems that could not be solved in a reasonable time otherwise. Also solve problems with higher precision and/or with more memory requirements. outline. 3

Computer platforms for parallel computing 1. Multiple interconnected computers - Cluster Computing, group of interconnected computers typically using Ethernet switch and physically all in one room. 2. A single computer system with multiple internal processors or cores - Usually shares a common main memory 3. Computer system with attached graphic processing unit (GPU) - GPUs have large number of execution cores, now used for high performance computing as well as original graphics application Combination of above most likely. outline. 4

UNCC cluster coit-grid 01. uncc. edu – coit-grid 7. uncc. edu Has all three types of platform. Login from on-campus or off-campus coitgrid 01 coitgrid 02 coitgrid 03 coit-grid 01 -4: Each two Xeon processors (3. 4 Ghz) 8 GB main memory switch coit-grid 05: Dell R 900, four quad-core Xeon processors (2. 93 Ghz) 64 GB main memory 1. 2 TB disk coitgrid 05 coitgrid 04 Login from within the campus only coitgrid 07 coitgrid 06 NVIDIA Tesla GPU (448 core Fermi) coit-grid 07: coit-grid 06: GPU server, X 5560 GPU server 2. 8 GHz quad-core Xeon with NVIDIA processor with NVIDIA 2050 GPU, 12 GB main memory All user’s home directories on coit -grid 05 (NFS)

Course Prerequisites UNC-C Catalog prerequisites: ITCS 2214 (Data Structures) and ITCS 3181/3182 (Computer Architecture). ITCS 3181/2 not strictly necessary so long as you know what a computer consists of (processors, memory and I/O) and how it operates. What is needed -- basic skills in C and Java. outline. 6

Course Text Parallel Programming: Techniques and Application Using Networked Workstations and Parallel Computers, 2 nd edition, by B. Wilkinson and M. Allen, Prentice Hall Inc. , 2005, ISBN 0 -13 -140563 -2. On-line notes are provided that are derived from the course text – with some updated materials from 2005 outline. 7

Home Pages For course notes, assignments, announcements, etc. : http: //coit-grid 01. uncc. edu/abw/ITCS 4145 F 12/ Password protected: Username: pablo Password: rp 19 zb 39 Please check before each class. Home page also found from link at http: //www. cs. uncc. edu/~abw outline. 8

Course Contents (See course slides for detailed description) Parallel computers: architectural types, shared memory, message passing, interconnection networks, potential for increased speed. (New) Parallel patterns for structured parallel programming, various patterns, use of Seeds framework to create distributed computations. Message passing: MPI message passing APIs, send, receive, collective operations. Running MPI programs on a cluster. outline. 9

Course Contents (Continued) (New) Introduction to the Paraguin compiler, parallel region, forall, broadcast, gather (New) More computation patterns and their application, divide and conquer, pipeline, iterative synchronous all-to-all, stencil. outline. 10

Course Contents (Continued) Programming with shared memory programming: Specifying parallelism, sharing data, critical sections, threads, Open. MP. Running threaded/ Open. MP programs on multi-core system, hybrid MPI/Open. MP programs. CPU-GPU systems: data parallel pattern, GPU architecture, programming in CUDA, issues for achieving high performance. outline. 11

Course Contents (Continued) Algorithms and applications: Selection from: • Sorting algorithms • Searching algorithms • Numerical algorithms • Image processing algorithms outline. 12

Course Grade Letter grades will be assigned according to the following scales and may be curved according to the performance of the entire class. UNC-Wilmington Undergraduate UNC-Wilmington Graduate UNC-Charlotte Undergraduate UNC-Charlotte Graduate Grade A AB+ B BC+ C CD+ D DF Grade A AB+ B BC+ C CU Grade A B C D F Grade A B C U Scale 93 - 100 90 - 92. 99 87 - 89. 99 83 - 86. 99 80 - 82. 99 77 - 79. 99 73 - 76. 99 70 - 72. 99 67 - 69. 99 63 - 66. 99 60 - 62. 99 0 - 59. 99 Scale 93 - 100 90 - 92. 99 87 - 89. 99 83 - 86. 99 80 - 82. 99 77 - 79. 99 73 - 76. 99 70 - 72. 99 0 - 69. 99 Scale 90 - 100 80 - 89. 99 70 - 79. 99 60 - 69. 99 0 - 59. 99 Scale 90 - 100 80 - 89. 99 70 - 79. 99 0 - 69. 99 outline. 13

Computation of Grades • Class quizzes (2) • Assignments (6) • Final exam 25% 50% 25% The assessment and percentages may be modified. Read small print outline. 14

Graduate students UNC-C ITCS 5145 and UNC-W CSC 537: There will be additional work in the assignments for graduate students, which will be extra credit for undergraduates. UNC-W CSC 537: Graduate students will give presentation in class on a research topic. outline. 15

Student Learning Outcomes (SLOs) The learning outcomes for the UNC-W undergraduate parallel computing course are: • • Students develop solutions and implement the solutions for scientific problems using multiple processors in a cluster. Students acquire knowledge in writing programs which require processes to synchronize and exchange data using Message Passing Interface (API). Students acquire knowledge in writing programs which require processes to spawn multiple treads that share memory on multiple core processors. Students learn to profile parallel programs by placing event probes and to log them for analysis and tuning of the program performance. The Estimate Curriculum Category Content (Semester hours) is: Area Core Advanced Area Core Algorithms 1. 0 Software design 0. 5 Data structures 0. 25 Computer organization and architecture Advanced Concepts of programming languages outline. 16

Code of Student Academic Integrity All submitted assignments must be your own work. Copied work or work done by more than one person (unless specifically instructed) will not be accepted - at the very minimum, zero credit - and may be subject to disciplinary action. The Code of Student Academic Integrity applies to both assignments and quizzes/tests. outline. 17

Due dates for assignments Will be posted on home page with assignment. Assignments can be turned in late but with a penalty. Late Penalty < 24 hours 10% > 24 hours and < 48 hours 20% > 48 hours and < 72 hours 30% > 72 hours 100% outline. 18

Attendance is expected. Attendance will be recorded. If you miss classes, it can have a deleterious effect on your grade: After 4 missed classes without good reason: 2 marks off overall accumulated marks (out of 100) for each additional class missed. If you must miss class, see the instructors beforehand to avoid losing marks. outline. 19

Mid-Term Unsatisfactory Grades • UNC-C requires midterm unsatisfactory grades to be submitted by the end of the 8 th week of class. • Mid term grades – Undergraduates: Satisfactory or D or F – Graduates: Satisfactory or C or U will be based upon all work graded by week 7, which probably will be two assignments and one class test. outline. 20

Instructors details UNC-Charlotte Barry Wilkinson Department of Computer Science University of North Carolina, Charlotte Home page: http: //www. cs. uncc. edu/~abw Email: abw@uncc. edu Office Hours Tuesday/Thursday: 9: 15 am to 10: 45 am outline. 21

Instructors details UNC-Wilmington Clayton Ferner Department of Computer Science University of North Carolina Wilmington Home page: http: //people. uncw. edu/cferner Email: cferner@uncw. edu Office Hours Tuesday/Thursday 2 pm to 3: 30 pm outline. 22

Expectations Read all materials provided. Read slides before class. Take additional notes during class Ask questions during class and participate in class discussion. outline. 23

NCREN facility (North Carolina Research and Education Network) Site interaction Each site will be using their NCREN facility, which provides microphones for individuals to speak. Please feel fee to speak at suitable times but remember that NCREN needs to be able to recognize the site and speaker, so first announce your institution and name. outline. 24

NCREN facility continued • Since most microphones are turned on all the time, any sound will be heard across all sites unless a site turns their microphones off, so try to avoid unnecessary sounds. • Fans from a laptop can be especially disturbing, so move laptop away from a microphone. • Turn cell phones off. outline. 25

NCREN Video Class Recording The lectures are available on-line after 24 hours. Details to be provided. outline. 26