Problem Parallelize serial applications that use files Examples





























- Slides: 29

Problem ● Parallelize (serial) applications that use files. – ● ● Examples: compression tools, logging utilities, databases. In general – applications that use files depend on sequential output, – serial append is the usual file I/O operation. Goal: – perform file I/O operations in parallel, – keep the sequential, serial append of the file.

Results ● ● Cilk runtime-support for serial append with good scalability. Three serial append schemes and implementations for Cilk: 1. ported Cheerio, previous parallel file I/O API (M. Debergalis), 2. simple prototype (with concurrent Linked Lists), 3. extension, more efficient data structure (concurrent doublelinked Skip Lists). ● Parallel bz 2 using PLIO.

Single Processor Serial Append FILE (serial append) computation DAG 1 2 7 5 3 8 6 4 11 9 12 10

Single Processor Serial Append FILE (serial append) 1 2 3 computation DAG 1 2 7 5 3 8 6 4 11 9 12 10

Single Processor Serial Append FILE (serial append) 1 2 3 4 5 6 7 computation DAG 1 2 7 5 3 8 6 4 11 9 12 10

Single Processor Serial Append FILE (serial append) 1 2 3 4 5 6 7 8 9 10 11 computation DAG 1 2 7 5 3 8 6 4 11 9 12 10 12

Single Processor Serial Append FILE (serial append) 1 2 3 5 1 7 5 3 6 7 8 9 10 11 Why not in parallel? ! computation DAG 2 4 8 6 4 11 9 12 10 12

Fast Serial Append Paralle. L file I/O (PLIO) support for Serial Append in Cilk Alexandru Caracaş

Outline ● Example – ● Semantics – ● modification of Cilk runtime system Implementation – ● view of Cilk Programmer Algorithm – ● single processor & multiprocessor Previous work Performance – Comparison

Multiprocessor Serial Append FILE (serial append) computation DAG 1 2 7 5 3 8 6 4 11 9 12 10

Multiprocessor Serial Append FILE (serial append) 1 2 7 computation DAG 1 2 7 5 3 8 6 4 11 9 12 10

Multiprocessor Serial Append FILE (serial append) 1 2 3 5 7 8 9 computation DAG 1 2 7 5 3 8 6 4 11 9 12 10

Multiprocessor Serial Append FILE (serial append) 1 2 3 4 5 6 7 8 9 10 computation DAG 1 2 7 5 3 8 6 4 11 9 12 10

Multiprocessor Serial Append FILE (serial append) 1 2 3 4 5 6 7 8 9 10 11 computation DAG 1 2 7 5 3 8 6 4 11 9 12 10 12

File Operations ● open (FILE, mode) / close (FILE). ● write (FILE, DATA, size) – ● processor writes to its PION. read (FILE, BUFFER, size) – processor reads from PION. ● ● Note: a seek operation may be required seek (FILE, offset, whence) – processor searches for the right PION in the ordered data structure

Semantics ● View of Cilk programmer: – Write operations ● – preserve the sequential, serial append. Read and Seek operations ● can occur only after the file has been closed, ● or on a newly opened file.

Approach (for Cilk) ● ● Bookkeeping (to reconstruct serial append) – Divide execution of the computation, – Meta-Data (PIONs) about the execution of the computation. Observation – ● Theorem – ● In Cilk, steals need to be accounted for during execution. expected # of steals = O ( PT∞ ). Corollary (see algorithm) – expected # of PIONs = O ( PT∞ ).

PION (Parallel I/O Node) ● ● Definition: a PION represents all the write operations to a file performed by a processor in between 2 steals. A PION contains: – # data bytes written, – victim processor ID, – pointer to written data. PION FILE 1 π1 2 π3 3 4 5 π2 6 7 8 9 π4 10 11 12

Algorithm ● All PIONSs are kept in an ordered data structure. – ● very simple Example: Linked List. On each steal operation performed by processor Pi from processor Pj: – create a new PION πi, – attach πi immediately after πj, the PION of Pj in the order data structure. PIONs π1 πj πk

Algorithm ● All PIONSs are kept in an ordered data structure. – ● very simple Example: Linked List. On each steal operation performed by processor Pi from processor Pj: – create a new PION πi, – attach πi immediately after πj, the PION of Pj in the order data structure. πi PIONs π1 πj πk

Algorithm ● All PIONSs are kept in an ordered data structure. – ● very simple Example: Linked List. On each steal operation performed by processor Pi from processor Pj: – create a new PION πi, – attach πi immediately after πj, the PION of Pj in the order data structure. PIONs π1 πj πi πk

Implementation ● Modified the Cilk runtime system to support desired operations. – ● Initial implementation: – ● concurrent Linked List (easier algorithms). Final implementation: – ● implemented hooks on the steal operations. concurrent double-linked Skip List. Ported Cheerio to Cilk 5. 4.

Details of Implementation ● Each processor has a buffer for the data in its own PIONs – ● Data structure to maintain the order of PIONs: – ● implemented as a file. Linked List, Skip List. Meta-Data (order maintenance structure of PIONs) – kept in memory, – saved to a file when serial append file is closed.

Skip List ● Similar performance with search trees: – O ( log (SIZE) ). NIL NIL

Double-Linked Skip List ● ● Based on Skip Lists (logarithmic performance). Cilk runtime-support in advanced implementation of PLIO as rank order statistics. NIL NIL

PLIO Performance ● no I/O vs writing 100 MB with PLIO (w/ linked list), ● Tests were run on yggdrasil a 32 proc Origin machine. ● Parallelism=32, ● Legend: – black: no I/O, – red: PLIO.

Improvements & Conclusion ● Possible Improvements: – Optimization of algorithm: ● delete PIONs with no data, ● cache oblivious Skip List, – File system support, – Experiment with other order maintenance data structures: ● ● B-Trees. Conclusion: – Cilk runtime-support for parallel I/O ● allows serial applications dependent on sequential output to be parallelized.

References – Robert D. Blumofe and Charles E. Leiserson. Scheduling multithreaded computations by work stealing. In Proceedings of the 35 th Annual Symposium on Foundations of Computer Science, pages 356 -368, Santa Fe, New Mexico, November 1994. – Matthew S. De. Bergalis. A parallel file I/O API for Cilk. Master's thesis, Department of Electrical Engineering and Computer Science, Massachusetts Institute of Technology, June 2000. – William Pugh. Concurrent Maintenance of Skip Lists. Departments of Computer Science, University of Maryland, CS-TR-2222. 1, June, 1990.

References – Thomas H. Cormen, Charles E. Leiserson, Donald L. Rivest and Clifford Stein. Introduction to Algorithms (2 nd Edition). MIT Press. Cambridge, Massachusetts, 2001. – Supercomputing Technology Group MIT Laboratory for Computer Science. Cilk 5. 3. 2 Reference Manual, November 2001. Available at http: //supertech. lcs. mit. edu/cilk/manual 5. 3. 2. pdf. – bz 2 source code. Available at http: //sources. redhat. com/bzip 2.
16 Files 1 Files 3 Files 1 Files
Serial vs Parallel Connections Serial Connections Serial connections
Serial attached SCSI Serial attached SCSI Uvod Serial
Serial Network Configuration Serial Over Broadband Serial Interface
Serial EEPROM API Supported Serial EEPROM parts Serial
CREST Examples Basic Examples Function Examples Limitation Examples
CREST Examples Basic Examples Function Examples Limitation Examples
Files Streams Files Introduction Files are used for
Parallel Computing Explained How to Parallelize a Code
CS 267 Assignment 3 Parallelize Graph Algorithms for
Chapter 7 Serial Devices Serial devices Terminal Modem
Serial Communication and Networking Jun Hu Today Serial
Serial Crime Original From Mr Syswerda Introduction Serial
SERIAL PERIPHERAL INTERFACE SPI M Serial Peripheral Interface
What is a Serial Killer A serial killer
Universal Serial Bus USB Universal Serial Bus USB
Asynchronous Serial Communication Peter Szyszko Agenda Serial Basics
Serial Number Restoration Serial Number A unique number
Device Management Serial Port CPU Memory Serial Device
Serial v s Parallel Communication n Serial Communication
VI includeSoftware Serial h Software Serial bt Serial12
Serial Interfaces Bit serial bus n New generation
Chapter 4 UART Serial Port Programming 1 Serial
Introduction to Forensic Science Serial Killers Serial Murders
INTRODUCTION TO FORENSIC SCIENCE Serial Killers Serial Murders
Serial Killers Higgins OBrien Spring 2012 Serial Killers
Synchronous Serial Standard Asynchronous Serial Standard Universal SynchronousAsynchronous
Serial Killers WHAT Defines a Serial Killer The
Serial Killers Serial Killers Kill three or more