NonBlocking Collective IO Routines Introduction IO is the
Non-Blocking Collective I/O Routines
Introduction • I/O is the main bottleneck in HPC applications. • To mitigate that performance, several approaches have been taken: – Collective I/O for aggregation – Non-blocking individual I/O – Higher level libraries (PHDF 5 , ADIOS, etc…)
Motivation • Routines for non-blocking individual I/O operations exist (MPI_File_i(read/write)(_at) • Non-blocking point-to-point (existing) and collective (to be added) communication operations have demonstrated benefits. • Split collective I/O operations have their restrictions and limitations. • What’s keeping us from adding non-blocking collective I/O operations? – Implementation
New Routines • MPI_File_iread_all (MPI_File file, void *buf, int count, MPI_Datatype, MPI_Status *status, MPI_Request *req); • MPI_File_iwrite_all (MPI_File file, void *buf, int count, MPI_Datatype, MPI_Status *status, MPI_Request *req); • MPI_File_iread_at_all (MPI_File file, MPI_Offset offset, void *buf, int count, MPI_Datatype, MPI_Status *status, MPI_Request *req); • MPI_File_iwrite_at_all (MPI_File file, MPI_Offset offset, void *buf, int count, MPI_Datatype, MPI_Status *status, MPI_Request *req);
Implementation • Major difference between collective communication and collective I/O operations: – each process is allowed to provide different volumes of data to a collective I/O operation, without having knowledge on the data volumes provided by other processes • Can’t post entire I/O operation to disk directly – Have to read/write data in cycles – Have to do Aggregation
Implementation • Need non-blocking collective communication – Fortunately will be available • Integrate with the progress engine – Test/Wait on the request like other non-blocking operations – Explicit or Implicit progress? • Different collective I/O algorithms
Conclusion • The need for non-blocking collective I/O is fairly high. • Implementation is the non-easy part. • Performance benefits can be substantial.
- Slides: 7