Software Overhead in Messaging Layers Messaging Layer Bridge

Software Overhead in Messaging Layers

Messaging Layer • Bridge between the hardware functionality and the user communication requirement • Network hardware feature – Arbitrary delivery order – Finite buffering – Limited fault handling • User communication requirement – In-order delivery – End-to-end flow control – Reliable transmission

Messaging Layer (cont. )

Software overhead • Main communication cost = hardware cost + software cost • Software cost dominates hardware cost • 50 -70% of the software cost are the direct gap between the network features and user requirements

Software overhead in messaging layers • Analyze the costs of communication functionality and network features • What overhead might be reduced if the underlining network provide higher level of service

CM-5 active message layer (CMAM) • CM-5 – Parallel machine up to a few thousand nodes connected with an incomplete fat tree topology • Active message – Communication mechanism intended to expose full hardware flexibility and performance of modern networks – Each message contain an address of a user-level handler which is executed on message arrival with the message body as an argument – The handler extract data from the network and integrate it into the ongoing computation • CMAM vs Send/Recv – User direct access to network, no OS involve, the data go directly to the user space computation

Software overhead cost analysis • Consider implementation of 3 protocols – Single-packet delivery – Finite sequence, multi-packet delivery – Indefinite sequence, multi-packet delivery • Use instruction counts for measurement

Single-packet delivery Description Call/Return NI Setup Write to NI Read from NI Check NI status Control Flow Source 3 2 7 3 20 Destination 10 3 12 2 27

Finite sequence, multi-packet delivery

Indefinite sequence, multi-packet delivery

Message size = 16 words

Message size = 1024 words

Messaging layer with high-level network feature • Given that CMAM is considered to be very efficient, there are 2 choices to reduce software overhead – Lower user requirement – Raise level of service provided by the network • Compressionless Routing (CR) – Order-preserving transmission – Deadlock freedom independent of packet acceptance guarantees – Fault-tolerant transmission at packet level

Single-packet delivery • Has the same cost as the previous CMAM case • However, it is guaranteed to be fault-free, no deadlock or buffer overflow

Finite sequence, multi-packet delivery

Indefinite sequence, multi-packet delivery

Message size = 16 words

Message size = 1024 words

Discussion • Larger packet sizes – Reduce overhead, especially for indefinite-sequence protocol • Improved network interfaces and DMA hardware – Network interface: • only make basic cost faster, but not protocol cost in messaging layer • Make it more important for messaging layer to be effective – DMA: • only reduce cost in moving large amounts of data

Discussion (Cont. ) • Implication of network design – Improving routing performance may increase software cost, e. g. out-of-order delivery • Reducing communication features – Fault-tolerance, in-order-delivery – Put burden on parallel software programmer • Use instruction counts for measurement instead of latency – Latency is hard to measure in a portable fashion

Conclusion • Software overhead is much larger than routing time and 50 -70% are messaging layer overhead • Software overhead in messaging layer is the cost of the gap between the network feature and the user requirement • This cost can be reduced to zero if the underlining network provides functionality that the user requires.