Transmit and Receive Contexts Sean Hefty Local Queue

Transmit and Receive Contexts Sean Hefty

Local Queue Management • Traditional view • Fixed number of entries • Each entry a fixed size • Memory optimized • Fixed allocation • Entries variable size www. openfabrics. org 2

Local Queue Data • Application request • Specified operation • 0 or more IO vectors (SGEs) • Provider limits • Size of transmit queue • Maximum number of IOVs www. openfabrics. org 3

Local Queue Data Size to record operation Size to record IO vector Max IOVs Alignment restriction www. openfabrics. org Size in bytes 4

Local Queue Management • Queue attributes – Total size, in bytes – IOV limit – max number of IOVs for a single operation – Operation alignment, in bytes • Calculate padding with each operation • Domain attributes – Operation size, in bytes • Amount of space consumed to track each request – IOV size, in bytes • Amount of space consumed for each IOV www. openfabrics. org 5

Local Queue Management • Generic calculation done by app: needed = ((op_size + iov_size * nsge) + op_alignment - 1) & ~(op_alignment - 1) • For queues with fixed sized entries: – needed = op_alignment • For queues with variable sized entries – needed = op_size + iov_size * nsge www. openfabrics. org 6

Injected (inline) Data • Queue attribute – Maximum injected data size • Queue space calculation needed = op_size + inject_size – (Subject to alignment) www. openfabrics. org 7