Malte Vesper Dirk Koch Vipin Kizheppatt and Suhaib

Malte Vesper, Dirk Koch, Vipin Kizheppatt, and Suhaib A. Fahmy Jet. Stream: An Open-Source high-performance PCI Express 3 streaming library for FPGA-to-Host and FPGA-to-FPGA communication https: //maltevesper. github. io/Jet. Stream

PCIe Hierarchy 2

PCIe Hierarchy 3

PCIe Hierarchy 4

PCIe Hierarchy 5

OSI Layers • PCIe implements 3 of 7 OSI layers: – Transaction layer – Data link layer – Physical layer • Xilinx/Altera hard IP handles physical and data link layer In the following Xilinx PCIe Gen 3 IP 6

Whats left to the user? • • • Chunk transfers (alignment/max packet sizes) Manage number of outstanding requests Provide mechanisms for channels Create headers Host driver 7

Related work • Commercial solutions – Expensive – Large and monolithic • Open source – No PCIe 3 x 8 (except ff. Link) – Often not multi FPGA details on comparison slide 8

Related Work PCIe. Gen EPEE 2 Dy. RACT 2 Zero. Copy DMA Asynchronous transfers RIFFA 2. 2 3 x 4 ff. Link 3 x 8 Xillybus 3 x 4 Northwest Logic 3 x 8 Jet. Stream 3 x 8 Multiboard support Board-to. Board Read MB/s Write MB/s 3280 3198 1474 1452 3040 6330 7060 3500 ? ? 5573 5981 6446 7086 9

Jet. Stream System Achitecture 10

Straddling Header Payload Header 2 Payload Header Payload 11

DMA Modes 12

DMA Modes 13

DMA Modes 14

Resourceusage 15

Transfersize vs Bandwidth 16

Speed 17

Stream benchmark 40 35 30 [GB/s] 25 20 15 10 5 0 Copy Add Copy Direct 0 1 2 3 Add Direct 4 18

API-Example 19

Features • • • 7 GB/s throughput Direct Board-to-Board transfers Transfer queues User interrupts/registerfiles Different DMA modes 20

Getting Jet. Stream https: //maltevesper. github. io/Jet. Stream/ 21

Future Work • Block. IP • Parameters to disable alignment, chunking, …. • PCIe 4? straddling on all interfaces • Altera port? 22

Thank you for your attention QUESTIONS https: //maltevesper. github. io/Jet. Stream