Use case of RDMA in Symantec storage software

Use case of RDMA in Symantec storage software stack Om Prakash Agarwal Symantec

Agenda • • • About Symantec Storage Software Stack Need for the faster interconnects Write operation without and with RDMA Implementation details: Write operation with RDMA Perf comparison (without and with RDMA) Symantec Proprietary: Use case of RDMA in Symantec storage stack 2

Applications CVM/CFS Hardware Kernel User About Symantec Storage Software Stack LLT HBA NIC HBA Shared storage Symantec Proprietary: Use case of RDMA in Symantec storage stack 3

Applications CVM/CFS Hardware Kernel User Need for the faster interconnects LLT NIC Local storage Shared Nothing Architecture Symantec Proprietary: Use case of RDMA in Symantec storage stack 4

Kernel Write operation without and with RDMA Hardware LLT CVM/CFS Buffer Bufptr Buffer LLT RNIC RDMA over Infini. Band or Ethernet NIC Symantec Proprietary: Use case of RDMA in Symantec storage stack RNIC 5

Write operation with RDMA receiver CVM/CFS sender 4 k 4 k 8 k 1 k 1 k Message 4 k 4 k 4 k 2 k Message TX map Destination buffer info LLT 4 k 4 k Symantec Proprietary: Use case of RDMA in Symantec storage stack RX map allocation Actual Allocation Buffer. Actual advertisement: 4 k buffers 4 k and 4 ksend 4 k 4 k 4 k Allocate 4 kdata 4 k 4 k sender 4 k Meta to 4 k node. 6

Perf comparison (without and with RDMA) 1) Throughput 2) Latency Setup configuration Ø Two node Linux (RHEL 6. 3) cluster Ø Connect. X 3 VPI NIC card from Mellanox ü IB -56 Gbps ü Ro. CE-40 Gbps ü Eth-40 Gbps Ø 3. 3 GHz Processor Symantec Proprietary: Use case of RDMA in Symantec storage stack 7

LLT perf comparison : Throughput Higher is better 193% 125% Symantec Proprietary: Use case of RDMA in Symantec storage stack 8

LLT perf comparison : Latency 62% Lower is better Symantec Proprietary: Use case of RDMA in Symantec storage stack 9

Thank You #OFADev. Workshop