NFVbench project update April 2020 Alec Hothan NFVbench

• NFVbench project update April 2020 Alec Hothan (NFVbench PTL, Cisco) Cedric Ollivier (NFVbench contributor, Orange) Francois Regis Menguy (NFVbench contributor, Orange)

Agenda • NFVbench project introduction and update • NFVbench use in Orange • Xtesting and CNTT integration • Q&A April 21, 2020 LFN April 2020 Virtual Event - NFVbench update 2

NFVbench Refresher • Open source data plane benchmarking designed for NFVi – Any “full stack” NFVi platform (Open. Stack, CN…) – Advanced integration with Open. Stack based platforms • Major focus – Infrastructure benchmarking – Production deployment use cases – Simplify benchmarking (use, standardized results, repeatability) – Very low cost and reasonable performance • Always on - can be built-in on every deployment sit April 21, 2020 LFN April 2020 Virtual Event - NFVbench update 3

NFVbench Features April 21, 2020 LFN April 2020 Virtual Event - NFVbench update 4

NFVbench Implementation • NFVbench is a CNF since day 1 (2015) – Single container design – Sweet spot: 2 x 25 Gbps, typical IMIX line rate or up to ~20 Mpps April 21, 2020 LFN April 2020 Virtual Event - NFVbench update 5

NFVbench Deployments • Adoption tracker (and counting) – About a dozen operators – Over a hundred lab and production cloud deployments worldwide • Overall feedbacks from users – Like: • features, convenience, low cost, ease of use, performance • on-site data plane infrastructure baselining – Challenges: • initial setup and deployment integration April 21, 2020 LFN April 2020 Virtual Event - NFVbench update 6

Deployment use cases • Lab: – full stack evaluation and comparison – regression testing – deployment capacity planning (density benchmarking) • Production: – Permanent on-site data plane validation and baselining – On-demand benchmarking/integrated diagnostic April 21, 2020 LFN April 2020 Virtual Event - NFVbench update 7

NFVbench in Orange Iaa. S • Overall testing strategy • NFVbench use cases April 21, 2020 LFN April 2020 Virtual Event - NFVbench update 8

OPNFV/CNTT in Orange Iaa. S • • Product validation using Functest CI chain based on Xtesting Testing strategy compliant with CNTT Shaker and VMTP already run for performance testing – Basic results – Another tool (or generator) needed for intensive benchmarking and further cases April 21, 2020 LFN April 2020 Virtual Event - NFVbench update 9

NFVbench in Orange laa. S • To know the network performance of numerous platforms • To validate platform after deployment or reconfiguration • To test Iaa. S network performance as easily and automatically as possible • To setup and fine tune tools and methods on a wide range of hardware and software v Tests performed manually until Xtesting development v Xtesting permit a smooth integration with other tools April 21, 2020 LFN April 2020 Virtual Event - NFVbench update 10

NFVbench use cases for Orange laa. S • Network performance methodology • Network performance benchmarking of a Iaa. S – SUT characterization • Maximum throughput (based on RFC 2544) • Latency at maximum throughput (Work In Progress) – Non-regression tests (CI chain) – NFVbench as a VM for east-west test cases – Visualization chain April 21, 2020 LFN April 2020 Virtual Event - NFVbench update 11

Xtesting integration • Main goal • CI chain April 21, 2020 LFN April 2020 Virtual Event - NFVbench update 12

NFVbench and Xtesting main objectives • Automate test campaign – Test execution – Dump test case result • CI integration – Generate Jenkins jobs – Deploy CI/CD chain – Result visualization – PASS/FAIL criteria • CNTT compliance (CI / RC) April 21, 2020 LFN April 2020 Virtual Event - NFVbench update 13

Xtesting test launcher • Easy to implement and integrate • Add Xtesting python library on NFVbench container – Add simple test case (rate 10 kpps in PVP mode) – Dedicated documentation for Xtesting usage • Test case file can be overridden easily April 21, 2020 LFN April 2020 Virtual Event - NFVbench update 14

Xtesting CI chain • Few commands to deploy Xtesting CI chain – CI/CD toolchains – List test cases to execute – Create job for NFVbench test cases • Ansible files (site. yml and host vars env) can be overridden easily April 21, 2020 LFN April 2020 Virtual Event - NFVbench update 15

NFVbench & Xtesting CI example April 21, 2020 LFN April 2020 Virtual Event - NFVbench update 16

Challenges • NFVbench included in CNTT – A pre-requisite for an adoption in Orange Iaa. S product testing (as product validation leverage on CNTT test cases) • Create a list of relevant network performance test cases for CNTT – Complementary to or in replacement of Shaker/VMTP – Study TRex or Prox implementation in Shaker ? April 21, 2020 LFN April 2020 Virtual Event - NFVbench update 17

NFVbench and Xtesting improvements • Define a process to run same test case multiple times – Aggregate results – Determine standard deviation – Take best result based on a couple of same tests • Define a process to run a variety of latency test cases based on maximum throughput – Retrieve maximum throughput based on previous NFVbench run – Run NFVbench test at a fixed rate (range of % of NDR result) • Propose a visualization chain for NFVbench results April 21, 2020 LFN April 2020 Virtual Event - NFVbench update 18

Further feedback? Please direct any questions or comments to opnfv-tech-discuss@lists. opnfv. org April 21, 2020 LFN April 2020 Virtual Event - NFVbench update 19