ONAP Maturity Testing Metrics Performance Tools JMeter Locust

ONAP Maturity Testing Metrics: Performance • Tools: - JMeter / Locust - Simulators: controller, VNFM, PNFs • Metrics - Concurrent users and threads - Throughput & Latency - Service instantiation • Speed • Capability of concurrent provisioning of multiple services • Related projects: VID, SO, SDNC, Multi. Cloud - Control loop time: • Related projects: DMaa. P, DCAE, Policy, A&AI, APPC/VFC, Multi. Cloud, etc. 1

Test of concurrency • Concurrency is tested for long-live API calls that take a few seconds or even longer to complete. The following test process applied to two cases: - Instantiation of an entire service. - API calls of a specific component. • Test process 1. 2. 3. 4. Select a target API and set up the related simulators and dependent modules. Set a concurrency number N Generate N API requests and send them to the target API. Obtain all responses and validate the correctness. 2

Test of throughput and latency • Test process 1. Select a target API, e. g. , A&AI read or write. 2. Set the range of API call rate, e. g. , from 1000 call/s to 10000 calls/s, and repeat the following a. At the selected call rate, generate API requests and send them to the target API. b. Obtain all responses, record the results including both latency and outcome (successfully process or rejected/dropped) 3. We expect two charts like below. Successfully processed API calls load latency load 3

Measurement of service instantiation time and closed-loop control time • The goal of this test is to measure the overall time consumed by a service instantiation process or closed-loop control process and subsequently break down to time used by each major step. • The results will reveal the most time-consuming steps and lead to further optimization. • This measurement relies on the analysis of each component’s log file. • Test process: - Complete an instantiation or closed-loop control process. - Set the time analysis granularity, e. g. , delta=1 second - Starting from the top-level, analyze the time used by each step. If the time use by a step > delta, then analyze the detailed operation of that step. See example below: • If a call to SDNC takes less than 1 second to complete, then the analysis stops at the SDNC level. • If a call to SDNC takes more than 1 second, then find out the time used by each step inside SDNC, such as AAI access, DB access, time used by configuring each VNF/PNF. 4