s Kubernetes Support for VM Container based VNFs
s Kubernetes Support for VM & Container based VNFs - Request to approve incubation(Po. C) in Casablanca Contact : Srini Addepalli or Victor (Intel) for questions/clarifications or to provide feedback
ONAP – Support for K 8 S based remote regions OSS BSS APP Orchestrators ONAP Multi Cloud Service • Current support: Openstack based remote Clouds, Support multiple Openstack variations – Windriver Titanium, VMWare VIO, Native Newton, Ocata. Only VM based VNFs. SDNC (Fabric/WAN Control) • Need support for containerized VNFs. • Need support where K 8 S is used for both VMs and Containers (To avoid multiple controls and also to utilize same compute nodes for both VMs and containers) Site (*) Site (With Openstack VIM) (Two logical edges – Openstack for VMs and K 8 S for containers) Site (With K 8 S for both VMs and Containers)
ONAP – VM and Container networking (Uniform) OSS BSS APP Orchestrators ONAP Multi Cloud Service Current: Networks are created and VM are placed in various networks (VLs and CPs – Virtual Links and Connection Points). SDNC (Fabric Control) POD VM VM C C Need: Some workloads can be containers and networking is expected to be extended to containers Site (edge/cloud) POD C K 8 S Master Network Controller Network 1 (VLAN/VXLAN/GENEVE) Network N (VLAN/VXLAN/GENEVE) v. Switch (e. g OVSDPDK) networking NFVI/VMM SRIOV networking Compute Servers – Hardware (CPU/Memory/SRIOV-NIC/FPGA/GPU) etc…
ONAP – VM and Container Storage (Uniform) OSS BSS APP Orchestrators ONAP Volumes are expected to be extended to Containers. SDNC (Fabric Control) Multi Cloud Service Site – Edge/Cloud POD VM NFVI/VMM VM C C K 8 S Master POD Network Controller C Ceph Storage Compute Servers – Hardware (CPU/Memory/SRIOV-NIC/FPGA/GPU) etc…
ONAP – Service Orchestration language (Model Driven – Model being TOSCA based) Current (Being defined in Modeling committee) - Standardizing on TOSCA based NSD Service description - Moving away from Openstack HOT templates. - Multi-Cloud OS plugins are expected to convert TOSCA VNFC information to Openstack API Child NSD VNFD VL Need: VNFC/VDU (VM based) Image Compute Storage Network HPA Monitoring Scaling User data VNFC/VDU VM VNFC/VDU (POD) - VNFC/VDU (POD) C 1 Image Compute, Storage Network, HPA Monitoring, Scaling User data C 2 Continue with TOSCA based Service description for VMs and containers managed by K 8 S. - Multi-Cloud K 8 S plugin convert from VNFC information K 8 S API at run time. - Multi Cloud to convert from VL and CP to Network controller (eg. OVN) (No K 8 S Yaml, Helm in the 1 st phase)
Current flow - HEAT based Service Orchestration (SO example) • • NSD Instantiation time 1 3 2 SO OOF 5 AAI 6 Multi-Cloud Service Remote Cloud regions 4 Policy frame work A Service consists of multiple HOTs Each HOT is a VNF, set of VLs and CPs 1. Service instantiation request 2. SO decomposes service to VNFs. For each VNF 3. Requests OOF to get best site and flavors (for each VDU) to place VNF. 4. OOF with the help of A&AI and Policy, identifies best site and flavors for each VDU 5. Responds to SO. SO replaces flavors in HOT 6. Requests Multi-Cloud service (with HOT template as input) to bring VMs in remote site. Not shown: SO & Multi-Cloud store Virtual inventory information in A&AI. OOF and Multi-Cloud interaction wrt capacity checks and SDN-C interaction from SO.
Proposal 1 (TOSCA based Service/VNFD/VDU definition)
TOSCA based workflow (Simplified) – VNF Onboarding 1. Operator onboard VNFs with example VNFD/VL/Volume information in TOSCA grammar. 2. Service creator creates service templates in TOSCA format without any considerations to the Cloud region on which this VNF would be brought up.
TOSCA based workflow (Simplified) – Discussions in progress in community in SO. VF-C Orchestration is already based on TOSCA • • • NSD Instantiation time 1 3 2 SO OOF 5 AAI 6 Multi-Cloud Service Remote Cloud regions 4 Policy frame work A service represented as NSD (a nested NSD) A NSD can contain multiple VNFs. Each VNF can contain multiple VDUs 1. Service instantiation request 2. SO decomposes NSD to child NSDs, decompose individual VNFs of each NSD For each VNFD 3. Requests OOF to get best site and flavor(s) 4. OOF with the help of A&AI and Policy, identifies best site and flavors 5. Responds to SO. 6. Requests Multi-Cloud service (with VNFD/VL TOSCA fragment as input) to bring VLs, Volumes and workloads in cloud region. It also sends metadata (example : OOF returned site and flavors information). Multi-Cloud service with the help of K 8 S plugin will use metadata and TOSCA based VNFD/VL/Volume input and makes K 8 A API calls to selected cloud-region.
Phase 1 Scope - Focus on Multi-Cloud Service Phase 1 : NSD Instantiation time Test Service (Simulating SO/VFC) Multi Cloud Service Openstack Plugins – VIO, R 2+DM Driven API K 8 S Plugin + OVN Plugin TOSCA parser Existing API - Multi- Cloud Service changes Test with VMs and containers. Test using scripts on top of Multi-Cloud Service(simulating SO actions) Identify TOSCA modeling changes. Provide feedback to bigger architecture to support K 8 S. Testing - DP Container: DPDK based Router - IOT infrastructure: Edge. XFoundry services as containers - Ensure that both VM and containers can exist on same network and works when placed on same compute node. Model Driven API: Titanium, OS ocata & newton - TOSCA based VNFD/VL in API + Metadata API categories : Compute, Storage and Networks. - Converts TOSCA to K 8 S API data – Deployment, Service, Endpoint Converts TOSCA VL fragment to OVN API, SRIOV API, flannel API Use Aria TOSCA parser to parse input. - Virtlet for VMs, Dockers for Containers. OVN for networking K 8 S Plugin (Go lang) K 8 S Master OVN N Controller Router VFW Edge. X Cntnr VM Minion Docker, virtlet, OVN S Controller, OVS-DPDK Site - Edge/Cloud Site (Mostly deployment related challenges)
Proposal 2 (TOSCA based Service/VNFD/VDU definition + Cloud technology specific Artifacts)
TOSCA + Cloud technology specific artifacts Many VNF vendors today provide Kubernetes deployment yaml examples or Helm deployment example. TOSCA is not popular for containers today. 1. VNF vendor provides set of Feedback from vendors: 1. VNF image URL - Kubernetes yaml for containers is ubiquitous today. Vendors 2. Cloud technology specific artifacts (example: may not like to provide deployment information in many Kubernetes deployment/service-account yaml formats. data, Helm charts, ARM, HOT) - TOSCA based container standards can take lot of time and 2. Operator onboards above information fear that it may not be able to provide all capabilities K 8 S 3. Service creator creates service templates in TOSCA format. provides today. - Many vendors test their VNFs on specific cloud technologies (K 8 S, Dockerswarm, mesos, K 8 S with HELM etc…) and some times VNF images are different for different cloud technologies.
TOSCA based workflow (Simplified) – With Cloud technology specific artifact NSD Instantiation time 1 3 2 SO OOF 5 AAI 6 Multi-Cloud Service Remote Cloud regions 4 Policy frame work 1. Service instantiation request 2. SO decomposes NSD to child NSDs, decompose individual VNFs of each NSD and also from artifacts For each VNF 3. Gathers information from the artifacts, the cloud technologies that this VNFs can be placed. Requests OOF to get best site and flavor(s) 4. OOF with the help of A&AI and Policy, identifies best site and flavors. OOF also uses cloud-technology information in A&AI in considerations. 5. Responds to SO. 6. Requests Multi-Cloud service with Metadata (example : OOF returned site and flavors information) and Cloud technology specific artifact (or artifact reference) Multi-Cloud service with the help of K 8 S plugin makes call to cloud region. Plugin is expected to interpret the artifact, modify based on metadata and then make calls to Cloud-region to bring up VNFs NO TOSCA interpreter needed in Multi Cloud
Phase 1 Scope - Focus on Multi-Cloud Service Phase 1 : NSD Instantiation time Test Service (Simulating SO/VFC) Multi Cloud Service Existing API Openstack Plugins – VIO, API (Metadata + artifact) K 8 S Plugin + OVN Plugin OVN N Controller Multi- Cloud Service changes Test with VMs and containers. Test using scripts on top of Multi-Cloud Service(simulating SO actions) Identify TOSCA modeling changes. Provide feedback to bigger architecture to support K 8 S. Testing - DP Container: DPDK based Router - IOT infrastructure: Edge. XFoundry services as containers - Ensure that both VM and containers can exist on same network and works when placed on same compute node. Site (Mostly deployment related challenges) Titanium, OS ocata & newton K 8 S Master - Router VFW Edge. X Cntnr VM Minion Docker, virtlet, OVN S Controller, OVS-DPDK Site - Edge/Cloud - Virtlet for VMs, Dockers for Containers. - OVN for networking APIs: - Create VNF (in K 8 S, it is equivalent to deployment) - Delete VNF - Query VNF - Scale out VNF (Not scaleout of individual Container in VNF as that is taken care by K 8 S master in the cloud-region) K 8 S plugin: - Takes metadata and K 8 S yaml artifact (which is opaque to rest of ONAP system) - Interprets artifact information, update using metadata before making API calls to cloud region.
Asks from Architecture Subcommittee • Approve Phase 1 • Approve repository creation • Next steps after approval: - Detailed architecture specifications – Model Driven API, K 8 S+OVN Plugin, TOSCA Parser - Document test cases - Work with Multi-Cloud PTL/Team - Request contributors and create JIRA EPICs/stories.
s
Items that came up in various meetings – For Discussions 1. Co-existence of VMs and containers in NSD and granularity a) Would a VNF contain all VMs or container or is the mix needed? b) Would VMs and container can come up on same compute node? 2. TOSCA Orchestration a) Support for TOSCA capabilities to represent container workloads (*) b) Support for Helm charts for VNFC c) Support for POD deployment spec for VNFC 3. Networking: a) ONAP controlling network across VMs and containers – Which CNI? OVN, Kuryr and others? OVN (*) b) Multi-Cloud plugin to program CNI 4. Placement decisions (Should not bypass OOF) a) Taking advantage of SO and OOF for placement decisions. Can SO decompose NSD to VNFC level? (Some work to be done at SO and OOF) b) Does SO call Multi-Cloud VIM for each VNFC? It needs to be for container support (Yes) c) Would all workloads in a VNF in one location or would there be requirement to put them in different regions? 5. Overlapping features between K 8 S and ONAP - Monitoring, Replication, Liveness checks etc. . Use ONAP, Use K 8 S? K 8 S(*) 6. Bare-metal containers – Yes 7. Containers in VMs – Topics for discussions 8. Support for other Container orchestration systems (Dockerswarm) & Orchestration management systems (Openshift, Rancher 2. 0 etc…) 9. Others? ? ?
s
- Slides: 18
