Choosing the containerized cloud provisioning tool that best

Choosing the containerized cloud provisioning tool that best suits your need Smruti Soumitra Khuntia, Solution Architect Pradeep Kumar K S, Solution Architect Hemanth Kumar Nakkina, Solution Architect Ericsson Internal | 2018 -02 -21

Agenda Ø Why containers for cloud provisioning? Ø Containerized Open. Stack deployment Ø Key attributes of Containerized Deployments Ø Airship overview Ø Kolla overview Ø Triple. O overview Ø Comparison Ericsson Internal | 2018 -02 -21

Why containers for cloud provisioning? Ø Microservices help to decompose complex operations into smaller composable parts Ø Quicker to bring them up and down Ø Containers can scale, upgrade and monitored independently Ø Increasing efficiency, reliability and scalability Ø Opensource and thus fosters innovation and rapid development Ericsson Internal | 2018 -02 -21

Containerised Open. Stack deployment Ericsson Internal | 2018 -02 -21

Key features of Containerized Cloud Provisioning tools Ø Ø Ø Ø Ø Container technologies/tools Deployment strategies Infrastructure Configurations Logging, Monitoring, Health & Auditing Bare metal provisioning Container Networking Security Storage Support Day 2 Operations & Upgrades Ericsson Internal | 2018 -02 -21

Containerized Cloud Deployment Solutions Ericsson Internal | 2018 -02 -21

Introduction to Airship — Airship is a collection of loosely coupled but interoperable open source tools that declaratively automate cloud provisioning. — Airship manages the full lifecycle of data center infrastructure to deliver a production -grade Kubernetes cluster with Helm deployed artifacts, including Open. Stack-Helm from raw bare metal infrastructure. — Airship allows operators to manage their infrastructure deployments and lifecycle through the declarative YAML documents that describe an Airship environment. Ericsson Internal | 2018 -02 -21

Architecture Ericsson Internal | 2018 -02 -21

Airship Workflow Ericsson Internal | 2018 -02 -21

Airship Workflow central 1) Specify all info in YAML files a) Application templates Openstack helm charts Kubernetes helm charts b) Security Generate Passphrases, certificates PKI structure c) Site info Node hardware Disks, storage (ceph) setup Ports and VLANs (Oob/PXE/OAM/Data) ~350 site 5) Run the seed script a) Self hosted Kubernetes cluster b) Install undercloud components - Bare metal deployer c) Install applications - Openstack control plane - (Other applications) d) Transform node to first cluster controller Genesis/ Controller 1 Controller 2 Controller 3 2) Combine & validate site. yaml Ericsson Internal | 2018 -02 -21 4) Manually provision ”genesis” node 3) Generate seed script Genesis. sh (40 MB) Controller 4 Worker 2) Combine & Worker validate

Kolla Ericsson Internal | 2018 -02 -21

Kolla Consists of three main projects Kolla-Open. Stack (Kolla) Ø Provides Dockerfiles for each service and each process in the services Ø Container external configurations Kolla Images Ø Ansible code to build those containers Ø Lets you build your own images on your own private registry or use public images Kolla-Ansible Ø Kolla provides production-ready containers and deployment tools Ø Ansible code to deploy, start, configure and re-configure/upgrade a system Ø Allows for complete customization Ø Uses lightweight Docker containers to deploy Open. Stack services using Ansible. Ericsson Internal | 2018 -02 -21

Kolla images build process Ericsson Internal | 2018 -02 -21

Kolla Deployment Model/Workflow Ø Setup the baseline OS § Get OS Installed § Install required tools § Clone or pip install the Kolla tools Ø Configure and install Docker § Ensure shared mounts are enabled § Configure an insecure registry or pass TLS certificates information for local registry Ericsson Internal | 2018 -02 -21

Kolla Deployment Model/Workflow Ø Setup globals. yml and passwords. yml files § Globals is a way to turn off and on services § Define network and storage interactions with the underlying system. § Passwords defines default passwords, keys, UUIDs etc. § Can be generated automatically using kolla-genpwd tool Ø Setup multi node inventory Kolla-ansible bootstrap-servers –i inventory_file Kolla-ansible deploy –i inventory_file Ericsson Internal | 2018 -02 -21

Triple. O Deployment Overview Undercloud Open. Stack Triple. O Deploys, Updates Overcloud Open. Stack Monitors Tenant Facing Cloud Deployment and Management Cloud § Knowns as Overcloud § Knowns as Undercloud § The cloud responsible to run tenant § Responsible for deployment of overcloud workloads § Provide infrastructure command control § Visible to Tenant administrator § Only applicable for cloud operator § Provides cloud resources to run tenant § Conducts all lifecycle management § Apps/VNFs. Ericsson Internal | 2018 -02 -21

Physical View Ericsson Internal | 2018 -02 -21

Triple. O Deployment Flow Ericsson Internal | 2018 -02 -21

Components of Containerized Triple. O Ø Pre-Built Docker Images § Kolla Ø Configuration Generation § docker-puppet. py Ø Container management § Paunch Ø Updates and Upgrades § Ansible with Triple. O Heat templates Ericsson Internal | 2018 -02 -21

Config generation for containerized services docker-puppet. json Generates configs Bootstrap container Puppet docker-puppet. py Docker-puppet. py generates configs for each service and available through puppet hiera Ericsson Internal | 2018 -02 -21

Kolla Start § config. json Service Container Kolla start is responsible for: § Copying configuration files § Setting permissions § Starting the Service process Kolla Start copy config set permission Config dir Start Process Ericsson Internal | 2018 -02 -21

Comparisons Project Container Type Supported Containers Deployment Type Kolla-Ansible Docker Kolla Containers Ansible Triple. O Docker Kolla Containers Ansible Airship Docker LOCI Containers Kubernetes and Helm Ericsson Internal | 2018 -02 -21

Container Technologies/Tools Ø Airship § Uses Kubernetes § Helm charts ( from Open. Stack Helm ) § Loci for image builds Ø Triple. O § Paunch § Podman/Docker § Kolla-ansible § Kolla for image builds Ericsson Internal | 2018 -02 -21

Infrastructure configurations Ø Airship § Defines YAML manifests at global, type, site level § Reuse the files defined at global and type § Only site level manifests get modified for each site § Everything is a YAML configuration. Complex initially to understand the YAML manifests § YAML manifests are released by Airship community as part of Treasuremap project every month § On the brighter side, the manifests provide you control to define settings at a very low level Ø Triple. O § Defines manifests in Triple. O-heat-templates under § puppet/services § docker/services § extraconfig/service § YAML files with custom keys to modify Ericsson Internal | 2018 -02 -21

Deployment strategy Ø Airship § Deployment is performed via Genesis node which bootstraps the kubernetes cluster with control and compute nodes § Genesis node itself becomes one of the control nodes post the initial bootstrap process § This would mean that under cloud components are always running on the actual site. § By this virtue, the site manifests remain on Deckhand database registry § This would provide the current/actual state of under cloud within the site itself § Thus not needing a separate hardware for managing under-cloud LCM. Ø Triple. O § Deployment of the under cloud happens via seed node which bootstraps other nodes using an under-cloud Open. Stack § Under-cloud Open. Stack will be used to deploy the Open. Stack overcloud using heat templates and Kolla § Under cloud would be more of a plug & play and can be shut-off and turned-on whenever required. Ericsson Internal | 2018 -02 -21

Bare metal server management Ø Airship § MAAS Plugin for Drydock § i. DRAC/i. LO management using IPMI plugin, redfish plugin for drydock § Integration with Kubernetes cluster-api with Metal 3‘ with Airship ( WIP) Ø Triple. O § Uses Ironic project for bootstrapping bare metal servers § Update configuration in heat templates Ericsson Internal | 2018 -02 -21

Networking Ø Airship § Uses calico by default for control plane networking § Calico provides ability to set network policies § Can deploy any Kubernetes CNI plugins by changing the Airship manifests, provided helm charts are available for CNI Ø Triple. O § Networking in Containerized Triple. O is similar to Non Containerized Triple. O § Host based networking is leveraged for all containerized services § Docker *Host* Driver is utilized. Ericsson Internal | 2018 -02 -21

Security Ø Airship § § § Utilizes all the Kubernetes native security features Calico network policies Setting security context for POD/container ( privileges/non-privileged, linux capabilities) App. Armor Seccomp Ø Triple. O § Audit rules § Firewall management rules § AIDE ( File directory integrity checker) Ericsson Internal | 2018 -02 -21

Logging, Monitoring & Auditing Ø Logging § Airship ü Uses EFK stack ( Fluentd, Elastic. Search, Kibana) § Triple. O ü Uses EFK stack ( Fluentd*, Elastic. Search, Kibana) Ø Monitoring § Airship ü Uses Prometheus, Elastic. Search, Grafana § Triple. O ü Uses Sensu* *might be deprecated in future releases Ericsson Internal | 2018 -02 -21

Storage Ø Airship ü Supports Ceph and NFS storage drivers Ø Triple. O ü Support Ceph and other NFS storage drivers ü Under-cloud can be configured to use Swift/Cinder Object Storage as image backend Ericsson Internal | 2018 -02 -21

Updates & Upgrades Ø Airship § § Typical process is to use the same set of Under cloud tools to do Day 2 operations and upgrades Update the YAML manifests and deploy the new manifests with Shipyard Host impacting components ( Kubelet, Docker, kernel upgrades) using Argo Workflows ( WIP ) Upgrades for most of the services are handled via helm charts Ø Triple. O § Ansible scripts to perform updates, pre/post update operations triggered via Open. Stack overcloud command § Ansible scripts for the under cloud upgrade (using FFU) Ericsson Internal | 2018 -02 -21