Mesos and Open Stack the perfect tag team

Mesos and Open. Stack: the perfect tag team for containers Ton Ngo Paolo Dettori Brad Topol @tango 245 @ingpaolodettori @bradtopol Austin Open. Stack Summit, 4/27/2016 1

Introduction • Motivation • Open. Stack Iaa. S: compute, network, storage • Mesos: efficient resource utilization • Kubernetes, Swarm: user experience for containers orchestration • Use cases • Optimized environment to deploy both Kubernetes and Swarm workloads • Sharing of network, storage from Open. Stack • Talk outline • Mesos, Kubernetes, Swarm architecture • Running on Open. Stack, demo 2

Background - Swarm in a nutshell • Exposes several Docker Engines as a single virtual Engine • Exposes standard Docker API • Tools that work with Docker can use Swarm to transparently scale to multiple hosts. Eg. , Compose, Jenkins, Docker client Swarm Manager Scheduler Docker daemon Discovery Docker daemon Containers Swarm Node 3

Background – Kubernetes in a nutshell Cluster state • Open source orchestration system for Docker containers – Handles scheduling onto nodes in a compute cluster – Actively manages workloads to ensure that their state matches the users declared intentions – "labels" and "pods” to group into logical units for easy management and discovery – Replication controllers, services – Model is quite different from native docker API / Swarm, cannot leverage Docker tools Kubernetes master -API server -Scheduler -Controller Mgr Etcd cluster Support HA mode Kubelet/ Proxy Docker Engine Host 4

Background: Mesos in a nutshell • Open-source cluster manager – Efficient resource isolation and sharing across distributed applications, or frameworks – Enables siloed applications to be consolidated on a shared pool of resources, delivering: • Higher utilization • Better application performance – Rich framework ecosystem • Hadoop, Spark, Kubernetes, Marathon, Docker, Rocket, Mongo. DB, Elastic Search … 5

Using Swarm & Kubernetes with Mesos • Mesos manages the actual resources on the cluster • Incoming API/CLI are stored in a queue, waiting for offers from Mesos • The framework’s scheduler is used to choose the target host from the Mesos offers • The framework sends a “task” to Mesos slave to create the container Swarm Docker CLI/API Kubernetes framework Scheduler Offers framework Tasks to Mesos Master Mesos Agents 6

Deploying on Open. Stack • Magnum • • Prep: key pair, image Create Mesos baymodel Create Mesos bay Launch service containers for Kubernetes, Swarm • Ansible scripts + Shade • • • Prep: key pair, image, network Create config file Deploy Nova instances Install software prereq Launch service containers for Mesos, Kubernetes, Swarm, etcd, zookeeper • Normal interface for Kubernetes, Swarm 7

Networking • Default networking: Docker Mesos slaves • Use host IP and port Nova instance • Kuryr for Swarm • Docker libnetwork using Neutron • Create network in Docker = Neutron network • Create containers connected to networks • Mesos patches needed for Swarm on Mesos • Kuryr for Kubernetes container container subnet Neutron network 1 Neutron network 2 • Spec under review for CNI plugin • Planned for Newton cycle 8

Storage • Default • Use host storage Mesos slaves • Cinder Nova instance • Additional storage for workload • Persistent storage • Cinder for Kubernetes pods • Docker volume driver • Kuryr • New mission to support storage • On-going discussion container Nova instance container as Docker storage Cinder container as container volume Cinder 9

Lifecycle operations • Add/remove nodes • Network • Manage private networks in cluster • Connect to existing network • Storage • Add block storage to nodes • Existing persistent storage from users. • Monitoring cluster health • Error recovery • Software upgrade: • node OS • Mesos • Framework: Kubernetes, Swarm 10

Demo 1. 2. 3. 4. 5. 6. Deploy cluster by Ansible scripts Open. Stack infrastructure Mesos UI Deploy container on Kubernetes framework Deploy container on Swarm framework How container creation scales 11

Communities • Open. Stack • Magnum • Kuryr • Mesos (selected contributions) • Optimistic Offer Phase 1 [MESOS-1607], Allocator Refactoring for Optimistic Offering [Mesos-4392, Mesos -4553], Fine grained offer [MESOS-3765], CNI support [MESOS-4641], Volume Isolator [MESOS-4355], Enhance Docker. Executer to support Docker's user-defined networks [MESOS-4369] • Kubernetes Mesos framework (selected contributions) • add docker labels to containers (PR #13612), MESOS: added docs on cold-standby HA mode for K 8 sm (PR #21894), etcd-based abac authorizor plugin (PR #23502), Create a generic interface from kubelet to cadvisor (PR #19708), collector interfaces and structs (PR #19951) • Swarm framework (selected contributions) • Include docker plugin mechanism in Swarm (PR #1646), ACL in Swarm (PR #1366), Pass the X-Registry. Auth info in create. Container for the internal pull (PR #1350), Change the scheduler lock in Mesos cluster (PR #1554), New network functions for mesos cluster: Create. Network(), Remove. Network(), and Networks() (PR #1633) 12

Thank you