Kubernetes intro How can we simplify Perf Sonar

Kubernetes intro

How can we simplify Perf. Sonar management? • • Need to run some software on a set of nodes Config is repeated on all nodes Want to autodiscover new nodes New nodes info should be added to config file and propagated Containers + Container orchestration system

Kubernetes definition • Kubernetes is ancient Greek for "Helmsman". Root of the word "Governor", "Cybernetics". • Kubernetes is a "Container Orchestrator" or "Cluster Manager". • Places containers on nodes • Recovers automatically from failure • Basic monitoring, logging, health checking • Enables containers to find each other.

Google grown • Based on ideas proven at Google over 10 years • Google launches 2 billion containers per week. • Part of a larger set of tools that make up the internal Google platform. • Everything at google runs in containers

Linux containers

Docker container: layers, union filesystem • Docker images are readonly templates from which Docker containers are launched. • Only top layer is writeable • Copy on write • Only changed layers need to be transferred • Natively supports ZFS for storing layers (zvol)

We need more than just packing and isolation • Scheduling: where should my container run? • Lifecycle and health: Keep my containers running despite failures • Discovery: Where are my containers now? • Monitoring: What’s happening with my containers? • Auth{n, z}: Control who can do things to my containers • Aggregates: Compose sets of containers into jobs • Scaling: Making jobs bigger or smaller • Provisioning storage: block volumes, shared FS

PODs • Pods are the basic building block of Kubernetes • Pods are the smallest deployable units that are created and managed in Kubernetes • A Pod is a group of one or more deeply coupled containers • Shared network • Shared storage • Options to run the containers • Each pod is assigned a unique, DYNAMIC IP address

Service An interface to a set of replicated pods, which acts as a load-balancer Provides an abstraction layer No need to worry about containers location.

Other units • Replica. Sets: Ensures that the number of desired pods "replicas" are running at any time. • Deployments: Declarative way to describe the desired state of the application (pods, replica sets). • Easy version updates for any software Daemon. Sets: runs a POD on every node in a cluster Stateful. Sets: clustered applications • • Startup/shutdown ordering • Stable hostname and storage • Clustered My. SQL, Zookeeper, Elasticsearch, Cassandra Volume plugins: Config. Maps, Secrets, Host. Path, rook (ceph), nfs, iscsi, git. Repo, …

Networking • The pods are scheduled on a flat shared network across all nodes. • Each pod can communicate without proxies and translations (NAT) with other pods within the cluster. • Several implementations are available: Flannel, Weave, Open. Vswitch, Calico, kube-router • Kube-router talks BGP • • Can assign AS to groups of nodes Can export routing rules Supports External. IP for services Automatically connects remote nodes with TCP tunnels – doesn’t have to be in same subnet

Our approach to Perf. Sonar install Perf. Sonar Master Archive - a Deployment 1 POD, keep it alive, restart if necessary Test. Points - a Daemon. Set Runs 1 copy of POD on every node in a cluster Esmond - a Service Points to currently running Perf. Sonar container Cassandra DB - a Stateful. Set Nodes have their stable names, can elect a master node Postgre. SQL DB - inside the Perf. Sonar POD Required, can’t separate

Cluster autoconfig Mesh. Config List of testpoints Config JSON Perf. Sonar Central MA Measurement results Kubernetes API Schedules testpoint PODs on all nodes in the cluster Test. Point