Container Uday Hiwarale 1 2 Container Uday Hiwarale

Container [Uday Hiwarale] 1

2 Container [Uday Hiwarale]

Definition 3 �A container is an isolated execution environment where one or many processes can run in isolation.

Containerization 4 �The action of creating containers and running your application as a process inside it; is known as containerization. �Since there is no Hypervisor used to create different isolated environments directly on host operating system (these are containers), containerization sometimes is called as OS Level Virtualization.

Linux Containerization 5 �Linux Kernel provides such containerization mechanism where you can create many containers on a single Linux hosts called as Linux Based Containers or LXCs.

Mechanisms Supporting Linux Containers 6 �As stated, a process inside the container has isolated environment. That includes � network interface (to obtain IP addresses), � process ids (PIDs), mount points etc. �Linux Kernel out of the box provides some of the features like Namespaces and Control Groups to make this happen.

Namespaces and Control Groups 7 �Namespaces are the features of Linux Kernel that partition kernel resources like Network Interface (net), Mount Points (mnt), Process Ids (PIDs) etc. Hence we can create sets of processes having same resource identifiers, for example, IP Addresses because they share different namespaces. �Control Groups are also namespaced and they control how much system resources like CPU and Memory is allocated to sets of processes.

General Properties of a Container 8 �In general, a container is nothing but a set of processes we just talked about. �A container has unique namespace and all processes running inside it will have their share of resources allocated by container’s control group. �Any process inside the container will not be able to see or interface with the resources allocated to other containers. �All containers shares same kernel (of the host operating system) and when a container needs different kernel, then virtualization has to be provided.

Content of a Container 9 �A container is a package that contains y our programs, application level binaries and libraries, environment variables etc. �It utilizes host’s operating system hence OS level libraries, binaries and drivers are shared by container engine.

VM vs. Container [Source: Google Cloud] 10

11 Application Containers [Gabor Nagy]

Application Containers 12 �Application containers are designed to package and run a single service. �Container technologies like Docker and Rocket are examples of application containers. �So even though they share the same kernel of the host there are subtle differences make them different, which I would like to talk about using the example of a Docker container:

Run a Single Service As a Container 13 �When a Docker container is launched, it runs a single process. This process is usually the one that runs your application when you create containers per application. �This very different from the traditional OS containers where you have multiple services running on the same OS.

Layers of Containers 14

Layer 15 �Any RUN commands you specify in the Dockerfile creates a new layer for the container. �In the end when you run your container, Docker combines these layers and runs your containers. �Layering helps Docker to reduce duplication and increases the re-use. This is very helpful when you want to create different containers for your components. �You can start with a base image that is common for all the components and then just add layers that are specific to your component. �Layering also helps when you want to rollback your changes as you can simply switch to the old layers, and there is almost no overhead involved in doing so.

Base Image [IBM] 16 �A base image is the image that is used to create all of your container images. �Your base image can be an official Docker image, such as Centos, or you can modify an official Docker image to suit your needs, or you can create your own base image from scratch.

17 Understanding Docker "Container Host" vs. "Container OS" for Linux and Windows Containers [Floyd Hilton]

Container Host 18 �A container host is also called a Host OS. �The Host OS is the operating system on which the Docker client and Docker daemon run. �In the case of Linux and non-Hyper-V containers, the Host OS shares its kernel with running Docker containers. �For Hyper-V each container has its own Hyper-V kernel.

Container OS 19 �A container OS is also called a Base OS. �The base OS refers to an image that contains an operating system such as Ubuntu, Cent. OS, or windowsservercore. �Typically, you would build your own image on top of a Base OS image so that you can take utilize parts of the OS. �Note that windows containers require a Base OS, while Linux containers do not

Operating System Kernel 20 �The Kernel manages lower level functions such as memory management, file system, network and process scheduling.

Example 21

Explanation of the Previous Diogram 22 �The Host OS is Ubuntu. �The Docker Client and the Docker Daemon (together called the Docker Engine) are running on the Host OS. �Each container shares the Host OS kernel. �Cent. OS and Busy. Box are Linux Base OS images. �The “No OS” container demonstrates that you do not NEED a base OS to run a container in Linux. You can create a Docker file that has a base image of scratch and then runs a binary that uses the kernel directly.

Function of Base Image OS [ITREAD 01] 23 �The base image OS is used for filesystem, binaries, etc.

Why do we use a OS Base Image with Docker if containers have no Guest OS? (1) [drookie] 24 �Since all Linux distributions run the same (yup, it's a bit simplified) Linux kernel and differ only in userland software, it's pretty easy to simulate a different distribution environment - by just installing that userland software and pretending it's another distribution. �Being specific, installing Cent. OS container inside Ubuntu OS will mean that you will get the userland from Cent. OS, while still running the same kernel, not even another kernel instance.

Why do we use a OS Base Image with Docker if containers have no Guest OS? (2) [drookie] 25 �So lightweight virtualization is like having isolated compartments within same OS. �On the contrary, real virtualization is having another full-fledged OS inside host OS. That's why docker cannot run Free. BSD or Windows inside Linux. �If that would be easier, you can think docker is kind of very sophisticated and advanced chroot environment.

26 Docker Images and Layers [docker]

Docker Images and Layers 27 �A Docker image is built up from a series of layers. �Each layer represents an instruction in the image’s Dockerfile. �Each layer except the very last one is read-only.

Example 28 FROM ubuntu: 15. 04 COPY. /app RUN make /app CMD python /app. py �This Dockerfile contains four commands, each of which creates a layer. �The FROM statement starts out by creating a layer from the ubuntu: 15. 04 image. �The COPY command adds some files from your Docker client’s current directory. �The RUN command builds your application using the make command. �Finally, the last layer specifies what command to run within the container.

Properties of Layers (1) 29 �Each layer is only a set of differences from the layer before it. �The layers are stacked on top of each other. �When you create a new container, you add a new writable layer on top of the underlying layers. �This layer is often called the “container layer”. �All changes made to the running container, such as writing new files, modifying existing files, and deleting files, are written to this thin writable container layer.

Properties of Layers [Uday Hiwarale] 30 �A docker image follows modified Union File System such as Au. FS. �Each instruction in Dockerfile creates a readonly Au. FS layer. �These layers are stacked on each other as mentioned in Dockerfile. �Each layer is only a set of differences from the layer before it.

A Container Based on the Ubuntu 15. 04 Image 31

Container and Layers 32 �The major difference between a container and an image is the top writable layer. �All writes to the container that add new or modify existing data are stored in this writable layer. �When the container is deleted, the writable layer is also deleted. The underlying image remains unchanged.

Share Access 33 �Because each container has its own writable container layer, and all changes are stored in this container layer, multiple containers can share access to the same underlying image and yet have their own data state.

Multiple Containers Sharing the Same Ubuntu 15. 04 image. 34

35

36 The Underlying Technology [Docker]

The Underlying Technology 37 �Docker is written in Go and takes advantage of several features of the Linux kernel to deliver its functionality.

Namespaces 38 �Docker uses a technology called namespaces to provide the isolated workspace called the container. �When you run a container, Docker creates a set of namespaces for that container. �These namespaces provide a layer of isolation. Each aspect of a container runs in a separate namespace and its access is limited to that namespace.

Linux Namespaces 39 �Docker Engine uses namespaces such as the following on Linux: namespace, control group. �The pid namespace: Process isolation (PID: Process ID). �The net namespace: Managing network interfaces (NET: Networking). �The ipc namespace: Managing access to IPC resources (IPC: Inter. Process Communication). �The mnt namespace: Managing filesystem mount points (MNT: Mount). �The uts namespace: Isolating kernel and version identifiers. (UTS: Unix Timesharing System).

Control groups 40 �Docker Engine on Linux also relies on another technology called control groups (cgroups). �A cgroup limits an application to a specific set of resources. �Control groups allow Docker Engine to share available hardware resources to containers and optionally enforce limits and constraints. For example, you can limit the memory available to a specific container.

Union file systems 41 �Union file systems, or Union. FS, are file systems that operate by creating layers, making them very lightweight and fast. �Docker Engine uses Union. FS to provide the building blocks for containers. �Docker Engine can use multiple Union. FS variants, including AUFS, btrfs, vfs, and Device. Mapper.

Container format 42 �Docker Engine combines the namespaces, control groups, and Union. FS into a wrapper called a container format. �The default container format is libcontainer. �In the future, Docker may support other container formats by integrating with technologies such as BSD Jails or Solaris Zones.

Finding Docker container processes from host point of view (1) [Stack. Overflow] 43 �You can use docker top command. �This command lists all processes running within your container. �For instance this command on a single process container on a box displays: UID PID PPID C STIME TTY TIME CMD root 14097 13930 0 23: 17 pts/6 00: 00 /bin/bash �To simply get the main process id within the container use this command: docker inspect -f '{{. State. Pid}}' <container id>

Finding Docker container processes from host point of view (2) [Stack. Overflow] 44 � Another way to get an overview of all Docker processes running on a host is using generic cgroup based systemd tools. � systemd-cgls will show all our cgroups and the processes running in them in a tree-view, like this:

Finding Docker container processes from host point of view (3) [Stack. Overflow] 45 �The process run in a docker container is a child of a process named containerd-shim (in Docker v 18. 09. 4) First figure out the process IDs of the containerd-shim processes. For each of them, find their child process. % pgrep containerd-shim 7105 7141 7248 �To find the child process of parent process 7105: % pgrep -P 71057127

46 Docker [Uday Hiwarale]

Docker 47 �Docker is nothing but a containerization software. �Docker engine is nothing but a container engine. Source

Docker Engine 48 �A docker engine consist of Docker daemon and other utilities to create, destroy and manage containers.

Docker Daemon 49 �Docker daemon is a process running in background that receives commands from local or remote Docker client(CLI) using HTTP REST protocol to manage containers. �Hence Docker is said to follow client-server architecture where server is Docker daemon.

What You Get after a Docker Installation? 50 �When you install Docker on your system, you get Docker engine, Docker command line interface (Docker client) and other GUI utilities. �When you start your Docker, it will start the Docker daemon.

Content of a Docker Container 51 �A docker contains application code and other dependencies. �These other dependencies is what makes a container a “container”. �These other dependencies consists of necessary (application specific) libraries, Binaries, and other resources which is needed for our application to function.

Docker Container Example 52 �An example of a container would be a node. js server. �So our application code would consist of server. js containing application code and node_module library. �But to run it, we need node installed in the container, hence we need a node binary file. �node. js might depend on other binaries and libraries, hence we need that too. �Then node. js needs an OS to run on for example Cent. OS, hence we need a customized binary for that too which Docker engine could utilize to talk to guest OS and kernel.

Docker Image 53 �The node server example we just talked about contains many pieces that need to be present in the container so that our application could work. �A Docker image is a zipped box that contains all these pieces.

Create a Container from an Image 54 �We instruct Docker client to create a container from this image. �Docker client instruct Docker daemon to unzip the image, reads the content and launch the container with server. js executing as a process. �Depending on other instructions in the image, Docker daemon might expose some ports from the container which we can listen to and/or mount volumes and do other things.

Dockerfile 55 �To create a Docker image, we need a Dockerfile. �Dockerfile is a configuration file with instructions to tell Docker engine, how to build an image. �These instructions can be what would be the base image, what would be the working directory inside the OS running inside the container, what application specific files need to be copied from the system, what ports need to be exposed in the container and other zillions of things.

Base Image 56 �A base image is an official image provided by Docker in which we will add our application specific code and instructions. �A base image can contain Cent. OS operating system installed with Apache server.

Container Host [Floyd Hilton] 57 �A container host is also called a Host OS. �The Host OS is the operating system on which the Docker client and Docker daemon run. �In the case of Linux and non-Hyper-V containers, the Host OS shares its kernel with running Docker containers. �For Hyper-V each container has its own Hyper-V kernel.

58

59 Docker [Docker]

Docker Overview 60 �Docker is an open platform for developing, shipping, and running applications. �Docker enables you to separate your applications from your infrastructure so you can deliver software quickly. �With Docker, you can manage your infrastructure in the same ways you manage your applications.

The Docker Platform 61 �Docker provides the ability to package and run an application in a loosely isolated environment called a container. �The isolation and security allow you to run many containers simultaneously on a given host.

Docker Engine 62 �Docker Engine is a client-server application with these major components: A server which is a type of long-running program called a daemon process (the dockerd command). A REST API which specifies interfaces that programs can use to talk to the daemon and instruct it what to do. A command line interface (CLI) client (the docker command).

Interactions between Components 63 • The CLI uses the Docker REST API to control or interact with the Docker daemon through scripting or direct CLI commands. • Many other Docker applications use the underlying API and CLI. • The daemon creates and manages Docker objects, such as images, containers, networks, and volumes.

Linux Docker Engine [edureka] 64 � In a Linux Operating system, there is a Docker client which can be accessed from the terminal and a Docker Host which runs the Docker Daemon. � We build our Docker images and run Docker containers by passing commands from the CLI client to the Docker Daemon.

Windows Docker Engine [edureka] 65 � In case of Windows/Mac there is an additional Docker Toolbox component inside the Docker host. � This Docker Toolbox is an installer to quickly and easily install and setup a Docker environment on your Windows/i. OS. � Docker Toolbox installs Docker Client, Machine, Compose (Mac only), Kitematic and Virtual. Box.

Docker Architecture 66 �Docker uses a client-server architecture.

Docker Client vs. Docker Daemon 67 �The Docker client talks to the Docker daemon, which does the heavy lifting of building, running, and distributing your Docker containers. �The Docker client and daemon can run on the same system, or you can connect a Docker client to a remote Docker daemon. �The Docker client and daemon communicate using a REST API, over UNIX sockets or a network interface.

The Docker daemon 68 �The Docker daemon (dockerd) listens for Docker API requests and manages Docker objects such as images, containers, networks, and volumes. �A daemon can also communicate with other daemons to manage Docker services.

The Docker client 69 �The Docker client (docker) is the primary way that many Docker users interact with Docker. �When you use commands such as docker run, the client sends these commands to dockerd, which carries them out. �The docker command uses the Docker API. �The Docker client can communicate with more than one daemon. �Docker client is a command line tool that allows the user to interact with the Docker daemon in client server manner using API’s[Pawan. Garia].

Docker registries 70 �A Docker registry stores Docker images. �Docker Hub is a public registry that anyone can use, and Docker is configured to look for images on Docker Hub by default. �You can even run your own private registry. �If you use Docker Datacenter (DDC), it includes Docker Trusted Registry (DTR). �When you use the docker pull or docker run commands, the required images are pulled from your configured registry. �When you use the docker push command, your image is pushed to your configured registry.

Image 71 �An image is a read-only template with instructions for creating a Docker container. �Often, an image is based on another image, with some additional customization. For example, you may build an image which is based on the ubuntu image, but installs the Apache web server and your application, as well as the configuration details needed to make your application run.

Image [Sebastian] 72 �A Docker image is containing everything needed to run an application as a container. �This includes: code runtime libraries environment variables configuration files �The image can then be deployed to any Docker environment and executable as a container.

Dockerfile 73 �You might create your own images or you might only use those created by others and published in a registry. �To build your own image, you create a Dockerfile with a simple syntax for defining the steps needed to create the image and run it. �Each instruction in a Dockerfile creates a layer in the image. �When you change the Dockerfile and rebuild the image, only those layers which have changed are rebuilt. �This is part of what makes images so lightweight, small, and fast, when compared to other virtualization technologies.

Container 74 �A container is a runnable instance of an image. �You can create, start, stop, move, or delete a container using the Docker API or CLI. �You can connect a container to one or more networks, attach storage to it, or even create a new image based on its current state. �A container is a process (or a groups of processes), but with more isolation from the OS than your run-of-themill process. BUT with less isolation than a VM, which comes with the tradeoff of less security [Jessica G. ].

Container [Saurabh] [] 75 � You can create Docker Containers, these containers will contain all the binaries and libraries required for your application or microservice in my case. � So your application is present in a container, or you have containerized your application. � Now, that same container can be used in the Test and Prod environment.

Container [Docker] 76 � Containers are an abstraction at the app layer that packages code and dependencies together. � Multiple containers can run on the same machine and share the OS kernel with other containers, each running as isolated processes in user space. � Containers take up less space than VMs (container images are typically tens of MBs in size), can handle more applications and require fewer VMs and Operating systems

Isolation of Containers 77 �By default, a container is relatively well isolated from other containers and its host machine. �You can control how isolated a container’s network, storage, or other underlying subsystems are from other containers or from the host machine. �A container is defined by its image as well as any configuration options you provide to it when you create or start it. �When a container is removed, any changes to its state that are not stored in persistent storage disappear.

Start a container [Pawan. Garia] 78 �docker run command is used to start a container from the Image with other command parameters. �Below image describes the process and commands used for creating a running container from simple text file.

Example docker run command 79 �The following command runs an ubuntu container, attaches interactively to your local command-line session, and runs /bin/bash. �$ docker run -i -t ubuntu /bin/bash

What Happens When the above Instruction Is Executed (1) 80 1. If you do not have the ubuntu image locally, Docker pulls it from your configured registry, as though you had run docker pull ubuntu manually. 2. Docker creates a new container, as though you had run a docker container create command manually. 3. Docker allocates a read-write filesystem to the container, as its final layer. This allows a running container to create or modify files and directories in its local filesystem.

What Happens When the above Instruction Is Executed (2) 81 Docker creates a network interface to connect the container to the default network, since you did not specify any networking options. This includes assigning an IP address to the container. By default, containers can connect to external networks using the host machine’s network connection. 5. Docker starts the container and executes /bin/bash. Because the container is running interactively and attached to your terminal (due to the -i and -t flags), you can provide input using your keyboard while the output is logged to your terminal. 6. When you type exit to terminate the /bin/bash command, the container stops but is not removed. You can start it again or remove it. 4.

Containers vs. Processes (1) [sites. google] 82 �Containers really are processes with their full environment. �A computer science textbook will define a process as having its own address space, program, CPU state, and process table entry. �But in today’s software environment this is no longer the full story. �The program text is actually memory mapped from the filesystem into the process address space and often consists of dozens of shared libraries in addition to the program itself, thus all these files are really part of the process.

Containers vs. Processes (2) [sites. google] 83

Containers vs. Processes (3) [sites. google] 84 �In addition, a running process usually requires a number of files from the environment (typically in /etc or /var/lib on Linux) and this is not just for config files or to store program data. For example, any program making SSL connections needs the root CA certs, most programs need locale info, etc. �All these shared libraries and files make the process super-dependent on its filesystem environment. What containers do is encapsulate the traditional process plus its filesystem environment. �The upshot of all this is that it’s much more appropriate to think of a container as being a better abstraction for a process than as a slimmed-down VM.

Containers vs. Processes (4) [sites. google] 85 �The right way to think about Docker is thus to view each container as an encapsulation of one program with all its dependencies. �The container can be dropped into (almost) any host and it has everything it needs to operate. �This way of using containers leads to small and modular software stacks and follows the Docker principle of one concern per container. �A blog post by Jerome Petazzoni has a good discussion on all this.

Containers vs. Processes (5) [sites. google] 86 � An aspect of Docker that leads many first-time users astray is that most containers are built by installing software into a base OS container, such as Ubuntu, Cent. OS, etc. � It’s thus easy to think that the container is actually going to “run” this base OS. � But that’s not true; rather, the base OS install only serves the purpose of starting out with a standard filesystem content as the environment on which the program being run in the container can rely upon. � One could install just the files actually required by the program into the container, but targeting such minimalist container content is often difficult, cumbersome and inflexible. � Besides, the way Docker uses overlay filesystems minimizes the cost of the extra unused files.

87

88

89

90 Supplementary Materials

91 OS Containers [Gabor Nagy]

OS Containers 92 �OS containers are virtual environments that share the kernel of the host operating system but provide user space isolation. If that would be easier, you can think docker is kind of very sophisticated and advanced chroot environment. [drookie] �You can install, configure and run different applications, libraries, etc. , just as you would on any OS. �Anything running inside a container can only see resources that have been assigned to that container.

A Purpose of OS Containers 93 �OS containers are designed to run multiple processes and services.

Advantages of OS Containers 94 �OS containers are useful when you want to run a fleet of identical or different flavors of distros. �Most of the times containers are created from templates or images that determine the structure and contents of the container. �It thus allows you to create containers that have identical environments with the same package versions and configurations across all containers.

Forms of OS Containers 95

Container Technologies 96 �Container technologies like LXC, Open. VZ, Linux VServer, BSD Jails, and Solaris zones are all suitable for creating OS containers. �Docker was built on top of LXC. If you look at the Docker FAQ, they mention a number of points which point out the differences between LXC and Docker.

Docker objects 97 �When you use Docker, you are creating and using images, containers, networks, volumes, plugins, and other objects.