CrossCloud Migration of Container Persistent Data Huawei xiang

Cross-Cloud Migration of Container Persistent Data Huawei: xiang xinyong 99 Cloud: zhu rong

Agenda 1. What is Container Persistent Data? 2. When do we need Container Persistent Data? 3. When do we migrate Container Persistent Data? 4. What can be used to migrate Container Persistent Data? 5. How to migrate Container Persistent Data cross-cloud? 6. Demo 7. What’s next?

What is Container Persistent Data? Actually containers are stateless, only data is stateful. Stateless Containers When a container is removed, the data is lost. Ephemeral Data Persistent Data When a container is removed, the data is still available.

Container Data Example Container Union File System Ephemeral Data Container Volume Persistent Data ● For Docker, Union File System uses Docker Storage Driver. . ● Docker Volume uses Docker Volume Driver.

Container Union File System

Container Volume ● Volumes are not controlled by the storage driver. ● Volumes are initialized when a container is created. If the container’s base image contains data at the specified mount point, that existing data is copied into the new volume upon volume initialization. ● Volumes can be shared and reused among containers. ● Changes to a volume are made directly. ● Changes to a volume will not be included when you update an image. ● Volumes persist even if the container itself is deleted.

When do we need Container Persistent Data? Containers Persistent Data Audit Logs Database Data . . .

When do we migrate Container Persistent Data? ● Business change -- acquisition and merger of business unit. ● Workload balance -- improve efficiency, performance and scalability of apps. ● Hardware/System upgrade -- adapt new changes of technology, practice … ● Cost cutting -- cut down the cost of data movement. ● Fault -- networking, disk … do not work well. ● ……

What can be used to migrate Container Persistent Data? ● Data Volume Container If you share some continuously updated data to the container, the simplest way is use data volume container, the data volume container is a common container, providing data for other container using the mount. ● Flocker Docker specify a particular volume driver to perform the actual work. This volume can be mounted when a container is created. Flocker is one kind of third-party Docker Volume Driver for Data Volume.

Data Volume Container ● Create Data Volume Container(eg: Postgres) sudo docker run -i -t -v /dbdata --name dbdata training/postgres /bin/bash ● Refer to Data Volume Container sudo docker run -i -t --volumes-from dbdata --name db 1 training/postgres /bin/bash ● Backup Data Volume Container sudo docker run --volumes-from dbdata -v $(pwd): /backup training/postgres tar cvf /backup. tar /dbdata ● Prepare new Data Volume Container sudo docker run -v /dbdata --name dbdata 2 training/postgres /bin/bash ● Restore Data Volume Container sudo docker run --volumes-from dbdata 2 -v $(pwd): /backup training/postgres tar xvf /backup. tar

Flocker Image is ref from Cluster. HQ Flocker provides migration containers and persistent volume features between hosts.

Container Persistent Data integrated with Open. Stack ● Magnum ● Zun ● Cinder ● Fuxi ● …….

How to migrate Container Persistent Data cross-cloud?

What’s Karbor? ● Karbor is an Open. Stack official project, and it’s former name is Smaug. ● The koala is an arboreal herbivorous marsupial native to Australia, and Karbor is koala's indigenous name. ● Karbor deals with protecting Data and Meta. Data that comprises an Open. Stack-deployed Application. ● Karbor is designed to provide a high-level framework to integrated with plugins.

Karbor API What can be protected? Protectabl e How to protect? Protection Plan Protection Provider Where to protect? What was protected? Protection Plugins & Bank (Admin) Checkpoint When To protect? How to restore? Scheduled Operations Restoration

Karbor Architecture v API Service Trigger API Protectable API Scheduled Operation API Plan API Providers API Checkpoint API Restoration API DB RPC Scheduled Operation Workflow RPC Providers RPC Triggers Trigger Engine Time Workflow Engine Providers Manager RPC Checkpoints RPC Protectable RPC Restoration Protectable Resource Type Restore Resource Plugin Event Protection Plugin Operation Engine Service Bank Plugin Protection Service Checkpoint

Migrate Container Persistent Data cross-cloud using Karbor Open. Stack Cloud #2 Open. Stack Cloud #1 Hosts Containers Karbor Bank Checkpoint ……. Volumes Migrate Restore Volumes Database Karbor

Environment exists in Open. Stack Cloud #1 Hosts Volumes Containers Kubernetes Docker Database

Migrate Persistent Data into Bank by Karbor Open. Stack Cloud #1 Karbor Cinder Volume Database Migrate Bank Checkpoint Swift Bank

Deploy environment in Open. Stack Cloud #2 Magnum Heat Nova Cinder Host VM Kubernetes Volume Docker

Restore Persistent Data from Bank by Karbor Open. Stack Cloud #2 Karbor Cinder Volume Restore Bank Checkpoint

Deploy database containers in Open. Stack Cloud #2 Host VM Containers Kubernetes Docker Mount Volume Database

Check whether new database containers work?

Demo

What’s next?

Data Migration as a Service?

All Resources could be migrated in Open. Stack Cloud #1 Cloud #2 Container VM VM Volume Share Migration Share Image …….

Proposed Migration Work. Flow Find Source Entities Migration Finished Specify Migration Target Create Migration Group Create Migration Task Launch Migration

Discuss more in details ● Karbor Design Session： Wed 26 5: 05 pm-5: 45 pm AC Hotel - P 3 - Montjuic Thu 27 4: 40 pm-5: 20 pm CCIB - Centre de Convencions Internacional de Barcelona - P 1 - Room 127 Fri 28 9: 00 am-9: 40 am CCIB - Centre de Convencions Internacional de Barcelona - P 1 - Room 123 Fri 28 9: 50 am-10: 30 am CCIB - Centre de Convencions Internacional de Barcelona - P 1 - Room 123 Fri 28 11: 00 am-11: 40 am CCIB - Centre de Convencions Internacional de Barcelona - P 1 - Room 123 Fri 28 11: 50 am-12: 30 pm CCIB - Centre de Convencions Internacional de Barcelona - P 1 - Room 123 ● IRC：#openstack-karbor

Thank you!