Encrypt Your Volumes with Barbican Agenda Introduction About

Encrypt Your Volumes with Barbican

Agenda • • • Introduction About On. Ramp Why Encrypt? Key Managers How it Works Our Encryption Story Gotchas & Limitations Demo Q&A

Meet the Speaker • Previously Built and Managed Lab Environments for VMware Certification Training (VCP 5/6) • Experience with Open. Stack: - Started with Icehouse on Ubuntu (manually) - Juno on Canonical Maa. S + Juju - Liberty on Cent. OS / RDO - Newton on Platform 9 Duncan Wannamaker Open. Stack Engineer On. Ramp • Open. Stack Engineer for On. Ramp • Deployed Barbican and support for Encrypted Volumes for our Open. Stack Based Public Cloud

But, Open. Stack is Far From Simple, Right?

Who Is On. Ramp? On. Ramp is a HITRUST-certified data center services company specializing in high security and compliant hybrid hosting.

On. Ramp’s Virtual Private Cloud Offers the ease-of-use of a public cloud, with the security of a private cloud. ■ Built on Open. Stack® ■ HITRUST-certified ■ Control costs with capped maximum resource usage ■ Open-source APIs enable simple migrations and eliminate vendor lock-in ■ Hardened Linux and Windows Images ■ Retain full control of your private networks, virtual machines, and storage

Why Encrypt? • Protect Data against Leaks - Personal Health Information (PHI) - Credit Card Payment Data (PCI) - Intellectual Property • Compliance Requirement - HIPAA requires encryption for data at rest and in-transit for PHI - PCI also requires encryption for data at rest and in-transit for cardholder data - In shared hosting environments, each Tenant must only have access to their own stuff Ø Per-Tenant or Per-Volume encryption keys facilitate this

Why Use a Key Manager? • Security Best Practice - You don’t leave keys in your car - You shouldn’t leave your keys next to your encrypted data • Compliance Requirement - PCI-DSS Requirements: Ø Store keys in the fewest possible locations Ø Store secret and private keys used to encrypt/decrypt cardholder data using a key encryption key at least as strong as the encryption key itself - HIPAA- HITRUST Requirements: Ø Keys shall be stored separately from encrypted data Ø Key manager systems should be physically protected by fewest number of custodians necessary

What is Barbican? Provides: Does Not Provide: • Re. ST API for Secrets Management • Graphical User Interface • Pluggable Backends • Key Splitting for Secure Import/Export Plain Text Keys using multiple Key Custodians - Simple Crypto - PKCS#11 and KMIP (HSM) • Integration with Nova, Cinder, and • Generation of X. 509 Certificates (Since Pike) Swift, Neutron, Heat, and many other OS projects • Volume Encryption • Integration with Key. Stone for Auth and RBAC • Built to Scale

What about Volume Encryption? • LUKS - Linux Unified Key Setup - Allows for multiple user keys or passwords per volume Ø Master Key always stays the same - Supports CPU Hardware Acceleration (it’s fast!) - Uses Crypt. Setup and DM-CRYPT Ø Decrypts full volume to a Local Block Device Ø Protects i. SCSI attached volumes Ø Can also protect ephemeral storage if using LVM • Queens = QEMU Native LUKS support - QEMU 2. 6 and Lib. Virt 2. 2 introduce native LUKS support

Transparent Encryption in Nova and Cinder Thanks to some fantastic work done by Johns Hopkins Applied Physics Laboratory and others… • Nova and Cinder Integration exists with Barbican • Volume Decryption Happens on the Hypervisor instead of within the Guest OS - No Agent Required Works with Any Operating System Works with Bootable Volumes Protects Data at Rest and In-Transit to your hypervisor Every volume is protected by it’s own unique key

How it Works: Creating an Encrypted Volume 1. User Gets a token from Keystone 2. User Asks Cinder to create volume (using token) 3. Cinder verifies the user token from keystone 4. Cinder then asks Barbican for a key 5. Barbican checks the Cinder Token against keystone 6. Barbican creates the secret and returns secret HREF to Cinder 7. Cinder stores the returned secret HREF into the volume metadata

How it Works: Mounting an Encrypted Volume Nova Keystone Receive User Mount Request Validate Request Cinder Barbican 2. 3. Validate Request Fetch Secret HREF from Cinder Validate Token Receive Secret HREF 4. 5. Return Secret HREF and Attach Volume 6. Validate Requestor Fetch Secret from Barbican Validate Token Mount Volume using Secret 1. Return Secret 7. 8. 9. Nova receives request to mount volume with token Nova Validates User Token Nova Fetches the Secret HREF from Cinder Validates Nova Token Cinder returns Secret HREF to Nova Fetches Secret from Barbican Validates Nova Token Barbican Returns Secret Nova Mounts Volume using Secret

On. Ramp Encryption Story • Followed Documentation - Wait, these docs are terrible! Ø Thankfully, Barbican Devs on IRC are very helpful! • Key. Stone: - Added Endpoints to Key. Stone - Added Barbican Service User - Added Creator Role • Installed the Barbican-API: - Configured Barbican’s database server - Configured Barbican - Barbican CLI Access Works!

On. Ramp Encryption Story • Cinder Integration Issues: - Blank Encrypted Volumes were created successfully - But they could not be created from an image • Nova Integration Issues: - Nova was unable to attach any encrypted volumes - It wasn’t even trying to talk to Barbican

Issue 1: Key orders not getting to Barbican Cloud-Hosted cinder-api On-Premise SSH VPN cinder-volume Key Orders Secret HREF: null Mount Request LAN key request no key 0000… for key 0000… href = all zeros! Cinder DB On-Premise Create Mount Volume Failed Encrypted Volume barbican

Issue 1 Fix: Key orders not getting to Barbican Cloud-Hosted cinder-api On-Premise SSH VPN Key Order valid href Secret HREF Cinder DB Secret HREF: valid Mount Request cinder-volume forwarder Create Mount Volume Encrypted Volume On-Premise LAN Key Order request secret Secret HREF return secret barbican

Issue 2: Nova Not Talking to Barbican • Nova error was mentioning a Fixed Key not being defined • Conf. Key. Manager (Nova Default for Volume Encryption) • Single fixed key for all volumes • Used for testing to substitute in for a real key manager, such as Barbican! Not for production. • Setting the api_class in nova. conf would always use Conf. Key. Manager and ignore the setting to use barbican • Submitted Bug: • https: //bugs. launchpad. net/nova/+bug/1704875 • Fixed in Pike or newer! • Manual Fix for older releases is to comment out lines 27 -31 in nova/keymgr/__init__. py

Some Gotchas using Encrypted Volumes • Live Migration does not work and is dangerous - Volume mounts without decryption on new host which will cause corruption if this is an active file system - Fixed in Queens! Ø Previous releases use symlinks on hypervisor Ø New Method uses QEMU Native LUKS support • Barbican Doesn’t Start after Reboot (Cent. OS Specific) - RDO Packaging Issue - Create /etc/tmpfiles. d/barbican. conf: Ø d /var/run/barbican 0755 barbican – - Bug report submitted…

Some Limitations using Encrypted Volumes • No mechanism to rotate volume encryption keys - Manual process of creating a new volume and copying over the contents - LUKs supports multiple user keys, so the capability is there • No UI for key management in horizon - for securely exporting and importing split-keys - for managing key ACL’s - for managing key expiration and revocation

Tips for Running in Production • Make sure your key manager and database are secured in a locked cabinet with limited physical access • Use a private barbican instance not accessible to tenants • Automated database backups • Use a highly available database cluster such as Galera • Use multiple barbican-api nodes behind a load balancer • Use SSL to protect key requests in-transit to hypervisors

Demo

Questions?

Barbican Po. C Guide and Resources • Step by step instructions in Git. Hub: https: //github. com/dwannamaker-onr/openstack-queens-barbican-guide • Video walkthrough available: - You. Tube Search: “Queens Barbican Po. C” Ø Installing Packstack for Queens (Part 1) Ø Installing the Barbican-API (Part 2) • IRC Channel: - #openstack-barbican on Free. Node

Thank you! Stay In Touch: duncan@randomhack. com @randomhack 888. 667. 2660 sales@onramp. com www. onr. com www. randomhack. com