1 Cinder OnBoarding Education Jay Bryant Lenovo April

1

Cinder On-Boarding Education Jay Bryant (Lenovo) April 30, 2019 - Denver

Agenda ● ● ● ● 3 Cinder’s Repos Brief overview of Cinder architecture/organization Cinder’s team Pointers to Upstream Institute education Applying Open. Stack Process Cinder’s code Testing changes to Cinder

Cinder Repos ● Main Cinder repository: ○ https: //github. com/openstack/cinder ● Cinderclient: ○ https: //github. com/openstack/python-cinderclient ● os-brick shared library: ○ https: //github. com/openstack/os-brick ● Brick Cinderclient for Standalone Cinder ○ https: //github. com/openstack/python-brick-cinderclient-ext ● cinderlib ○ https: //github. com/openstack/cinderlib ● Cinder Specs: ○ https: //github. com/openstack/cinder-specs 4

Cinder Overview ● Provides persistent block storage resources for an Open. Stack cloud. ● Supports multiple backends. Approximately 60 volume drivers in tree. 5

Vendor Backends ● Drivers from many different vendors ● Different protocols used depending on the driver ○ i. SCSI ○ Fibre Channel ○ RBD ○ Remote. FS ○ Etc. 6

Backend Drivers ●Datacore (FC, i. SCSI) ●Datera (i. SCSI) ●Dell/EMC PS Series (i. SCSI) ●Dell/EMC Scale. IO (scaleio) ●Dell/EMC Unity. Driver (FC, i. SCSI) ●Dell/EMC VMAX (FC, i. SCSI) ●Dell/EMC VNX (i. SCSI/FC) ●Dell/EMC Xtrem. IO (i. SCSI/FC) ●DRBD (DRBD/i. SCSI) ●Fujitsu ETERNUS (i. SCSI/FC) ●Hedvig (i. SCSI) ●HPE 3 PAR (i. SCSI/FC) ●HPE Left. Hand (i. SCSI) ●HPE MSA (i. SCSI/FC) ●Huawei Fusion. Storage DSWare (i. SCSI) ●IBM Flashsystem (i. SCSI/FC) ●IBM GPFS (GPFS/NFS) 7 ●IBM Storage (i. SCSI/FC) ●IBM Storwize SVC (i. SCSI/FC) ●Infinidat Infinibox (FC) ●Inspur (i. SCSI) ●Inspur AS 13000 (i. SCSI) ●Kaminario (i. SCSI) ●Lenovo (i. SCSI/FC) ●LVM (i. SCSI) – Reference* ●NEC M-Series (FC/i. SCSI) ●Net. App C-Mode (i. SCSI/FC/NFS) ●Net. App ONTAP (i. SCSI/NFS/FC) ●Nexenta (i. SCSI/NFS) ●Nexenta Edge (i. SCSI/NBD) ●NFS – Reference ●Nimble Storage (i. SCSI/FC) ●Prophet. Stor (i. SCSI/FC) ●Pure Storage (i. SCSI/FC) ●Qnap (i. SCSI) ●Quobyte (quobyte) ●RBD (Ceph) - Reference ●Sheepdog (sheepdog) ●Solid. Fire (i. SCSI) ●Storpool (storpool) ●Storage Performance Developer Kit (SPDK) (NVMe-o. F) ●Synology (i. SCSI) ●Tintri (NFS) ●Veritas CNFS (NFS) ●Veritas Hyper. Scale ●Virtuozzo Storage (NFS) ●VMWare (VMDK) ●Windows (SMB) ●Zadara (i. SCSI) ●ZFS (i. SCSI/NFS) Note: Supported drivers list. Full list may be seen here: https: //docs. openstack. org/cinder/latest/drivers. html

Volume Attachment Cinder Nova Note that i. SCSI is just an example – several additional protocols are supported (e. g. , FC, NFS) VM instance LVM /dev/vda i. SCSI target Legend Persistent volume control Persistent volume data KVM i. SCSI initiator 8

Cinder’s Team ● PTL ○ Jay Bryant (jungleboyj) ● Core Team ○ Sean Mc. Ginnis (smcginnis) ○ Walt Boring (hemna) ○ Tommy. Like Hu (tommylikehu) ○ Gorka Eguileor (geguileo) ○ John Griffith (jgriffith) ○ Eric Harney (eharney) ○ Ivan Kolodyazhny (e 0 ne) ○ Xing Yang (Xyang) ○ Brian Rosmaita (rosmaita) ○ Yikun Jiang (yikunkero) ○ Rajat Dhasmana (whoami-rajat) 9

Upstream Institute ● https: //docs. openstack. org/upstream-training/ ● Day and a half education session to help people new to Open. Stack ● Lead by Open. Stack Foundation members and volunteers ● Miss the Denver session … No problem, join us in Shanghai! 10

Upstream Institute - Key Info ● VM image to get your development started ● Open. Stack account setup ○ https: //docs. openstack. org/upstream-training/workflow-reg-andaccounts. html ● Launchpad/Storyboard introduction ○ https: //docs. openstack. org/upstream-training/workflow-task-tracking. html ● Workflow introduction ○ https: //docs. openstack. org/upstream-training/workflow-trainingcontribution-process. html ● Introduction to gerrit and code reviews ○ https: //docs. openstack. org/upstream-training/workflow-gerrit. html ○ https: //docs. openstack. org/upstream-training/workflow-reviewing. html 11

Cinder in Open. Stack’s Processes 12

Cinder Bugs and Blueprints ● Still currently using Launchpad ● https: //launchpad. net/cinder 13

Bug Tags ● ● ● ● 14 low-hanging-fruit doc i 18 n security ops <SERIES>-rc-potential <SERIES>-backport-potential driver/drivers gate-failure ceph cinder backup-service create-volume-from-snapshot

Cinder Specs ● ● ● 15 https: //github. com/openstack/cinder-specs We request specs for more complicated changes Organized by release (specs/<release>) Template from which to start: specs/template. rst Before pushing your spec run ‘tox -e docs, py 27, pep 8’ ○ Look for any errors and please correct before pushing for review

IRC and Cinder Meetings ● #openstack-cinder ○ Please join us for all things Cinder ○ Try @? for some (╯°□°)╯� ┻━┻ fun ○ (not working on Cinder but in openstack-meeting ● Weekly Meeting ○ Wednesdays at 16: 00 UTC ○ #openstack-meeting ○ https: //wiki. openstack. org/wiki/Cinder. Meetings ○ Add agenda items with your IRC nickname to the meeting agenda/notes etherpad 16

Typical Development Cycle ● Stein release schedule ○ https: //releases. openstack. org/stein/schedule. html ● Milestone 2 ○ Spec freeze ○ New driver merge deadline ● Milestone 3 ○ New feature freeze ○ Move focus to testing and bug fixes ○ Need 3 rd party CI’s functional to avoid addition of unsupported flag ● RC ○ Bug fixes only ● Some flexibility around dates allowed with core team and PTL’s discretion 17

3 rd Party CI 18 ● 3 rd Party CI is required for all Cinder drivers ○ https: //wiki. openstack. org/wiki/Cinder/tested-3 rd. Party-drivers ● Purpose ○ To ensure that a vendor’s driver functions properly as changes merge to the Cinder tree ○ To show that patches against a vendor’s driver work ● Policy (Proposed) ○ 3 rd Party CI must run successfully against a weekly job executed to test that CI’s are reporting ○ 3 rd Party CI must fail a job submitted that should fail all 3 rd Party CI testing. ○ A vendor CI that doesn’t meet these requirements during a release cycle is marked ‘Unsupported’ and consumers have to update their configs to indicate they wish to run an unsupported driver ○ Continued non-compliance results in driver removal in the next release.

3 rd Party CI - Review Implications ● If a patch fails all CI’s, something is wrong ● Patches against a driver must pass the vendor’s CI before being merged 19

About Reviews ● We need your help! ● Good way to get involved with Cinder ○ Learn the code and the project organization ○ Don’t need to know how Cinder works to catch logic errors and typos ● Gets you visibility to the Cinder team ● Review inboxes: ○ Cinder review inbox ○ os-brick review inbox ○ python-brick-cinderclient-ex review inbox ● Track how we are doing: ○ http: //cinderstats. ivehearditbothways. com/ 20

Cinder Code Layout - Top Level ● api-ref -- Source documentation for v 1, 2 and 3 Cinder API ● cinder -- Cinder’s source directory with high level implementations at the top level (i. e. exceptions, i 18 n, policy, utils, etc. ) ● doc -- Developer documentation that is published to the Open. Stack website ● etc -- Config files that are not automatically generated ● rally-jobs -- Rally tasks and plugins to be run by Open. Stack CI ● releasenotes -- Contains release notes for patches ● tools -- Config generation script, driver list generation, etc. 21

Cinder Code Layout - /cinder ● ● ● ● ● 22 api -- v 1, 2, 3 and API extension implementation backup -- Backup driver API and implementations brick -- Legacy brick directory, mostly migrated to os-brick library cmd -- Starter scripts for Cinder’s processes (api, scheduler, volume, etc. ) common -- Shared files for config, sqlalchemy, etc. compute -- Source for interfacing with Nova config -- Source files for config file generator consistencygroup -- Consistency Group API implementation db -- sqlalchemy migration scripts and DB API implementation

Cinder Code Layout - /cinder (cont. ) ● ● ● ● 23 group -- Generic Group API implementation hacking -- Hacking check implementation image -- Source for interfacing with Glance interface -- Base interface definition for driver validation and reference keymgr -- Simple security key implementation locale -- Translation files for log messages message -- User message API implementation objects -- Implementation of Cinder’s Oslo Versioned Objects scheduler -- Scheduling option implementations tests -- Compliance, functional, tempest and unit test implementation transfer -- API implementation of volume transfer volume -- Implementation of API and drivers for volumes wsgi -- Cinder Web Server Gateway Interface implementation zonemanager -- Fibre Channel Zonemanager implementation

Release Notes ● Developer’s way to communicate to the end user ● Most important for changes that impact Cinder’s functionality ● Documents: ○ Security issue fixes ○ New features ○ Upgrade notes ○ Known issues ○ Deprecation notes ● Not required for every change, but important for anything that directly impacts packagers or users 24

Upgrade Checks ● Helps operators to ensure that their cloud is compatible with the upgrade ● Changes that may impact an upgrade should include a check ○ Option removal ○ Driver removal ○ Database changes ○ Etc. ● Not required for every change, but good for catching those cases where one might worry ‘what if the environment were set up this way? ’ 25

Tox Tests ● Tox is a test runner used by most Open. Stack projects ● Many of the same test targets, but a few specific to Cinder ● Before submitting a patch, best to at least run py 27 and pep 8 targets (tox -e py 27, fast 8) 26 py 27, py 35 Runs unit tests using either Python 2. 7 or 3. 5 genopts Must be run when adding any new config options to regenerate opts. py file used to create the sample config pep 8, fast 8 Runs code checks to make sure it conforms to coding style and hacking checks to look for specific conditions releasenotes Validates new release notes (must commit first!) compliance Useful for driver developers, will validate all required driver interfaces are implemented

Q&A Twitter & IRC: Jungleboy. J E-mail: jsbryant@electronicjungle. net