STO 3305 BES Replicating VMware VVols A technical

[STO 3305 BES] Replicating VMware VVols: A technical deep dive into VVol array based replication in v. Sphere 6. 5 Claudio Calisto – Storage Solutions Architect Nick Dyer – Principal SE, Nimble

A Brief History of 3 PAR, Nimble & VMware VVols Design partnership between HPE and VMware Virtual Volumes VMworld Nimble Tech Preview VVol Beta VVol introduced with HPE as 1 of 5 original design partners 3 PAR 1 of 3 partners ready on Beta Day 1 Aug 2011 Jul 2014 VVol 2. 0 GA with v. Sphere 6. 5 Nimble tech preview released 3 PAR & Nimble ready on Day 1 Jun 2015 Nov 2016 May 2012 Mar 2015 Mar 2016 3 PAR Reference Platform VVol GA with v. Sphere 6. 0 Nimble GA 3 PAR selected as the FC reference platform 3 PAR 1 of 4 partners ready Nimble releases VVol support Today Continued development 3 PAR, Nimble and VMware continue working to develop and refine VVols

Why did VMware create a new storage architecture? Todays exercise: Challenges in external storage architectures § LUN Centric – storage preallocated into silos § Difficult Reclamation – space reclamation is a manual process § No Visibility – cannot see inside VMFS volume § Hardware Centric – must use vendor tools & plug-ins § Poor Efficiency – typically overprovisioning LUNs § Long Provisioning – time consuming manual requests § Increased Administration – must always go to storage admin § Data Services – array data services not aligned with VMs Key take-away: There are many hidden challenges with traditional external storage architectures in v. Sphere which decrease efficiency and increase management

What are VMware VVols? – New v. Sphere storage architecture to replace VMFS/NFS – Standard (VASA) for all storage vendors to adhere to – Enables v. Sphere to write VMs natively to storage arrays – Common storage management tasks are automated – Designed to be dynamic, eliminates LUN provisioning – Storage array features can be applied to individual VMs Key take-away: VVols provides automated and dynamic management of storage resources and enables storage arrays to interact directly with VMs

VVols VMFS How VVols transforms storage in v. Sphere LUN-Centric Static Complex Time-consuming Siloed storage pools, array services aligned to LUNs Pre-allocated storage, over- provisioned resources Complicated management using vendor specific tools Longer provisioning, manual space reclamation VM-Centric Dynamic Simple Effortless No siloed storage pools, array services aligned to VMs Dynamically allocated storage, using only what is needed Simplified management using v. Sphere interfaces Instant provisioning, automatic space reclamation

File Block What changes between file and block protocols with VVols Host adapter I/O transport Host present File system VM storage Storage visibility i. SCSI initiator (sw/hw) or HBA Network or fabric Storage LUNs container v. Sphere managed native (VMFS) VVols VMDK files Data store VM level Level Network Storage Mount point container Array v. Sphere managed native (NFS) VVols VMDK files VM level NFS Client

Overview of VVols Storage Architecture – Protocol Endpoint: Logical I/O proxy that serves as the data path between ESXi hosts to VMs and their respective VVols ESXi hosts v. Center Server VM VM VM SPBM – VASA Provider: Software component that mediates out-of- band communication for VVols traffic between v. Center Server, ESXi hosts & storage array Control Path Data Path – Storage Container: Pool of raw storage capacity that – Virtual Volume (VVol): Container that encapsulates VM files, virtual disks and their derivatives – Storage Profile: Set of rules that define storage requirements for VMs based on capabilities provided by storage array (same as VSAN) Protocol Endpoint VVOLs becomes a logical grouping of VVols, seen as a virtual datastore by ESXi hosts VASA Provider Storage Container Storage Array

Storage Policy-Based Management 1. Storage Array advertises capabilities via VASA Provider 7. 2 K RAID 1 Encrypt Cache 15 K RAID 6 Replica Thin SSD RAID 5 Qo. S Dedupe 4. SPBM provisions VM on appropriate storage as defined by policy and maintains compliance 2. Storage Policies created in v. Sphere and assigned array capabilities Policy 1 Policy 2 Policy 3 7. 2 K 15 K SSD RAID 5 RAID 6 RAID 1 Thin Encrypt Cache Replica Thin Dedupe Qo. S VM VM VM 3. VMs assigned a Storage Policy based on requirements and SLAs

Top Reasons Customers will want to Start Using VVols Now 1. Don’t have to go all in 2. Get early experience 3. Get your disk space back 4. Available in all v. Sphere editions 5. Let the array do the heavy lifting 6. Start using SPBM now 7. Snapshots don’t suck anymore 8. Easier for IT Generalists 9. One architecture to rule them all 1. Start You don’t to go 2. 3. 4. 5. Get Available Let early your array VVol in all doaall 7. Snapshots don’t 8. Easier for IT 6. using SPBM 9. One architecture 10. Thethe VM isdisk now in with VVols experience space v. Sphere the heavy back editions lifting suck anymore Generalists now to rule them all unit of storage You can use VVols alongside It’s An array an more wait architecture is over-provisioning until a purpose the last not built ahours I/O or No longer have toaget wait Don’t need todarkness be storage get left out, started …and init’s the bind Because all about the VM, ofmore VMFS, easy migration minute. manual feature, engine, How space array nothing long reclamation, to did license, you are for I/O-the intensive snapshot admin, fully manage storage using same SPBM that NFS, i. SCSI, Fiber nothem LUNs –features the VM is using Storage v. Motion, take wait keep will more to your powerful switch VMFS from and thin one ESX has as day better toyour deletions toarray commit, plus from within v. Sphere VSAN users have been Channel, who cares? VVols now areplace first class citizen and yourhas time it visibility ESXi possibly visibility or from into automatically storage v. Sphere Client backups will complete faster enjoying provides awith unified storage array VM-level to Web Client? resources than v. Sphere has architecture across protocols

VVol Replication Deep Dive

Introduction – Array replication support via SPBM introduced in v. Sphere 6. 5 (VASA 3. 0) – Nimble supported replication in v. Sphere 6. 0 – Nimble and 3 PAR first to complete VVol replication implementations (before merger) – Replication done at the VVol level (VM) not the datastore level like SRM (LUN) – Replication Groups automatically created on array contain VVol objects – Designed to be managed in v. Sphere, not on array side Nimble and 3 PAR were both VMware design partners on VVols 2. 0

Components of VVol Replication Fault Domains [new] – Something that fails as a whole (3 PAR = storage array, Nimble = Nimble group) Replication Groups [new] – Replicate VVol-based VMs between fault domains – Groups maintain consistent point[s]-in-time – 3 PAR - Only maintains most recent point in time, which is no older than the most recent RPO – Nimble – Uses Volume Collections [most recent point for RGs] - plugin (workflows for replication) – RPOs can be “stretched” when adding VMs or VVols to a replication group – Groups are in a Source, Target, In. Test or Failed. Over state – Terms Source/Primary/Protected are interchangeable – Terms Target/Secondary/Recovery are interchangeable

VVol Replication diagram v. Sphere Storage Policy (SPBM) “Local” Array (Fault Domain) VVols Storage Container Replication Group “Remote” Array (Fault Domain)

Preparing for VVol Replication v. Sphere 3 PAR Nimble • v. Sphere 6. 5 required • 3 PAR OS 3. 3. 1 required • v. Sphere 6. 0 or 6. 5 • Register VASA Provider • Remote Copy license • Mount Storage Containers (VVol datastores) at primary and secondary sites • Connect arrays via FC or i. SCSI • Nimble. OS 3. 6. 0 required for VASA 3. 0 • Define Storage Policy with replication rules (Components) • Create Storage Containers on primary and secondary arrays • Define remote targets for Remote Copy • No licenses required • Configure source and destination as replication partners • Create Folders (storage containers) on source and destination

Creating Nimble Folder / Storage Container – Nimble folders are used for general organization – Setting VVol management type makes it a Storage Container – Must set a capacity limit – Can optionally set a folder-level performance limit

Replication Storage Policies Summary – Granular (per VM/Virtual Disk) – Empowers v. Sphere admin – No need to coordinate with Storage admin – No need to involve Storage admin during disaster recovery – Storage admin may need to do some clean-up after a true disaster – New Component feature for SPBM allows Components to be defined once and re-used – Replication Components contain rules and constraints related to array replication – Components are attached to policies which are attached to VMs – SPBM maintains compliance to ensure VM resides on storage that can meet policy definition

Example Storage Policy [3 PAR] Target array & Storage Container Target & Snapshot CPG drive tiers Need only specify one replication constraint to enable VVol replication Remote Copy mode (Periodic only) Desired RPO Minutes (5 min) Where possible, other VVol capabilities (i. e. deduplication) are mirrored at remote site if remote array/CPG allows for it

Example Storage Policy [Nimble] Protection schedule by minute, hour, daily or weekly Can set local and remote snapshot retention policies Choose replication partner and frequency Option to delete replicas from partner when source deleted

Creating a new VM with replication Choose an existing Replication Group or Automatic

Changing a VM’s Replication Policy Select VM Policies, Edit VM Storage Policies

Changing a VM’s Replication Policy Selecting a Storage Policy with Replication Constraints

Changing a VM’s Replication Policy Selecting either a common or per storage object Replication Group

Replication Groups on the Array s 932 cli% showvvolvm -sc San. Jose. Container -rcopy ------(MB)-----VM_Name Guest. OS VM_State Num_vv Physical Logical VMSan. Jose other 26 x. Linux 64 Guest Unbound 2 10290 5120 Tiny. VM 2 other 26 x. Linux 64 Guest Unbound 2 9522 5120 Tiny. VM 1 other 26 x. Linux 64 Guest Unbound 2 10290 5120 Tiny. VM 3 other 26 x. Linux 64 Guest Unbound 3 7986 6144 VMFremont other 26 x. Linux 64 Guest Unbound 2 8754 5120 -------------------------------5 total 11 46842 26624 Rcopy. Status Rcopy. Group Output continued below Rcopy. Role Rcopy. Last. Sync. Time Synced VMSan. Josgrp_3 c 12 e 41 Primary 2017 -07 -06 16: 20: 00 PDT Synced Tiny. VM 2 grp_80 ed 051 Primary 2017 -07 -06 16: 22: 00 PDT None NA Synced Tiny. VM 3 grp_1 a 3 d 5 ba. r 99931 Secondary 2017 -07 -06 16: 20: 10 PDT Synced VMFremongrp_ca 4027 f. r 99931 Secondary 2017 -07 -06 16: 21: 10 PDT ------------------------------------

3 PAR SSMC User Interface - VVol replication info Remote Copy Group Remote Copy Role Remote Copy Status

Nimble Replication Groups

Disaster Recovery with VVols 26

v. Sphere Replication Components DR Orchestrator v. Center ESXi Protocol Endpoint v. Center (Power. CLI) ESXi VASA Provider Protocol Endpoint ESXi VASA Provider Remote Copy Protected Site VVol Storage Container Recovery Site

Types of Disaster Recovery operations Planned Failover Unplanned Failover Test Failover • Controlled • Uncontrolled • Controlled • Typically used for disaster avoidance, planned maintenance or relocating VMs • Typically loss of power or hardware failure • Typically used to validate VM recoverability • Can be per VM or all VMs • Primary and recovery site reverse roles • Not usually a per VM event, all VMs recovered • Non-impactful • Replica VMs cloned and made visible to v. Sphere at recovery site

Recovery and Cloning – Nimble v. Center Plugin – Snapshot based recovery – Choose from all available snapshots – Local VM Recovery – In-place restore of VM – Cloned as a new VM – Local Virtual Disk Recovery – In-place restore of virtual disk – Cloned as a new disk to same VM – Cloned as a new disk to different VM – Remote VM Recovery – Cloned as new VM at remote site – No need to stop replication

Planned Failover Workflow with VVols Before Failover v. Sphere API v. Center ESXi VM 1 v. Center VM 3 ESXi Replica VVol Storage Container Source Replication Group v. Sphere API VM 2 Source VVol Array Power. CLI C 1 C 2 D 1 D 2 Storage Container C 1 C 2 D 1 D 2 Sn 1 Sw 2 C 3 Local Site D 3 Target Replication Group C 3 D 3 Remote Site Array

Planned Failover Workflow with VVols Discover VM-to-group relationship (Get-Spbm. Replication. Group/Pair) v. Sphere API v. Center ESXi VM 1 Power. CLI v. Sphere API v. Center VM 2 Array VM 3 C 1 C 2 D 1 D 2 Sn 1 Sw 2 C 3 Local Site D 3 C 3 D 3 Remote Site ESXi Array

Planned Failover Workflow with VVols Power down source VMs (stop-VM) v. Sphere API v. Center ESXi VM 1 Power. CLI v. Sphere API v. Center VM 2 Array VM 3 C 1 C 2 D 1 D 2 Sn 1 Sw 2 C 3 Local Site D 3 C 3 D 3 Remote Site ESXi Array

Planned Failover Workflow with VVols Perform final sync (Sync-Spbm. Replication. Group) v. Sphere API v. Center ESXi Array VM 1 Power. CLI v. Sphere API v. Center VM 2 VM 3 C 1 C 2 D 1 D 2 Sn 1 C 3 Local Site Sn 1 D 3 C 3 D 3 Remote Site ESXi Array

Planned Failover Workflow with VVols Issue a planned failover (Start-Spbm. Replication. Failover) v. Sphere API v. Center ESXi Array VM 1 Power. CLI v. Sphere API v. Center VM 2 VM 3 C 1 C 2 D 1 D 2 Sn 1 Source Replication Group C 3 Local Site Sn 11 D 3 Failed-over Target Replication Group C 3 D 3 Remote Site ESXi Array

Planned Failover Workflow with VVols Register newly-failed-over VMs (New-VM) v. Sphere API v. Center ESXi Array VM 1 Power. CLI v. Sphere API VM 1 VM 2 v. Center VM 2 VM 3 C 1 C 2 D 1 D 2 Sn 1 Source Replication Group C 3 Local Site Sn 1 D 3 Failed-over Replication Group C 3 D 3 Remote Site ESXi Array

Planned Failover Workflow with VVols Apply an SPBM replication Policy to the VMs (Set-Spbm. Entity. Configuration) v. Sphere API v. Center ESXi Array VM 1 Power. CLI v. Sphere API VM 1 VM 2 Replication Policy VM 2 VM 3 C 1 C 2 D 1 D 2 Sn 1 Source Replication Group C 3 Local Site Sn 1 D 3 Failed-over Replication Group C 3 D 3 Remote Site v. Center ESXi Array

Planned Failover Workflow with VVols Unregister VMs at primary site (remove-VM) v. Sphere API v. Center ESXi Array VM 1 Power. CLI v. Sphere API VM 1 VM 2 v. Center VM 2 VM 3 C 1 C 2 D 1 D 2 Sn 1 Source Replication Group C 3 Local Site Sn 1 D 3 Failed-over Replication Group C 3 D 3 Remote Site ESXi Array

Planned Failover Workflow with VVols Power up VMs at Failed-Over site (start-VM) v. Sphere API v. Center v. Sphere API VM 1 ESXi Array Power. CLI v. Center VM 2 VM 3 C 1 C 2 D 1 D 2 Sn 1 Source Replication Group C 3 Local Site Array Sn 1 D 3 Failed-over Replication Group Sw 1 Sw 2 C 3 D 3 Remote Site ESXi

Planned Failover Workflow with VVols Power up VMs at Failed-Over site (Start-Spbm. Replication. Reverse) v. Sphere API v. Center v. Sphere API VM 1 ESXi Array Power. CLI v. Center VM 2 VM 3 C 1 C 2 D 1 D 2 Sn 1 Target Source Replication Group C 3 Local Site Array Sn 1 D 3 Source Failed-over Replication Group Sw 1 Sw 2 C 3 D 3 Remote Site ESXi

Demo time! 40

Switch to 3 PAR Replication unplanned failover demo Play STO 3305 BUS_Siebert_Unplanned. Failover. mp 4 (5: 11) 41

Switch to Nimble Replication demo Play STO 3305 BUS_Siebert_Nimble. Demo. mp 4 (2: 30) 42

Closing 43

The HPE 3 PAR & Nimble advantage with VVOLs Solid and mature Simple and reliable 6+ years of development VMware design partner Fibre Channel reference platform Internal VASA Provider No external failure points Zero step installation Innovative and efficient Rich and powerful Snapshots on different tiers Smallest capacity VM footprint Manage VVols folder by folder App level optimized storage policies Architectures optimized for VVols VM Recovery directly from v. Center

Call To Action – Find out more While at VMworld: Anytime: – Visit HPE Booth #D 301 to talk to our experts – Stalk us on Twitter – @Ericsiebert – @Julian_cates – @Nick_Dyer_ – Check out the VVols Hands-On Lab for some hands-on with Nimble VVols and replication – Attend VVols Partner Panel session ID # – Attend HPE in-booth VVol sessions – Download DR scripts from github – Bookmark Around The Storage Block blog – 3 PAR VVol Key docs: – 3 PAR VMware VVol Implementation Guide – 3 PAR VMware VVol Replication Guide – Nimble VVol Key docs: – VMware Integration Guide – VMware v. Sphere 6 Deployment Considerations

Thank You

Backup slides 47

Summarize and simplify this Two Primaries Window During Planned and Unplanned failover a window exists when you have two source/primary replication groups – During this time window, VMs changed as well as added/removed from replication groups, will require resolution when the groups are reversed. – With traditional storage, any changes at the original source datastore are completely wiped out – With VVols, it’s possible to reverse some of the actions taken at the original source site. For example: – A VM deleted at the primary is kept alive until the reverse-replication operation is performed. At which point, the VM is recovered, assuming it still exists at the new source site. – Adding a new VM to the original source replication group will not be lost, but instead, auto-dismissed from the group when the reverse operation occurs. – VM snapshots taken at the source replication group will be lost upon reversal. 48

Handling Conflicts with Two Primary Replication Groups Deleting a VM at the original source v. Sphere API v. Center ESXi Array VM 1 Power. CLI v. Sphere API VM 1 VM 2 v. Center VM 2 VM 3 C 1 C 2 D 1 D 2 Sn 1 Source Replication Group C 3 Local Site Sn 1 D 3 Failed-over Replication Group C 3 D 3 Remote Site ESXi Array

Handling Conflicts with Two Primary Replication Groups Adding a VM to the original source after fail-over v. Sphere API v. Center ESXi VM 1 C 1 D 1 Array VM 4 VM 2 C 4 D 4 C 3 Local Site v. Sphere API VM 1 v. Center VM 2 VM 3 C 2 C 1 C 2 D 1 D 2 Sn 1 Source Replication Group Power. CLI Sn 1 Failedover Replication Group D 3 C 3 D 3 Remote Site ESXi Array

Planned Failover Workflow with VVols In-conflict VVols are automatically removed from the group, but not destroyed v. Sphere API v. Center VM 4 ESXi C 1 Array D 1 C 4 D 4 Power. CLI v. Sphere API VM 1 v. Center VM 2 C 1 C 2 D 1 D 2 Sn 1 Target Source Replication Group Sn 1 C 4 D 4 C 3 Local Site VM 3 D 3 Source Failed-over Replication Group C 3 D 3 Remote Site ESXi Array

Benefits of VVols on Nimble Storage Embedded VASA Provider • No need to manage additional resources • Highly available • Automated registration of VP Automatic creation of PE and host access control management VASA Provider Management • Folders • Manage VVOLs folder by folder • Folders can grow and shrink dynamically Backup and Recovery • Replicate VM using array based replication • Recover VM using Nimble v. Center Plugin

Nimble VVol Implementation Storage Policy Based Management • New policy-based framework from VMware • Foundation of VMware’ Software Defined Datacenter control pane • Based on the VASA management protocol Nimble Policy Based Management • Built-in application abstractions • Pick from drop-down list - Exchange, SQL, VDI, Splunk, ESX etc. • Auto-selects optimal storage settings Virtual Volumes • Makes Nimble storage natively VM aware • Enables virtual disk level offload of data services • Snapshots, replication, encryption

Registering Nimble VASA Provider – Done through Nimble Web UI – Simply check the box 54

Additional SPBM Policy Options – Application Policy – Protection Schedule – Qo. S / Performance – Deduplication – Data Encryption – All-Flash

VVol Replication Objects VVol Storage Container Replication Group VVol Config, Data, RO/RW snapshots and k/v replicated. SWAP and storage policies not replicated Names of Storage Containers and CPGs shared between arrays Array A – a. k. a Fault Domain Array B

Point-in-Time Snapshot for Failover Test [3 PAR] VVol Snapshot relationship Replication Group C D C’ D’ VM 1 RO’ C D C’ D’ Data & K/V C VM 2 D RO VM 2 VVol Datastore RO VM 2 RO’

3 PAR VASA Replication Capabilities Built on top of HPE 3 PAR Remote Copy rcopy. Target. Container • Specifies the combined array sys. Name and Storage Container. • Multiple values possible for each target sys. Name, one for each Storage Container at that target. • Syntax: sys. Name: storage. Container. Name rcopy. Mode • Specifies requested remote copy mode. • Periodic [only mode supported today for VVols] • Remote Copy itself supports synchronous, streaming and multi-target synchronous / periodic (SLD) rcopy. RPO • Specifies periodic sync period. • Range: (5 min - 1 year) • Requires rcopy. Mode be set to Periodic

3 PAR VASA Replication Capabilities Built on top of HPE 3 PAR Remote Copy rcopy. Remote. CPG • Specifies the Provisioning Group (a. k. a CPG, a template for the volume physical characteristics, such RAID, and device type) to be used for remote copy at the target site for disk VVols. • If not specified defaults to CPG capability, if specified. If not specified, VASA Provider selects a CPG. rcopy. Remote. Snap. CPG • Specifies the CPG to be used for remote copy at the target site for base volumes • One selectable from SPBM dropdown • If not specified defaults to same as rcopy. Remote. CPG capability Where possible, other VVol capabilities are mirrored at the remote site. For example, if deduplication is selected at the local site, a replica VVol on the remote array would be created with deduplication, if the remote array and remote CPG allow for it.

Disaster Recovery with VVol-based VMs Types of failover Planned – Changes the responsibility for mastership of the VVol-based VMs from the Source/Primary site to the Target/Secondary site. – Source and Target roles are reversed. – Involves: – Collecting replicated VM and group information at the Source and Target sites. – Powering Down VMs at Source site – Issuing a failover operation (Spbm-Failover. Replication. Group in Power. CLI) – Unregistering VMs at the source site – Registering the VMs at the failover/target site. – Applying a storage policy that allows replication back to the original Source site. – Issuing a reverse operation (Spbm-Reverse. Replication. Group in Power. CLI) – Optionally powering up the failed-over VMs 60

Disaster Recovery with VVol-based VMs Types of failover Unplanned – Changes the responsibility for mastership of the VVol-based VMs from the Source/Primary site to the Target/Secondary site. – Source and Target roles are reversed (eventually). – When the disaster occurs: – – – There’s no undo option. Collecting replicated group information at the Target/Recovery site Once you issue a Planned Issuing a failover operation (Spbm-Failover. Replication. Group in Power. CLI) or Unplanned failover, the Registering the VMs at the failover/target site. only option is to continue forward. Issue a reverse, Applying a storage policy that allows replication back to the original Source site. and optionally another Optionally powering up the failed-over VMs – After the original Source site has recovered from it’s disaster: – Powering Down VMs at Source site (if needed) – Unregistering VMs at the source site – Issuing a reverse operation (Spbm-Reverse. Replication. Group in Power. CLI) failover and reverse to bring things back up at the original site. 61

Disaster Recovery with VVol-based VMs Types of failover Test – Allows testing of replicated VVol-based VMs at the secondary site, by making copies of the replica VVols and exposing them to ESXi hosts for VM testing. – Source and Target roles are NOT reversed. – Involves: – Collecting replicated VM and group information at the Source and Target sites. – Issuing a test-failover operation (Start-Spbm. Replication. Test. Failover in Power. CLI) – Registering the new test-VMs at the target site. – Applying a storage policy that allows replication back to the original Source site – To help verify those replication policies are valid. – Powering up, testing and powering down the in-test VMs – Un-registering the test-VMs – Issuing a stop-test-failover operation (Stop-Spbm. Replication. Test. Failover in Power. CLI) – Once the test is stopped, all VMs created by the test are destroyed permanently. 62

Storage Administrator’s Responsibility after a Disaster [may remove this slide] – Clean-up of automatically-created replication groups is automatic, unless a true disaster occurs. – tbd 63