Architecting Fibre Channel HA Solutions Rick Jooss richard
Architecting Fibre Channel HA Solutions Rick Jooss richard. jooss@netapp. com
Agenda 4 CFModes 4 Single System Imagine 4 Multipathing 4 Host Clustering 4 Storage System Backend HA 4 Q&A 2
Agenda 4 CFModes 4 Single System Image 4 Multipathing 4 Host Clustering 4 Storage System Backend HA 4 Q&A 3
CFMODE – Cluster Failover Mode 4 What is CFMODE? – FCP Setting – Determines behavior of FC Target Ports, particularly during a CFO event 4 Why is there more than one CFMODE? – Original CFMODE (standby) did not work for all host types (HP-UX, AIX) – Original CFMODE did not work with the 270 C because it only has a single FC port 4
Available Paths - Standby Mode Host Switch/Fabric 1 Switch/Fabric 2 Solid Blue are paths to the LUNs being served by Controller 1 HA Configuration 0 c 0 d 0 a 0 b 0 c 0 d 0 a Controller 1 Dashed Purple are paths to the LUNs being served by Controller 2 0 b Controller 2 LUNs 5
Path Access (Switch Failure) – Standby Mode Host MP layer works 1 around Switch/Fabric will the failure a failure experience Switch/Fabric 1 Switch/Fabric 2 Solid and Blue are paths to the LUNs being served by Controller 1 HA Configuration 0 c 0 d 0 a 0 b 0 c 0 d 0 a Controller 1 0 b Dashed and Purple are paths to the LUNs being served by Controller 2 LUNs 6
Path Access (CFO event) - Standby Mode Host Conntroller 2 1 Takes will over Controller experience a failure all operations Switch/Fabric 1 Switch/Fabric 2 Solid and Blue are paths to the LUNs being served by Controller 1 HA Configuration 0 c 0 d 0 a 0 b 0 c 0 d 0 a Controller 1 0 b Dashed and Purple are paths to the LUNs being served by Controller 2 LUNs 7
Path Access (CFO event) - Standby Mode Host Switch/Fabric 1 WWN 1 HA Configuration 0 c Switch/Fabric 2 WWN 3 0 d 0 a WWN 4 0 b WWN 5 WWN 6 WWN 7 0 c 0 d 0 a Controller 1 WWN 8 LUNs 8 Solid and Blue are paths to the LUNs being served by Controller 1 0 b Controller 2 LUNs Filer Head 12 will Takes over Controller all operations experience a failure MP layer is not involved in switchover Dashed and Purple are paths to the LUNs being served by Controller 2
Available Paths - Partner Mode Host Switch/Fabric 1 Switch/Fabric 2 Solid Blue are paths to the LUNs being served by Controller 1 HA Configuration 0 c 0 d 0 a 0 b 0 c 0 d 0 a Controller 1 Dashed Purple are paths to the LUNs being served by Controller 2 0 b Controller 2 LUNs 9
Available Paths - Partner Mode – FAS 3000 Default Configuration Host Switch/Fabric 1 Switch/Fabric 2 Solid Blue are paths to the LUNs being served by Controller 1 HA Configuration 0 c 0 d 0 c Controller 1 0 d Controller 2 LUNs 10 Dashed Purple are paths to the LUNs being served by Controller 2
Available Paths - Dual Fabric Host Switch/Fabric 1 Switch/Fabric 2 Solid Blue are paths to the LUNs being served by Controller 1 HA Configuration 0 c_0 0 c_2 0 c_0 Controller 1 Controller 2 LUNs 11 Dashed Purple are paths to the LUNs being served by Controller 2
Agenda 4 CFModes 4 Single System Imagine 4 Multipathing 4 Host Clustering 4 Storage System Backend HA 4 Q&A 12
What is the single system image cfmode? 4 Universal cfmode – Works on all HA storage systems – Works on all switches 4 Presents the HA configuration as a single target 4 All LUNs are visible on all controller ports 4 All hosts require multipathing software 13
Available Paths - Single System Image – Single Card Host Switch/Fabric 1 Switch/Fabric 2 Solid Blue are paths to the LUNs being served by Controller 1 HA Configuration 0 c 0 d 0 c Controller 1 0 d Dashed Purple are paths to the LUNs being served by Controller 2 LUNs 14
Path Access (Switch Failure) - Single System Image – Single Card Host Switch/Fabric MP layer works 1 around will the failure a failure experience Switch/Fabric 1 Switch/Fabric 2 Solid and Blue are paths to the LUNs being served by Head 1 HA Configuration 0 c 0 d 0 c Controller 1 0 d Dashed and Purple are paths to the LUNs being served by Head 2 Controller 2 LUNs 15
Path Access (CFO event) - Single System Image – Single Card Host Switch/Fabric 1 Switch/Fabric 2 2 will takes over Controller 1 all operations experience a failure MP layer works around the failure Solid Blue are paths to the LUNs being served by Controller 1 HA Configuration 0 c 0 d 0 c Controller 1 0 d Dashed Purple are paths to the LUNs being served by Controller 2 LUNs 16
Available Paths - Single System Image – Single Port Host Switch/Fabric 1 Switch/Fabric 2 Solid Blue are paths to the LUNs being served by Controller 1 HA Configuration 0 d 0 d Controller 1 Dashed Purple are paths to the LUNs being served by Controller 2 LUNs 17
Available Paths - Single System Image – Single Port Host Loop Mode HA Configuration Loop Mode 0 d 0 d Controller 1 Dashed Purple are paths to the LUNs being served by Controller 2 LUNs 18 Solid Blue are paths to the LUNs being served by Controller 1
Why SSI mode? 4 Works in all configurations 4 Makes us look more like other SAN vendors 4 Reduces port burn without using FC Loop – Fully redundant config requires only 1 “wire” per controller, instead of 2. 4 Simpler wiring, no a/b port distinctions and no requirement to run the same cables from each controller to the same switch. 19
Management changes 4 Unified LUN mapping address space across the HA configuration. – Controller prevents these conflicts by checking with the partner controller. 4 If the controller interconnect is down, some operations are disabled by default – Igroup add, lun map, lun online, igroup set ostype 20
SSI Roadmap 4 Introduced in ONTAP 7. 1 4 Refer to FCP host compatibility matrix http: //now. netapp. com/NOW/knowledge /docs/san/fcp_iscsi_config/index. shtml for specific host support 21
Agenda 4 CFModes 4 Single System Imagine 4 Multipathing 4 Host Clustering 4 Storage System Backend HA 4 Q&A 22
Multipathing 4 Multipathing provides multiple paths from the host to the external storage device 4 Provides High-Availability – Protects against path failures – Ensures high availability of applications and data by eliminating single points of failure 4 Provides Improved Performance – Increases potential performance by utilizing multiple paths 23
Multipathing Host Switch/Fabric 1 HA Configuration 0 c Switch/Fabric 2 0 d 0 c Controller 1 0 d Controller 2 LUNs 24
A/P (active passive) policy – Single LUN Hosts Switch/Fabric 1 HA Configuration Switch/Fabric 2 0 c 0 d 0 c Controller 1 0 d Controller 2 LUNs 25
A/P (active passive) policy – No Round Robining Hosts Switch/Fabric 1 HA Configuration Switch/Fabric 2 0 c 0 c 0 d Controller 1 0 d Controller 2 LUN 1 LUN 3 LUN 2 26 LUN 4
A/P (active passive) policy - Round Robining Hosts Switch/Fabric 1 HA Configuration Switch/Fabric 2 0 c 0 d 0 c Controller 1 0 d Controller 2 LUN 1 LUN 3 LUN 2 27 LUN 4
A/P (active/passive) 4 Active/Passive Configuration – 1 active path to a single LUN • Performance to a LUN is limited by that paths capability (HBA, switch, target port) – Possible to round robin multiple LUNs across multiple paths – All other paths to the LUN are passive – On failover • Primary paths are tried first • Secondary paths are used if no primary paths are available 28
A/A (Active active) policy (cfmode = standby) Hosts Switch/Fabric 1 HA Configuration 0 c Switch/Fabric 2 0 d 0 a 0 b 0 c 0 d 0 a Controller 1 0 b Controller 2 LUNs 29
A/A (active/active) 4 Host accessing data from a single LUN across multiple paths simultaneously – Typically used for load balancing • Round Robin • Least Queue Depth • Weighted – On failure I/Os are sent down remaining available paths 30
A/A/A (asymmetric active) Host Switch/Fabric 1 HA Configuration 0 c Switch/Fabric 2 0 d 0 c Controller 1 0 d Controller 2 LUNs 31
A/A/A (asymmetric active) 4 Distinguishes between primary and secondary paths 4 Does active/active across primary paths only 4 Only uses secondary paths when no primary are available 32
Net. App’s Multipathing Strategy 42 pronged strategy – Support for “native” solutions • What most customers rightly feel best about – Support for host and storage independent solution • VERITAS • Allows common solution across various server as well as storage variants 33
Multipathing For Windows 4 Windows MPIO – Uses the Microsoft standard infrastructure – A/P Policy – Automatically chooses primary paths for failover before trying proxy ones – In standby the LUNS are automatically round robined across all paths MPIO Partner/SSI cfmode A/P Standby cfmode A/P Dual Fabric cfmode A/P 34
Multi. Pathing For Solaris DMP 4. 0 MPx. IO Partner/SSI cfmode A/A/A A/P Standby cfmode A/A N/A Dual Fabric cfmode A/P 35
Multi. Pathing For Solaris 4 VERITAS DMP 4. 0 – Net. App ASL 4. 0 – Supports A/P, A/A, & A/A/A (Active Passive Concurrent) 4 SUN Native MPx. IO – – – Not supported with standby cfmode Supports A/P Can be A/A but required manual failback Manual configuration required Round Robining of the LUNs possible Sometimes called • Traffic Manager • Leadville Stack 36
Multi. Pathing For Linux 4 Qlogic – A/P Policy – Manually configured – Round Robining of LUNs is possible 4 DCM – Linux native solution Qlogic DM Partner/SSI cfmode A/P A/A/A Standby cfmode A/P A/A Dual Fabric cfmode A/P 37
Multi. Pathing For AIX DMP 4. 0 SANpath MPIO Partner/SSI cfmode A/A/A Standby cfmode N/A NA Dual Fabric cfmode A/P A/P 38
Multi. Pathing For AIX 4 SANpath – A/A/A – Automatically chooses primary paths for failover before trying proxy ones – Special policy for SCSI-2 reservation – Required for host clustering HACMP – Can only use A/P 4 VERITAS DMP 4. 0 – Only supports A/A/A 4 IBM MPIO – IBM native solution with Net. App PCM 39
Multipathing for HP-UX PVLinks DMP 3. 5 Partner/SSI cfmode A/P Standby cfmode N/A Dual Fabric cfmode A/P 40
Multipathing for HP-UX 4 PVlinks/LVM – – A/P policy Single active path per LUN, user controlled Ordering for remaining paths for failover ntap_config_paths • NETAPP script to define path ordering based on filer path types: primary, proxy • automatically round robin primary paths among all LUNS – Supports both FCP and i. SCSI paths 4 VERITAS DMP 3. 5 – A/P Policy 41
Multipathing for VMware 4 VMware – A/P Policy – Manually configured – Round Robining of LUNs possible VMware Partner/SSI cfmode A/P Standby cfmode A/P Dual Fabric cfmode A/P 42
Multipathing for Netware 4 Novell – A/P Policy – Manually configured – Round Robining of LUNs possible Novell Partner/SSI cfmode A/P Standby cfmode A/P Dual Fabric cfmode A/P 43
Fibre Channel SAN Host Support Partner/SSI cfmode Windows “NTAP DSM” Linux: Qlogic “Failover Mode” Standby cfmode Dual Fabric cfmode A/P A/P A/P VMware Multipathing A/P A/P Solaris “DMP” A/A/A A/P Solaris “MPx. IO” A/P N/A A/P AIX “SANpath” A/A/A N/A A/P HP-UX “PVLinks” A/P N/A A/P Novell A/P A/P 44
Agenda 4 CFModes 4 Single System Imagine 4 Multipathing 4 Host Clustering 4 Storage System Backend HA 4 Q&A 45
Host Clustering & Storage 4 LUNs need to be made visible to host simultaneously 4 Some Host Clustering solutions require SCSI reservations to avoid to split brain 46
Host Clustering for Microsoft 4 Microsoft Cluster – Snap. Drive is integrated to help configuration – WIN 2 K 3 allows single HBA for both boot device & shared storage – Cannot grow LUN online in cluster • Snap. Drive ability to very quickly grow a LUN minimizes the pain caused by this 47
Host Clustering for VERITAS 4 VCS – By default does not us I/O fencing to protect against split brain – I/O fencing requires SCSI-3 reservations – 7. 0. 3 will have SCSI-3 reservations that are compatible with VERITAS – Does not do failover on FC links 48
Host Clustering for HP-UX 4 Service. Guard – 1 to 3 node clusters using SCSI-2 locks as arbitrator to avoid split brain – Does not do failover in dead FC links 49
Host Clustering for AIX 4 HACMP – Uses SCSI-2 locks as arbitrator to avoid split brain • “setsp –b 2” to enable locks with SANpath • SCSI-2 locks to active/active are mutually exclusive 50
Fibre Channel SAN Host Support OS Vendor HBA Multipath Host Cluster Native SANpath HACMP LVM JFS/2 Raw Emulex MPIO MSCS MMC NTFS Volume Mgr File System ext 3 ext 2 Reiser QLogic Oracle 9 i, 10 g RAC Emulex Veritas DMP Veritas VCS Veritas Vx. VM Veritas Vx. FS Native HP PVLInks Veritas DMP MC Service. Guard Veritas VCS LVM Veritas Vx. VM JFS/ HFS Raw Veritas Vx. FS Emulex QLogic VMWare MSCS Virtual. Center (VMotion) VMware VMFS 2. x Raw QLogic Novell Clusters 51 NSS
Shared Storage
Agenda 4 CFModes 4 Single System Imagine 4 Multipathing 4 Host Clustering 4 Storage System Backend HA 4 Q&A 53
Enables Dual Path HA Protect Against Cable Pulls Or Breaks Protect Against Single HBA Failure X X Protect Against Storage Controller (eg. ESH 2) Hot Swap X Key Benefits q Full storage hardware redundancy in HA systems q Prevent cluster failover events due to many storage issues. q Complements CFO for improved HA and resiliency Loop 1 Loop 2 Loop 4 Loop 3 54
Switched Back-End 4 Dual Active Paths for HA Environments – Reduces the number of HA failovers – Improve overall HA performance – Data ONTAP tries to balance load across paths 4 Sync. Mirror – Sync. Mirror requires 100% disk overhead – Proper configuration survives all single failures 55
Agenda 4 CFModes 4 Single System Imagine 4 Multipathing 4 Host Clustering 4 Storage System Backend HA 4 Q&A? 56
- Slides: 56