Protecting Your Crown Jewels in a Virtual World
- Slides: 70
Protecting Your Crown Jewels in a Virtual World: Strategic Systems Recovery January 29, 2013 3: 45 - 5 PM New York Hilton Legal. Tech® New York January 29 – 31, 2013
Speakers: • Marcus Bluestein, Kraft Kennedy, Moderator • Steven Shock, Irell & Manella LLP • Jim Mc. Cue, Rodey Law Firm • Steve Skidmore, Martin, Clearwater & Bell LLP Legal. Tech® New York 2013 2
Industry Trends Marcus Bluestein Chief Technology Officer Kraft Kennedy bluestein@kraftkennedy. com Legal. Tech® New York 2013 3
Industry Trends Use of server virtualization increase Servers are highly virtualized again… 70 60 50 40 30 20 10 0 2012 Legal. Tech® New York 2013 2011 2010 2009 4
Industry Trends …as does use of SANs. Legal. Tech® New York 2013 5
Industry Trends Traditional backup systems are declining Legal. Tech® New York 2013 6
Industry Trends Firms increase use of disk-based backup Legal. Tech® New York 2013 7
Industry Trends …increase use of replication technology, Legal. Tech® New York 2013 8
Industry Trends …and slowly begin to explore online backup. Legal. Tech® New York 2013 9
RPO and RTO RECOVERY POINT OBJECTIVE (RPO) The amount of data loss associated with a service failure. The Recovery Point Objective (RPO) is the point in time to which you must recover data as defined by your organization. This is generally a definition of what an organization determines is an "acceptable loss" in a disaster situation. If the RPO of a company is 2 hours and the time it takes to get the data back into production is 5 hours, the RPO is still 2 hours. Based on this RPO the data must be restored to within 2 hours of the disaster. RECOVERY TIME OBJECTIVE (RTO) The amount of downtime associated with a service failure. This is meant to describe the amount of time between when a service becomes unavailable to users and when service is restored. This includes time to try to fix the problem without resorting to the recovery procedures, the time to enact the recovery process, testing of the recovered system and communication to the users. Legal. Tech® New York 2013 10
RPO and RTO Applications Perceived RTO Application Perceived RPO Achievable RTO Achievable RPO Desired RTO Desired RPO Messaging DMS Accounting Lit Support Remote Access CRM Docketing HR Records Management Knowledge Management Other practice area specific systems Legal. Tech® New York 2013 11
Strategic Disaster Recovery with Exchange 2010 Steven Shock Chief Technology Officer Irell & Manella LLP SShock@irell. com (310) 203 -7511 Legal. Tech® New York 2013 12
Irell & Manella LLP • 450 users • 2 Offices • IP Litigation Legal. Tech® New York 2013 • Concierge Service • Retention Policy 13
Irell Environment • • • Los Angeles Primary Data Center Newport Beach Data Center Arizona Data Center HP Server – DL 380: 20 Hosts HP Storage – HP P 4000 Lefthand: 1. 5 Petabyte raw storage VMWare 5. 1 - SRM Cisco Nexus Core Switching MPLS – 100 mbps each location Watchguard Branch Office VPN – LA & AZ Legal. Tech® New York 2013 14
BASIC CONCEPTS Legal. Tech® New York 2013 15
Client Connections MAPI Connects through the CAS Role What “Mailbox” Server is listed in Outlook Profile? Legal. Tech® New York 2013 16
Database Availability Group • Replication Boundary • High Availability vs. Site Resiliency – HA is Site-based – Site Resiliency Across Sites • Consider Recovery Objectives • Quorum Isolation Legal. Tech® New York 2013 17
Redundancies • Servers / VMs • Load Balancing Legal. Tech® New York 2013 • Multiple Database Copies • Shadow Redundancy 18
Design Philosophy • New Design Philosophy – Recovery Objectives – Database Design • Dependent on Environment – Simple – Complex – Moderately Complex • Staggering Number of Options! Legal. Tech® New York 2013 19
Design Options Too Many Options! • High Availability • • Exchange Native VMware Third Party On-premise Third Party Email Continuity • Client Connectivity • • • Outlook Any. Where Outlook Web. App SSLVPN / UAG Mobile Devices MDM Legal. Tech® New York 2013 • Replication • • Exchange Native Third Party Software SAN-based Future Integration • Disaster Recovery • Exchange Native • Third Party Email Continuity • Third Party On-premise • Software Based • SAN Solutions 20
Recovery Objectives • Reasonable Objectives are Usually Exceeded – Don’t Increase Firm Expectations! • What Failures are you Designing for? – Failure Domain(s) • Time Objective (RTO) • Point Objective (RPO) Legal. Tech® New York 2013 21
Database • Design Philosophies: Old and New – Policy – Backups – Recovery Objectives • Large Databases • Archive Database • Lagged Copies Legal. Tech® New York 2013 22
Database • Single Item Recovery • Recovery Database • Override Mount Dial • Offline Database Copy Seeding – http: //blogs. technet. com/b/timmcmic/archive/2011/07/12/exchange-2010 -using-vss-toperform-an-online-offline-database-seed. aspx Legal. Tech® New York 2013 23
CAS / Hub Roles • Multiple Servers Hosting Roles • Production and DR • NLB for CAS • Hub – Native HA Legal. Tech® New York 2013 24
Network / WAN • WAN Redundancy – Links – Carriers – Equipment • Replication Network • DAG Replication Maximum Latency – Adjust Cluster Settings? Legal. Tech® New York 2013 25
Backup / Recovery Portable Database Recovery Database SAN Replication Legal. Tech® New York 2013 Monolithic Point in Time No Backups 26
Other Options • HA at Virtualization Level • SAN Replication – Database or Full Environment • Managed & Cloud Services Legal. Tech® New York 2013 27
DR Testing • Worse Than a Disaster? • Full Failover vs. Unit Testing • VMware / Site Recovery Manager – Restore Environment in a “Bubble” – Restoring DAG Can be Difficult – Consider New Server and Database Portability Legal. Tech® New York 2013 28
Site Failover • Database Failover • Activation Blocking • CAS Proxying Legal. Tech® New York 2013 • Site Failover – Start/Restore DAG – Update DNS • Internet Mail 29
Site Failback • Restore / Resume Servers • Resume / Update Database Copies • Verify Health • Failover Databases • Update DNS Legal. Tech® New York 2013 30
Lessons Learned - 1 • NLB / Kerberos Authentication – Consider DR site – http: //technet. microsoft. com/en-us/library/ff 808312. aspx • DAG in a Single Site? – Put DAG in DAC Mode Anyway – Datacenter. Activation. Mode [DAGName] –Datacenter. Activation. Mode DAGOnly • Only One CAS Server? – Still Consider using CASArray Setting – Future Growth Legal. Tech® New York 2013 31
Lessons Learned - 2 • Failback URL – http: //blogs. technet. com/b/mbaher/archive/2010/11/27/robert-s-rules-of-exchange -namespace-planning. aspx • Namespace Considerations – Internet Name(s) – MAPI – Don’t Mix! Legal. Tech® New York 2013 32
Steven Shock Chief Technology Officer Irell & Manella LLP SShock@irell. com (310) 203 -7511 Legal. Tech® New York 2013 33
Q & A: Steven Shock Legal. Tech® New York 2013 34
Back Up/Recovery with VMWare Jim Mc. Cue Information Systems Manager Rodey Law Firm jmccue@rodey. com Legal. Tech® New York 2013 35
Rodey Environment • Overview – 75 attorneys/150 people/2 offices – Litigation/Business – 30 mb QMOE WAN Albuquerque – Santa Fe Legal. Tech® New York 2013 36
Rodey Environment • Virtualization – 80% virtual – 2 datacenters (Albuquerque, Santa Fe) – 4 VMWare 4. 1 esxi hosts w H/A – 30 Guests • Exchange 2010 • Aderant • Autonomy • BES, File Servers, DCs, Mail. Marshal Legal. Tech® New York 2013 37
Rodey Environment • Equal. Logic SANS – Albuquerque 2 x PS 6000 XV • 13 TB usable • 3. 5 TB data + replication – Santa Fe PS 6000 E • 11 TB usable • 3 TB data + replication • Local disk backup storage – Albuquerque, Santa Fe • Drobo B 1200 i: 12 TB • Disk to tape: Dell Powervault lto 5 tape autoloader Legal. Tech® New York 2013 38
Rodey Goals • RPO (Recovery Point Objective)=4 hour • RTO (Recovery Time Objective)=8 hours, 16 hours – Tier one applications=8 hours • Network Authentication • E-Mail • Documents • Time Entry & Billing – Tier two applications=16 hours • E-discovery & Litigation DBs • BES • Citrix Legal. Tech® New York 2013 39
Rodey Goals • D/R – Move operations to other office – Replication • Restores – VM Level Recovery – Application Level Recovery • Files, SQL, Exchange, A/D • Archive to tape->move to disk offsite in the future Legal. Tech® New York 2013 40
Equal. Logic • Thin provisioning • Snapshots • Replication – 6. 0 firmware: Synchronous • Equal. Logic Snapshot Manager/VMware Edition (ASM/VE) – VMWare Tools quiescence – VMWare consistent image • Equal. Logic Snapshot Manager/Microsoft Edition (ASM/ME) – Volume Shadow Copy: Exchange, SQL – Requires guest attached storage Legal. Tech® New York 2013 – Transactional consistent image 41
Veeam Backup & Replication • Bare metal hypervisor backup – Vmware/Hyper-V • Deduplication • Compression • Changed Block Tracking (VMware CBT) Legal. Tech® New York 2013 42
Veeam Backup & Replication • Application aware processing – Exchange, SQL, AD, Sharepoint – Veeam agent -> MS VSS agents – No permanent guest agent – Transactional consistent image – Log truncation Legal. Tech® New York 2013 43
Veeam 6 – distributed processing Legal. Tech® New York 2013 44
Veeam Backups • Full, • Incremental • Reverse Incremental Legal. Tech® New York 2013 45
Veeam Replication • Replication – Full + VMWare snapshots – Network mapping and re-IP Legal. Tech® New York 2013 46
Veeam Replication • Replication Recovery – Failover – Failback Legal. Tech® New York 2013 47
Veeam Verification • Sure. Backup – – Live Verification Virtual Lab Application Group Scheduled Legal. Tech® New York 2013 48
Veeam Verification • Application Group - Role Legal. Tech® New York 2013 49
Veeam Restores Legal. Tech® New York 2013 50
Veeam Restores • Instant Recovery – Mount backup file as NFS – Track VM changes to store – Vmotion backup file +changes • VM – Overwrite, new location • VM Hard Disks • Guest Files • Application Items: Application Item Recover – Exchange, SQL, AD Legal. Tech® New York 2013 51
What does Rodey do? • Backups – Veeam to disk – Backup Exec disk to tape • Replication – Veeam to remote SAN • Snapshots – Equallogic to same SAN Legal. Tech® New York 2013 52
Equal. Logic & Veeam • Application specific tips – Active Directory • avoid restore/failover – SQL • Use native SQL backups to disk • Don’t truncate logs with Veeam – Exchange • Tie to A/D backup • DAGs - watch for failover Legal. Tech® New York 2013 53
Equal. Logic & Veeam • Things to watch out for – Stagger scheduling • • Backups Replication Snapshots Backup of backups – Open snapshots: alarms – Virtual or physical – Don’t backup to VMWare disks! • Guest attached storage Legal. Tech® New York 2013 54
Jim Mc. Cue Information Systems Manager Rodey Law Firm jmccue@rodey. com (505) 766 -7504 Legal. Tech® New York 2013 55
Q & A: Jim Mc. Cue Legal. Tech® New York 2013 56
Backing Up with Evault Steve Skidmore IT Director Martin Clearwater & Bell LLP skidms@mcblaw. com Legal. Tech® New York 2013 57
Backing Up with Evault • Martin Clearwater & Bell LLP – Litigation Firm – IT Department Legal. Tech® New York 2013 58
Backing Up with Evault • Risk Assessment – Security Risk Tolerance • Interruption to business can be costly • Responsibility to clients – Geographical Location • United Nations, Israeli Consulate, Grand Central Station – Hardware / Software Infrastructure – Regulatory Compliance Requirement Legal. Tech® New York 2013 59
Backing Up with Evault • Risk Assessment – Disaster – Outages – Planned Maintenance Legal. Tech® New York 2013 60
Backing Up with Evault • Risk Assessment – Essential Applications – Recommended Applications – Non-Essential Applications Legal. Tech® New York 2013 61
Backing Up with Evault • Risk Assessment – Who is going to access the recovery system? – Where are they going to access from? – How will they access the system? Legal. Tech® New York 2013 62
Backing up with Evault • Things to consider – Traditional backups • Significant capital • Costly to scale • Multiple systems – Cloud-based DR services • Turnkey solution • Virtualized environments • Overburdened • Budget-friendly Legal. Tech® New York 2013 63
Backing up with Evault • Cloud Disaster Recovery Service – Is data securely transferred? – Is data securely stored? – How does the authentication process work? – Meet regulatory requirements? – Bandwidth capacity? Legal. Tech® New York 2013 64
Backing up with Evault • Cloud Disaster Recovery Service – Secure, Virtual hot cloud environment – SSAE 16 -compliant data center – ISO Certified – HIPAA Compliant – Offsite Solution – 4 hr SLA Legal. Tech® New York 2013 65
Backing up with Evault • Hybrid Approach – On Prem – Cloud Legal. Tech® New York 2013 66
Backing up with Evault • Setup – Application Plug-ins – Backup Agents – Collector S/W – Additional NIC – Subnet – VPN Legal. Tech® New York 2013 67
Backing up with Evault • Setup – Replication • Vaults • Servers – Utilities • Migrate • Optimization • Test Legal. Tech® New York 2013 68
Q & A: Steve Skidmore Legal. Tech® New York 2013 69
Q & A: General Legal. Tech® New York 2013 70
