Business Continuity Disaster Recovery Solutions Workforce Continuity Briefing
Business Continuity & Disaster Recovery Solutions Workforce Continuity Briefing for Virginia Business Recovery Association GLOBAL CAPABILITY. PERSONAL ACCOUNTABILITY. Jonathan Nguyen-Duy Verizon Business Continuity Services February 21, 2008 © 2008 Verizon. All Rights Reserved. PTEXXXXX XX/08
PROPRIETARY STATEMENT This document and any attached materials are the sole property of Verizon and are not to be used by you other than to evaluate Verizon’s service. This document and any attached materials are not to be disseminated, distributed, or otherwise conveyed throughout your organization to employees without a need for this information or to any third parties without the express written permission of Verizon. The Verizon and Verizon Business names and logos and all other names, logos, and slogans identifying Verizon’s products and services are trademarks and service marks or registered trademarks and service marks of Verizon Trademark Services LLC or its affiliates in the United States and/or other countries. All other trademarks and service marks are the property of their respective owners. 2
Agenda • BCDR/COOP Challenges – Key issues – Extended global enterprise • Verizon Business Services – Natural disasters – Pandemic flu – Remote working – Workforce continuity – Practical solutions • Summary – Key insights and recommendations 3
History of disaster recovery and business continuity In the beginning……. – Mainframe computer-based only – Technology driven – Disaster recovery only – Expensive subscriptions and recovery – Long term recovery questionable – Hard sell to management And now…. . – Business driven – Disaster recovery and business continuity, both required – Regulatory requirements – Enterprise-wide – Telecommunications requirements critical – Cost of doing business 4
Verizon Business BCDR/COOP & The Extended Enterprise – Expanding perimeters and interdependencies means more opportunities for interruptions » Natural disasters » Cyber threats » Biological threats » Change management » Power outages » Supply chain dependencies – Increasing exposure and risks » Compliance » Lost revenue, capitalization » Public trust – Requires all hazards approach » Network security » Physical security » BCDR/COOP planning » Applications resilience » SCM 5
Verizon Business Continuance and Emergency Management Capabilities GLOBAL CAPABILITY. PERSONAL ACCOUNTABILITY. © 2008 Verizon. All Rights Reserved. PTEXXXXX XX/08
Verizon Business BCEM Program • Global business continuity planning organization – 35, 000 employees – 321 offices in 75 countries across six continents – Global IP footprint serving 2, 700+ cities in 150 countries – More than 200 state-of-the-art data centers in 22 countries – Unique capabilities MERIT Team – Partner with federal agencies and task forces • Capable of addressing local, regional and international events – Automated BCP tools – Integrated multi-functional approach » Network Operations » Managed Services » Security » Customer Support » Public Relations • Certified Business Continuity professionals, processes, and products – Utilize federal and industry standards » National Incident Management System » Disaster Recovery Institute and Business Continuity Institute 7
Verizon Best Practices ERT and MERITSM Teams • Verizon Business Emergency Response Teams (ERT’s) support tactical recovery operations in each geographic region • Major Emergency Response Incident Team SM (MERIT ) - Verizon’s fully capable hazardous materials team. – Technical systems restoration – Outside plant and facilities engineering – Customer equipment restoration – Environmental health and safety • Capable of entering and recovering in any hazardous environment – Asbestos, Train Derailments, Major Flooding, Glycol Spills, Anthrax Contamination. 8
Verizon Best Practices Mobile Communications Command Center • 40 -foot and 53 -foot self-contained vehicles. • 24 individual stations 9 • Vo. IP telephones, wireless IP telephones, and both wired and wireless Ethernet access MC 3 Fleet Deployments • Free Local, long distance, and international calls, as well as access the Internet and send or receive faxes and televisions • Troop Deployment, Fort Carson, Colorado, February – April 2004 • Based in Richardson, Texas. However, a MC 3 can reach either coast within 24 hours. • Missouri Tornado, Pierce City, Missouri, May 2003 • Hurricane Katrina, New Orleans and Gulf Coast, August – September 2005 • Hurricanes Charley, Frances, Ivan, and Jeanne, Florida area, August – October 2004 • Troop Deployment, Fort Hood, Texas, January – March 2004 • Southern California Fires, San Diego, November 2003 • New York City, 9 -11, September 2001
Verizon Best Practices Workforce Continuity • Prepared for a wide range of business interruptions • Enterprise-wide skill set inventory for 247 K+ employees • Mitigation Strategies – Remote Working – Management of IT Systems and Applications – Automation Tools – Work from Home – Staggered Shifts – Hygiene and Cleaning Practices • Response Strategies – Prioritization of Critical Business Processes – Identify the Most Important Functions – Allocate Resources for Continued Operations – Contractor/Vendor Support – Global Work Shifting & Evacuations 10
Verizon Business Pandemic Response GLOBAL CAPABILITY. PERSONAL ACCOUNTABILITY. © 2008 Verizon. All Rights Reserved. PTEXXXXX XX/08
Pandemic Challenges • Can you operate with minimal contact between staff and clients? – Minimizing contact – Mitigation via teleworking, Web conferences, etc. – Effectiveness of remote working • Can you effectively operate with key staff incapacitated for months or permanently? – Loss of personnel – Develop autonomous decision making processes. • How will you operate if your supply chains are disrupted? – Assess and plan for multiple scenarios. – Build redundant capabilities. – Precontract & predeploy resources 12
Current WHO Pandemic Alert Phase • Stage 3 – No or very limited human-to-human transmission 13
Known cases of H 5 N 1 as of February 15, 2008 • 361 cases • 227 deaths • 62% Mortality • Middle East • China • SE Asia • 02/13/08 latest death reported in Indonesia – Symptoms Feb. 2 nd – Died Feb. 13 th Source: World Health Organization, 2008 14
Planning for Pandemic Influenza • Verizon has served on multiple federal, state, and local pandemic planning programs • 2005 Government and industry first planning efforts • 2006 – 2008 National security escalation – U. S. Chamber of Commerce – National Telecom Security Advisory Committee – National Infrastructure Advisory Committee – DHS – Verizon 15
Verizon Best Practices Global Influenza Pandemic Planning • Verizon Business has developed: – A Global Influenza Pandemic Plan (“GIPP”) – Preventative and responsive strategies » increased absenteeism in the later stages of the pandemic » quarantine requirements » worker infection » care for infected family members » child care in the wake of school closings – Pandemic Alert System (complements WHO Alert Phases) • Verizon Business Pandemic Planning – 24 x 7 monitoring – DHS, CDC, WHO etc. – 50% staff absences – two weeks at the height of a pandemic wave and lower levels of staff absence for a few weeks on either side of the peak (approximately 8 week period). 16
Verizon Business – Overall Pandemic Strategy Stage A Stage B Stage C Stage D Reports of influenza virus affecting certain animal populations, but not occurring in countries where Verizon Business facilities are located. Influenza virus affecting animal populations in countries where Verizon Business facilities are located, and there are confirmed cases by the WHO of animal to human transmission. Influenza virus has mutated and there are confirmed cases by the WHO of human to human transmission in countries where Verizon Business facilities are located. The WHO declares an epidemic or pandemic in countries where Verizon Business facilities are located. World Health Organization Inter-pandemic Period Phase 1 17 Phase 2 Pandemic Alert Period Phase 3 Phase 4 Pandemic Period Phase 5 Phase 6
Alert Notification Services Traditional Approaches Not Adequate • East Africa Embassy Bombings – Manual Call Trees – 25 people working full time – 12 days to contact 480 people • Beirut U. S. Evacuations – Manual Calling Trees – Limited to landlines and wireless – No polling capability • Fortune 100 NYC Financial Services Firm – – September 11 th couldn’t reach 6000 employees Manual Call Trees and Broadcast Voicemails 8 days from alert to acknowledgement No multi-modal capabilities • Future situations could be worse 18
Alert Notification Solution Requirements Create alert notifications dynamically for rapid, simultaneous delivery to employees, first responders, and customers. • Priority Delivery – Per customer needs. • Phone, Web and Black. Berry – At any time. ® • International Delivery – 125 countries, with time zone awareness. • Interactivity – Respond or transfer to an operator or live conference call. • Message Escalation – Ensure message reaches the intended recipient. • Delivery Manager – Avoid overloading phone systems or switches. • Message Center – Retrieve messages on answering machines or posted to employee dial-in numbers. • Tracking and Reporting – Instant view of message delivery and response. Generate scheduled and ad hoc reports. 19
Verizon Business Continuity Plan in Action Enabling the Resilient Enterprise 1 2 3 4 Hurricane approaches company executes its Business Continuity Plan. Managed BC planning application sends documents to Incident Response Teams. Web-based Operations Center tracks incident, coordinates response. Integrated alertnotification system sends voice and text messages to GPS enabled wireless devices being used to track vehicles and personnel. Employees use wireless broadband to access private IP for email & collaborative apps, avoiding congested public IP networks. Verizon Business Continuity Plan in Action 20
Workforce Continuity Insights GLOBAL CAPABILITY. PERSONAL ACCOUNTABILITY. © 2008 Verizon. All Rights Reserved. PTEXXXXX XX/08
Regional Workforce Continuity Challenges Communications Networks • Will telecommuting work during a crisis? • Are commercial networks prepared? What we know… • Most enterprises have not planned for workforce continuity • The commercial, best effort, residential Internet access structure may not handle increased load • Impossible to precisely predict what would happen • Networks are largely automated and redundant • 99. 999% level of availability = minimal impact on critical operations • Shift in traffic patterns or bandwidth demand will cause congestion • Congestion during peak hours 3: 00 – 8: 00 PM • Congestion at aggregation points near residential communities 22
Regional Workforce Continuity Challenges Communications Networks 23 Source: Department of Homeland Security
Regional Workforce Continuity Challenges Communications Networks 24 Source: Department of Homeland Security
Regional Workforce Continuity Challenges Remote Working Requirements • Remote Working Connectivity – Up-to-date PC & software – Broadband Internet access – Security – tokens, PKI cards, passwords – Voice – land lines, cell, VOIP soft phones – Voice-Mail Forwarding & unified messaging – Updated contact information • Remote Working Infrastructure – – UPS & batteries for all devices Dedicated workspace Applications Signed Telecommuting/Contingency Work Agreement – Electronic media storage » CD and DVD media drives » Zip disks, memory sticks, flash keys – Shipping and receiving processes 25 • Alternate Worksite-Office Basics – LAN – Internet access, routers, firewalls, etc. – Telecom – Printers, copiers, faxes – PDAs, cell phones, radios – Monitors, PCs, laptops – Task lighting – Office supplies – Office furniture – 2 sq meter work area – Secure storage and disposal of records – PPE & hygiene supplies
Regional Workforce Continuity Challenges Remote Working Strategies 26 § During disasters, communications industry focuses on service maintenance rather than order fulfillment § Provisioning moratorium § Provisioning intervals will increase § Training time for employees will increase § Increase your business process intervals § Redundant systems, back up sites, employee communications and alternate work sites § Consider how employees can access alternate facilities § Consider an alert notification system § Surge demand for network resources and call center services § Remote access support § Demand for HR information § Shift bandwidth intensive process to non-peak hours § Filter bandwidth intensive attachments on email
Service Restoration Priority Plan, prioritize and pre-deploy 1. Verizon Communications needs to manage recovery 2. Telecommunications Service Priority (TSP) service 3. Essential Government Services 4. Public Safety Services 5. Network Infrastructure 6. Priorities of Federal, State, and Local governments 7. Other services 27
Remote Work Practical Solutions § Telecommunications Priorities § Telecommunications Service Priority § Wireless Priority Services § Government Emergency Telecommunications Service Cards § First Steps § Mission Critical Processes and Applications § SCM, CRM, ERP § Systems and applications § Bandwidth, access, hardware requirements etc. § Skill-sets and personnel § Secure Remote Access Considerations § Private networks and Wireless broadband access § Access Continuity – fiber to satellite broadband § Quality of Service and SLAs § Connect to private MPLS network to avoid local public congestion points § Real-time automatic desktop backup – restoration & replication § Enhance security and help desk services 28
Remote Work Practical Solutions Wireless Broadband Access • Purchase a PC Card, Express. Card, or USB Modem to connect your personal computer to wireless broadband network • Impact Assessment • Procure and support – Hardware – Software – Security – Help desk – Wireless services Broadband service Faster downloads — typical speeds of 600 Kbps to 1. 4 Mbps 1 Faster uploads — typical speeds of 500 -800 Kbps 1 Lower latency — improved performance for many applications bursting up to 144 Regular service - bursting up to 144 Kbps, with typical speeds of 60 to 80 Kbps 29
Remote Work Practical Solutions Wireless Router – Failover connection Broadband Access Wireless Router offers a failover option – Simplified deployment and may be more cost-effective than ISDN or other wireline solutions. – Primary connection Broadband Access Wireless Router provides a cost-effective and reliable alternative to terrestrial wireline connections – Other remote applications Broadband Access Wireless Router can be used for remote banking and ATM machines, remote-location cash registers, or mobile applications or portable communications kits for temporary sites. 30
Verizon Business Mobile Office Solution § Quickly deployable § Highly mobile § Access to Voice, Data, and Network (VPN/Internet) 31
Verizon Business Mobile Office Deployment Options §Fixed: § WAN Connection Can Be Terrestrial or Satellite Based § Radio Interface For Two-Way Mobile Radios §Fixed mobile: § Ease Of Configuration And Setup § In The Field Very Quickly §Swift deploy: § Rapid Setup § Rugged Cases 32
Verizon Business Mobile Office Edge Detail Remote Satellite Receiver 802. x Wireless PC PC Wired Vo. IP Phone Satellite Hub LAN Switch Or T 1 Access Verizon Business Vo. IP Server ® Verizon Internet ® Edge Access Dual Ethernet/T 1 Access Device Mix of 4 -24 Wired Analog Phones or OPX Lines Corporate VPN Edge = Customer Edge Router (edge access equipment depicted above) 33 802. x Wireless IP Phone
Conventional Configuration vs. Verizon Business Mobile Office PCs PBX Private Line Network PBX Video Router Video VPN Frame Relay Network ATM Network Satellite Gateway Internet Verizon VOIP ® Router Terrestrial T 1/E 1/DSL Router Video PBX PCs Video PBX Traditional Frame Relay, ATM, or Private Line Networks 34 PCs PBX Edge Access Router Video Verizon Business Mobile Office ®
Access Continuity Satellite Services • North America & EMEA coverage • IP over satellite technology • Qo. S for real time and critical applications • Built-in router functionality • VLAN & VRF compatible for private transmission • Seamless integration with (MPLS) Private IP, true Private IP end-to-end • Private IP Satellite Automatic Disaster Recovery • Redundant Hub locations – large customers can be split among two Hubs • Geographic Hub diversity – Hubs located geographically apart for disaster mitigation • Terrestrial redundancy – two diverse Hubs with diverse connectivity • Auto DR – Remotes on one Hub can automatically fail over to the other if Hub fails • Site visits – dispatches not needed to re-point in case of Hub failure 35
Access Continuity Typical Satellite Applications • Private IP (PIP) and Public Internet Access • Disaster Recovery and Business Continuity • Point-of-Sales (POS) • Data, Vo. IP, & Video over IP • FTP (File Transfer) • Web access/browsing, HTTP • Lotus Notes, Oracle • SAP, Citrix version 1. 8 and newer • SSL (an application layer VPN) • TCP and Web Acceleration over VSAT IP network • Integration & Support Verizon Wireless Applications 36
Satellite Services Disaster Recovery Overview • Provides true Private IP (MPLS) disaster recovery • VSAT acts as a backup to terrestrial Private IP (PIP) • Upon failure, the VSAT automatically takes over and all routing updates are sent via the VSAT link • Upon restoral, terrestrial link is primary and VSAT again becomes backup, also automatically • Customers grouped and sized according to their backup needs and number of sites 37
Satellite Services Hurricanes Katrina/Rita Backup & Restoral • Restored critical services for the FAA, Army, Navy, insurers and other customers after the hurricanes. • Voice, video, insurance applications and government services. – Flyaway Trailers for: » FEMA » Army » Navy • Rapid deployment of fly-away VSAT systems including trailers with voice, data and fax service to a variety of gulf locations. • Some locations only had our trailer for all means of communication 38
Private IP Satellite and PIP National Retailer Disaster Recovery Advantage 39
Disaster Recovery via Private IP Satellite & Verizon Wireless Network 40
Business Continuity Example Wireless IP Enables Mobility and Disaster Recovery • Re-establish agency service in areas struck by disasters • Re-route to temporary location or remote home workers • Help reduce total downtime • Rapidly set up permanent or temporary claims centers – Satellite and EVDO high-speed, secure wireless Source: Nemertes Research Group Inc. , 2006 Vo. IP 41 Mobile Phone IM Unified Messaging Remote Home Workers Vo. IP Network
Verizon Business BCDR/COOP Challenges Summary GLOBAL CAPABILITY. PERSONAL ACCOUNTABILITY. © 2008 Verizon. All Rights Reserved. PTEXXXXX XX/08
911, Katrina and London Bombings Insights • SCM and interdependent systems increase risk • Integrate network and COOP plans • Third-party supplier and partner relationships • Lodging, food, supplies, etc. • Plan for “worst case” and regular scenarios – Takes time to successfully implement a resilient enterprise – Deploy crisis communications system • Factor in data recovery and storage into plans – What is your recovery time objective – What are the critical systems, processes and skill sets • Purchase and deploy communications assets • Be flexible – more process than product 43 24
- Slides: 43