Emergency Management Planning Business Continuity IT Partners March

Emergency Management Planning Business Continuity IT Partners March 17, 2005 © Gerald Isaacson 2005

Business Continuity Planning A few definitions to get started: … a “disaster” • • The cake I was baking to bring to Xmas dinner He lost a laptop with the only copy of his thesis She lost her research and papers in the lab fire Payroll system failed the day before payday Asbestos released in a dorm renovation The death of a student The Northeast blackout The recent tsunami April 19, 2005 © Gerald Isaacson 2005

Business Continuity Planning … a “disaster” is an event, often unexpected, that seriously disrupts your usual operations or processes and can have long term impact on your normal way of life or that of your organization. April 19, 2005 © Gerald Isaacson 2005

Business Continuity Planning A few definitions to get started: … RTO [Recovery Time Objective] the point in time when you must have at least the critical aspects of your business operational again. April 19, 2005 © Gerald Isaacson 2005

Business Continuity Planning A few definitions to get started: …RPO [Recovery Point Objective] The last copy of your data that is out of harm’s way – hopefully it is recently current. April 19, 2005 © Gerald Isaacson 2005

Business Continuity Planning A few definitions to get started: … Business Continuity Planning April 19, 2005 © Gerald Isaacson 2005

Business Continuity Planning … it’s not rocket science April 19, 2005 © Gerald Isaacson 2005

Business Continuity Planning … it is: • a process to minimize the impact of a major disruption to normal operations • a process to enable restoration of critical assets • a process to restore normalcy to MIT as soon as possible after a crisis. … it is not just: • recovery of information technology resources April 19, 2005 © Gerald Isaacson 2005

Business Continuity Planning … and it is the phase of crisis management that follows the immediate actions taken to protect life and property and contain the event … it begins when the situation has been stabilized. April 19, 2005 © Gerald Isaacson 2005

Business Continuity Planning … and it is now a national standard for both the public and private sectors April 19, 2005 © Gerald Isaacson 2005

Business Continuity Planning An ECAR report in March of this year, from Baylor Medical Center and the University of Houston, in the aftermath of hurricane Allison, posed the following list of questions to ask to determine your resilience to a disaster. April 19, 2005 © Gerald Isaacson 2005

Business Continuity Planning ECAR Research Bulletin Mar 1 2005 April 19, 2005 © Gerald Isaacson 2005

Business Continuity Planning An ECAR report in March of this year, from Baylor Medical Center and the University of Houston, in the aftermath of hurricane Allison, posed the following list of questions to ask to determine your resilience to a disaster. The answers constitute your Business Continuity Plan April 19, 2005 © Gerald Isaacson 2005

Business Continuity Planning The Risk Matrix PROBABILITY LOW IGNORE IMPACT HIGH April 19, 2005 PLAN © Gerald Isaacson 2005 HIGH NORMAL PROCEDURES CHANGE SOMETHING

Business Continuity Planning Network Operations Disruptions Power Hardware Source: Gartner Group and Comdisco April 19, 2005 © Gerald Isaacson 2005

Business Continuity Planning April 19, 2005 © Gerald Isaacson 2005

Business Continuity Planning Mt. St. Helens – May 1980 – new threats arise April 19, 2005 © Gerald Isaacson 2005

Business Continuity Planning It’s different now…. April 19, 2005 © Gerald Isaacson 2005

Business Continuity Planning …but we still have the usual concerns © ECAR Research Bulletin Mar 1 2005 April 19, 2005 © Gerald Isaacson 2005

Business Continuity Planning • When is it a Crisis? Minutes Hours Days Continuity Continuum April 19, 2005 © Gerald Isaacson 2005 Weeks

Business Continuity Planning A crisis timeline -- April 19, 2005 © Gerald Isaacson 2005

Business Continuity Planning Alarm Notification to First Responders Data center fire Restoration of Critical Processing April 19, 2005 © Gerald Isaacson 2005

Business Continuity Planning Activate the Emergency Operations Center Restoration of Critical Processing April 19, 2005 © Gerald Isaacson 2005

Business Continuity Planning IT decision to move to a backup facility Restoration of Critical Processing April 19, 2005 © Gerald Isaacson 2005

Business Continuity Planning Assemble IT recovery team at appropriate sites Restoration of Critical Processing April 19, 2005 © Gerald Isaacson 2005

Business Continuity Planning Obtain backup tapes from off-premises storage Restoration of Critical Processing April 19, 2005 © Gerald Isaacson 2005

Business Continuity Planning Acquire and install backup hardware and network connections Restoration of Critical Processing April 19, 2005 © Gerald Isaacson 2005

Business Continuity Planning Restore Operating System and Network Restoration of Critical Processing April 19, 2005 © Gerald Isaacson 2005

Business Continuity Planning Reload database and other data Restoration of Critical Processing April 19, 2005 © Gerald Isaacson 2005

Business Continuity Planning Restore Critical Applications Restoration of Critical Processing April 19, 2005 © Gerald Isaacson 2005

Business Continuity Planning Begin Critical Processing This is your Recovery Time Objective (RTO) Restoration of Critical Processing April 19, 2005 © Gerald Isaacson 2005

Business Continuity Planning Recovery Time Objective (RTO) Do you measure it in hours or weeks? April 19, 2005 © Gerald Isaacson 2005

Business Continuity Planning Recovery Time Objective (RTO) Do you measure it in hours or weeks? When do you need to start to reach it on time? April 19, 2005 © Gerald Isaacson 2005

Business Continuity Planning Windows are Closing April 19, 2005 © Gerald Isaacson 2005

Business Continuity Planning Increasing dependencies on electronic capabilities to do your job have shortened the recovery timeframe. April 19, 2005 © Gerald Isaacson 2005

Business Continuity Planning Increasing dependencies on electronic capabilities to do your job have shortened the recovery timeframe. This often leads to a disconnect between user’s expectations and the organization’s ability to meet them. April 19, 2005 © Gerald Isaacson 2005

Business Continuity Planning High Level Look at a Recovery Effort Lost Data Vital Records Restore Technology Capability Notifications Restore Communications Data Recovery Objective Restore Business Functions Resume Move to Return Business Alternate Home Site (If necessary) Data Synchronization Recovery Time Objective April 19, 2005 © Gerald Isaacson 2005 © Lucent technologies

Business Continuity Planning Continuity Plan Development Phases • Understand the need April 19, 2005 © Gerald Isaacson 2005

Business Continuity Planning Continuity Plan Development Phases • Understand the need • Define the risk to the DLC and define the level of criticality April 19, 2005 © Gerald Isaacson 2005

Business Continuity Planning Criticality Levels • • Category III Category IV April 19, 2005 Must be up in hours Must be up in days Must be up in weeks Can wait till operations are normal © Gerald Isaacson 2005

Business Continuity Planning Continuity Plan Development Phases • • Understand the need Define the risk to the DLC and define the level of criticality • Give an individual responsibility and authority for overall planning April 19, 2005 © Gerald Isaacson 2005

Business Continuity Planning Continuity Plan Development Phases • • • Understand the need Define the risk to the DLC and define the level of criticality Give an individual responsibility and authority for overall planning • Ensure that organizational units – accounting, facilities, residential life, laboratories, etc. understand their individual responsibility for recovery of their operations. April 19, 2005 © Gerald Isaacson 2005

Business Continuity Planning Continuity Plan Development Phases • DLC’s develop their FARM (Functional Area Recovery Management) teams • The Business Continuity Planning Team – (BCMT) consists of FARM Team Coordinators • The BCMT is represented at the MIT Emergency Operations Center (EOC) April 19, 2005 © Gerald Isaacson 2005

Business Continuity Planning Continuity Plan Development Phases • Put together the recovery team • Document the Plan April 19, 2005 © Gerald Isaacson 2005

Business Continuity Planning The Table of Contents from the TLO FARM Team Plan April

Business Continuity Planning • Key Plan Components – Contact list – up to date – Resources needed – not just computing April 19, 2005 © Gerald Isaacson 2005

Business Continuity Planning • Key Plan Components – Contact list – up to date – Resources needed – not just computing – Emergency procedures April 19, 2005 © Gerald Isaacson 2005

Business Continuity Planning Continuity Plan Development Phases • Document the plans for each department, lab or center • Use a common template or framework • Plan review by Business Continuity Management Team (BCMT) and Information Systems & Technology April 19, 2005 © Gerald Isaacson 2005

Business Continuity Planning Continuity Plan Development Phases • Test, test April 19, 2005 ©

Business Continuity Planning Simulation Testing – MIT’s Central Utility Plant • • • On an August afternoon, a fuel line rupture in the Central Utility Plant results in a fire. When the sprinkler system operates, the ensuing flood creates a hazardous waste issue due to the oil and ACM. The sprinkler operation also operates protective disconnects and the power is shut down. There have been 5 injuries of CUP workers because of this incident. After about 4 hours it is determined that the CUP’s clean up and return to service will take at least 24 hours, and the 80% of the campus that the CUP serves will remain without power. April 19, 2005 © Gerald Isaacson 2005

Business Continuity Planning Continuity Plan Development Phases • Test, test • Maintain the plans – a very difficult process but critical to the long term viability of any plan April 19, 2005 © Gerald Isaacson 2005

Business Continuity Planning What’s next? • • Who should we be talking to in your area? Who should we be talking to who is not here? Any overall concerns about the project? Any specific concerns that we should be addressing? April 19, 2005 © Gerald Isaacson 2005

Business Continuity Planning Gerald Isaacson, CISSP William Mc. Shea, CFPS gii@mit. edu wmcshea@mit. edu 617 253 -1440 617 253 -9491 32 -013 web. mit. edu/bcmt http: //web. mit. edu/environ ment/ehs/ April 19, 2005 © Gerald Isaacson 2005