Skip to main content
it helpdesk support

Severe Storms and IT – It is All About Forethought

Severe Storms and IT – It is All About Forethought

September 22, 2016

We are just past the peak of the Hurricane season and thankfully we have been very lucky this year; however, trouble is just a storm away.  Endurance wants to take this opportunity to remind our community that taking time to plan for major weather related events can pay dividends if those plans are ever needed.  These events are chaotic, confusing and dangerous in the best case scenario and without proper fore thought they can be considerably worse.

This article is focused on the IT related business components of weather related disaster, however, Endurance wants to remind our friends that personal weather related planning is perhaps even more important and there are a multitude of resources available to assist you with personal planning. 

Have an Emergency Preparedness Plan

Each organization should have an emergency preparedness plan documented.  Resources should be nominated to fulfill various parts of the plan, with backups named in the event of their unavailability.  Each emergency team member should know their part of the plan and the general staff should know their responsibilities.  This plan should be reviewed at least annually and tested.

Have a Communications Plan

Have a communication plan and a secondary communications plan.  When severe weather occurs, it can knock out your primary means of communications, such as Cellular communications and Internet (email).  In the event that all communications is lost, it is also useful to name a rendezvous location outside the region that can serve as a meet up location.

Decide What Constitutes and Emergency

As some key decision makers may not be available as the emergency draws near, each organization should pre-determine what constitutes a disaster and decide what conditions will cause them to put their emergency preparedness plan into action. 

Ordered Evacuation

In the state of Virginia, if the Governor determines that a storm poses a serious risk, he or she will order an evacuation of affected area.  At that point the LE will reverse the turnstiles on the on ramps headed into our area and all traffic will be sent out of the affected region.  It is important to note that if this order is given, you should leave immediately.  In this scenario, it is estimated that it could take 17-36 hours to get out of the area.  Add in accidents, breakdowns, people running out of gas and other delays, it is not unreasonable to assume that it could take considerably longer.   Given that the Governor will be under political pressure to wait as long as possible before declaring the mandatory evacuation, having our coworkers leave early, should be something we all contemplate.

With respect to business planning, this also means that for some severe events, staff may need to leave up to four days in advance and will not be available for last minute plan execution.  This is something to keep in mind when planning the response to a foreseeable disaster.

See a Disaster Coming

Some weather related disasters can be seen coming days or weeks in advance, so each organization should nominate an individual and a backup within their organization who monitors the weather and reports status to senior management.

These individuals should monitor the following information sources on a regular basis:

Know your Emergency Partners

Bring together outside entities and companies that you will need and pre-negotiate their services before disaster occurs.  For example, having a meeting to discuss with your IT partner how they can help you plan to shut down and restart strategy is a good idea, however, if that discussion is left until right before the event, the needed resources may not be available to help as they may working to save their lives and property.  The contact information for these partners should be included in the plan and stored with each team member.

Backups Checks

Backups are your last line of recovery, so always know the status of your backups and validate your backup selection lists.  If there are issues with your backups, addressing them and ensuring the completeness of the backups should be considered a high priority.

Test Your Backups

There is nothing more frustrating than going to restore your data in the middle of an emergency situation and finding out that they cannot be restored due to some issue, so perform regular test restores to ensure the integrity of the media and backups.

 Portable Media

Have your backups on multiple media and stored offsite in case the primary facility is destroyed.  The 3-2-1 rule for backups is considered a way of ensuring there is always a copy of restorable data:

  • Have at least three copies of your data.
  • Store the copies on two different types of media.
  • Keep one copy offsite.

In the event of a severe emergency, there should be an individual whose responsibility is to take the portable media out of the area.

DR Replication/DR/BC Testing 

If you have a DR site, check your DR replication status well in advance of the disaster and perform cut over tests several times per year.  If your DR replication is not up to date, it can sometimes take days or weeks to bring the environment back into a complete status.  As such, it is a good idea to monitor this status regularly.

Shutdown/Restart Sequences

When the time comes, have a shut sequence in the plan with assigned resources.  When the disaster passes, there should also be a restart sequence with assigned resources.  Remember to wait until power is fully restored before plugging equipment back into the power grid.  Many times the power copies will bring power up and down during these events which can result in surges which destroy equipment.  Waiting a couple of hours to ensure that power is fully restored can says thousands of dollars.

Protecting shut down equipment

If it is not feasible to move the equipment out of the region, unplug the equipment, label it and move it away from windows towards the center of the building, elevate it off the floor and cover equipment with Plastic.  This will provide some protection from wind and water in the event that they breach the building. 

Insurance Contact Information

With our every increasing reliance on technology, most of our records and information are stored electronically.  Should the worst happen and damage does occur that prevents your ability to access that information, having your insurance information (contact/policy number, carrier information) readily available in paper format will help to place your claims in a reasonable timeframe.

We'll take care of every detail.

Even if you don't know exactly what you need, our experts make it easy to talk about your project and work out the requirements. We'll quickly help frame it up and add some structure so it can be properly estimated and ultimately developed and delivered.