Download the authoritative guide: Enterprise Data Storage 2018: Optimizing Your Storage Infrastructure
10 Disaster Recovery Lessons from Hurricane Sandy
Now that time has passed since Hurricane Sandy struck the East Coast, businesses in the region have had time to assess the damage and begin the recovery process. Some escaped relatively unscathed, not even losing power during the storm; others will take months to recover – if they are able to do so at all.
Economists warn that Sandy could be one of the costliest hurricanes in U.S. history – even though it was far from the most powerful to hit the country. At the time of writing, FEMA had already approved $844 million in assistance. New Jersey Governor Chris Christie's office has estimated that the cost to New Jersey's economy alone will reach $29.4 billion, and New York Governor Andrew Cuomo predicts $33 billion in costs for his state.
We talked with some IT professionals who experienced the storm and its aftermath firsthand to see what disaster preparedness and recovery advice they would offer. They suggested ten lessons IT managers should take away from the event.
1. Your Business Isn't Safe
Headquartered in Melville, NY, on Long Island, FalconStor, a data protection and storage virtualization vendor, experienced Sandy firsthand. Ralph Wynn, senior product marketing manager for the company, said one of the key things IT managers should learn from the storm is that "Your business is not safe, no matter where it is located."
Hurricanes don't hit New York and New Jersey very often, but Sandy demonstrated that "not very often" isn't the same as "never." Even if you feel safe in your particular location, that doesn't mean you actually are safe. According to Wynn, disaster recovery planning typically doesn't happen until someone at a business recognizes the truth that a disaster is going to happen – it's just a matter of when.
2. Plan, Plan, Plan (and Then Plan Some More)
People on the ground in New York said that, with a few notable exceptions, large enterprises did have adequate disaster recovery plans in place. In fact, publicly traded companies are required to have such plans in order to meet their compliance requirements.
But for small businesses, the story was a lot different.
Joe Hillis is the operations director for the Information Technology Disaster Resource Center (ITDRC), a non-profit made up of IT professionals who go into disaster areas and help the people recover. When asked how many small business IT departments typically have disaster recovery plans, he said, "I've not met a one yet in my career that has one."
Hillis urged the IT managers at those small firms, "Put a plan in place. I don't care if you write it on the back of a napkin. The main thing is, identify what's important to your business and find a place to keep it safe offsite somewhere... Just have a plan to know what you're going to do."
"Plan, plan, plan," agreed Wynn. "Ask the critical questions – what happens if the administrative staff can't get to the office? What are our fail-safes? Do we have a way to move all the operations to another location? Is that alternate location also going to be affected by the event that's taken place? Do we have written guidelines on who is in charge of what? That type of planning needs to happen, and it needs to come from the top down."
3. Test Your Solutions
Disaster recovery planning alone isn't enough. The experts agreed that companies need to test the technology they plan to use in a disaster situation in order to make sure that it will perform as intended.
Sean Hull, an independent scalability consultant located in Manhattan, works with several data centers and large enterprises in the area. He noted that companies need "to do fire drills with different scenarios to find out if we lost this and we lost this, what would happen? How long would it take us to rebuild? Would we have all our data? Could we move it to another data center?"
Wynn noted that, since Sandy, FalconStor has seen an uptick in customers who want to use the firm's testing services. "They want to have a mechanism in place that allows them to run a stress test of their environments, but they don't have the equipment or the manpower to do that on their own on a regular basis," he explained.
"So they are looking to vendors such as FalconStor to provide some way for them to use their existing infrastructure or give them a mechanism where they can actually test failing over services such as Exchange or CRM through Oracle over to another building, another site, or even moving them from physical environments to virtual environments."
4. Consider the Cloud
Public cloud providers are becoming an integral part of disaster preparedness for firms of all sizes. Hillis noted that in the case of small businesses impacted by Sandy, "If someone had a cloud-based solution, they could at least go to a hotel in New Jersey or Washington or somewhere where they had power and still resume their computer operations. I'm a big proponent of doing that. It doesn't make sense for everybody, but there are a lot of businesses that it does make sense for."
Wynn and Hull both said that mid-size and large enterprises should also look at using cloud providers – perhaps even multiple cloud providers, as part of their disaster planning.
5. Don't Put All Your Eggs In One Basket
Experts recommend that companies have a failover site and/or the capability to move critical operations to the cloud so that work can continue after a disaster. The best plans often use a combination of both and include multiple sites and vendors.
Hull recommended, "Look at using multiple providers. Don't put all your eggs in one basket, so to speak. That can really make a big difference. If a hosting service like Amazon has an outage maybe in their northern Virginia data center, it can take out a number of Internet businesses. If those firms have assets and code that's also hosted at, say, Joyent or Rackspace in parallel, then an outage that affects one provider most likely wouldn't affect more than one of them."
Similarly, Wynn recommends "a two-prong approach – having your own facility or maybe leasing a facility and then a vendor such as Amazon, Rackspace, maybe partially move it to the cloud to a data center that's located somewhere else geographically."