Businesses Report Systems Glitches from "Big Blackout" Page 2


Want the latest storage insights?

Download the authoritative guide: Enterprise Data Storage 2018: Optimizing Your Storage Infrastructure

Double the number of calls for support

Meanwhile, ActionFront Data Recovery Labs received about 260 phone calls on Monday, twice the typical daily average of 130 calls. ActionFront's offices are located in Buffalo, Atlanta, Chicago, Santa Clara, and Toronto. Almost all the calls represented "new business," reports Nick Majors, company president.

"A lot of businesses were still closed on Friday. Even if a business was open, people were still trying to reconfigure systems by themselves last week. By Monday, though, our phones started ringing all day along."

Most callers reported damaged hard disk drives, but others pointed to compromised RAID systems or glitches in "advanced operating systems."

Majors suspects, though, that some reported computer troubles are not really blackout-related at all. "It's possible that some people are blaming problems on the power failure, just to cover up for their own failures," he theorizes.

CitySoft, Inc. also received twice the usual number of calls at the start of this week. David Rosenthal, president of the New York City-based IT consultancy firm, says he expected to get calls about hard drive problems stemming from abrupt shutdowns. A number of callers, though, got in touch with CitySoft about "fried" computer peripherals.

"At one company, all monitors from one particular manufacturer got fried. At another, all printers from a different manufacturer got fried. I can't really blame the vendors, though. I think there must have been some sort of a big power surge about a microsecond before blackout. This type of thing would be outside the vendors' usual design parameters."

Computer Associates has a mirrored data center — but didn't need it

Big systems vendor Computer Associates has a pre-established backup site in the Midwest that mirrors its main data center at company headquarters in Islandia, NY.

"We are headquartered on Long Island. If we'd gone 50 miles west (of Islandia), the mirrored site would have been located in New York City. So the company made a decision that if we had to go farther west anyway, we might as well put the site in the Midwest," says Walt Thomas, CA's CIO.

The Midwest site is colocated at another CA facility. Above and beyond their usual job duties, staff at the Midwest facility are trained to take over emergency backup data center operations.

As things turned out, CA didn't need to resort to the mirrored data center at all last week. The lights came on in Islandia at 4 AM on Friday morning.

Until then, CA temporarily depended upon on-site electrical generators. "If we'd run out of fuel before the power came back on, we could have then refueled the generator — although whether we could have gotten the supplies from the fuel companies is another question," says Thomas. "On the other hand, it's more costly to implement a mirrored data center than to use a generator. These are the kinds of tradeoffs you have to weigh."

An Ounce of Prevention...

All in all, disaster prevention measures and recovery systems appear to have held up remarkably well in their most widespread test in recent memory. With damage to critical corporate data apparently nominal for most companies, the tried and true adage of "an ounce of prevention is worth a pound of cure" appears to have proven its point once again.

» See All Articles by Columnist Jacqueline Emigh

Submit a Comment


People are discussing this article with 0 comment(s)