Sometimes it takes a crisis to deal with a long-simmering problem. That’s what happened at NYU School of Medicine to force the institution’s IT department to turn to Compellent Technologies for an enterprise-class storage area network (SAN).
Up and running without incident since last summer, Compellent’s Storage Center SAN provides continuous protection of e-mail and data for the institution’s 12,000 users, according to Jeff Berliner, director of systems at NYU School of Medicine.
Among the nation’s top schools of medicine, NYU School of Medicine students and faculty rely on IT services to share research data, submit grant applications and distribute course materials. The school’s IT department manages e-mail, file servers and other administrative software for users.
A Growing Problem
The School of Medicine’s IT department operated like many academic IT organizations. “We placed few restrictions on people. Faculty had freedom to conduct their research and had few IT policies to think about,” says Berliner. What that meant when it came to storage was no quotas on usage.
“In retrospect, that was a big error in our thinking,” says Berliner, as he watched a small problem grow into big trouble.
The school’s IT group consists of about 35 people, seven of whom make up the systems team, along with Berliner. E-mail, which is heavily relied on by students, faculty and staff, was the problem that forced NYU into a new storage solution.
“We created a culture of people who saved everything and didn’t worry about cleaning up after themselves,” he says.
NYU School of Medicine has approximately 250 servers in all. The data center houses about 20 HP AlphaServers, and there are another 200 servers within the institution, mostly HP ProLiant and Dell PowerEdge, some Tru Unix, some Linux and Windows, and a handful of Apple Xserve, VMS and Novell. The e-mail server was an Alpha Tru64 Unix server. The storage architecture used at the school was a one-off solution that consisted of direct attached storage (DAS) comprised of SCSI disk RAID arrays.
According to Berliner, the school had used an omnibus e-mail server for 10 years. Beginning in the 2002 timeframe, the IT systems staff saw a strong growth in the amount of storage being used, from 20GB to over 300GB by the end of 2003. A year later, storage usage grew to 650GB.
The excessive storage usage posed no repercussions to users, but the IT systems staff saw a problem growing out of control.
In February 2005, the problem Berliner and his staff saw coming occurred. A power event in the data center caused the e-mail server to crash. “Of course, the largest partitions that held the e-mail database were impacted,” he says.
The IT staff was able to get the system back up and running, but the task took about a week. “Users were down for five days,” he says.
There’s never a good time for a server crash, but this incident occurred when students were studying for exams and faculty had National Institute of Health (NIH) grants to submit. “What this event really impacted was the school’s ability to compete in the scientific community,” says Berliner.
Time to Operate
The School of Medicine’s IT systems group was aware that the institution required an enterprise storage solution long before the worst of its troubles began. In fact, Berliner began window-shopping for a new storage solution in late 2004.
“I met with all of the major storage vendors, but most enterprise solutions were too expensive for our academic budget, and at the time didn’t support Tru64,” he says.
Berliner was steered to Compellent by CPU Options, a reseller that worked with the school’s IT department. “Compellent was the only vendor to offer Tru64 support,” he says.
Tru64 support was just one of several criteria that Berliner required of a storage solution. Other requirements were an enterprise-class system that allowed room for growth, the ability to work native with the school’s hardware and software, and snapshot capability — and all that within the school’s budget.
Not only did Compellent’s Storage Center SAN fit the bill, but the vendor’s solution was also file-agnostic, managing data at the block level.
“This feature was very important to us because the nature of our e-mail database is such that we have a trillion tiny data files, a system which is less than ideal for a file-based storage system,” says Berliner. Less than ideal file-based storage was why 300GB of data took three days to restore. “The files are poorly positioned to do any streaming onto tape,” he explains.
The good news for the IT team was that Berliner had seen the Compellent solution prior to the e-mail system crash in February 2005. So NYU was able to seal the deal with the vendor ASAP.
Operation a Success
NYU brought in the Compellent Storage Center last April. The solution consisted of a dual-controller Storage Center, or two Compellent controllers connected to two Fibre Channel enclosures, and 4.5TB of raw storage. The solution was woven together with McData switches.
With the help of CPU Options and Compellent technicians, the system was installed and in production by May. Between set up and production, data was brought over to the new system mailbox by mailbox, according to Berliner.
Except for one incident involving a disk problem on the old system that occurred three-quarters of the way through the migration and resulted in a two-day outage, the new product installation went smoothly.
“The upside to the disk problem was that we recovered the old system to the SAN,” says Berliner.
Not only is the IT team pleased with the Storage Center SAN, but also says the reporting tools are excellent. “We can see what, if any, problem areas exist, we can forecast data usage, and we have data instant replay for continuous data protection (CDP),” says Berliner.
Going forward, the medical school plans to implement the Remote Instant Replay feature on the Compellent Storage Center. Remote Instant Replay, in combination with Data Instant Replay, provides business continuity by providing remote and simultaneous replication to another location, according to the company. Berliner hopes to install Remote Instant Replay, which requires two Storage Centers to back each other up over an IP link, sometime this year.
More immediately, the school will be upgrading to a new release of the 3 Series operating system from the 2 Series it now has in place. “The 3 Series includes greater iSCSI support and will allow us to bring Linux devices onto the SAN,” says Berliner.
For more storage features, visit Enterprise Storage Forum Special Reports