At the Orange County Sheriff’s Department (OCSD) in California, seven days of data stored on disk wasn’t enough to keep up with requests for data in a timely manner. But that was all the department’s storage infrastructure could handle until OCSD partnered with Data Domain to capture more data on disk and run tape out of town.
Today, using Data Domain de-duplication appliances, OCSD has access to backup data on disk for 90 days, and the goal is to reach six months, said Doug Blackburn, Exchange Administration and Backup Administrator for the county, which is based in Santa Ana, Calif.
With a longer retention period for backup data on disk, OCSD would be better able to meet court requirements for e-discovery, for example, eliminating the need to search through backup tapes.
OCSD has 4,000 users, as many as 5,500 including other accounts. The sheriff’s department is comprised of six main organizational areas known as Operations, Custody, Investigations, Special Services, Court Operations and Reserves.
Up until recently, the sheriff’s department held seven days worth of data on disk and then moved it to weekly tapes, then monthly tapes, until, finally, the data was overwritten. “Our limitation was the amount of disk space we had,” said Blackburn.
Using tapes for data protection made requests for data tedious and costly. The department knew there had to be a better way to manage its data.
Data Piles Up
OCSD backs up about 1.5 terabytes of data every night, from file system storage and documentation, primarily using CommVault for backup, a systems it’s used for the last three years. However, off-loading data stored on disk — about 2 terabytes of space on an EMC Clariion storage system — to tape after just seven days put the organization at a disadvantage when requests for retrieval came in, about 10 times a month on average.
“Going back to tapes to retrieve and restore data was timely and expensive, taking anywhere from one to three days or longer,” said Blackburn.
Tapes, which are kept at another county site, would have to be shipped to the organization’s main data center, combed through, and then shipped back. Also, the tapes could be read only on the tape drive they were recorded on. In some cases, the tapes had to be requested from Iron Mountain, where OCSD would keep tape stored for long-term retention. “There was also a cost associated with getting the tapes from Iron Mountain,” he said.
Time for a Change
The county’s IT department reached a point where it was time to consider new options in order to achieve longer data retention on disk. The options they considered included buying more storage, looking at compression technologies and virtual tape libraries (VTLs).
Buying more storage capacity was easy to calculate. If OCSD bought 80 TB of storage, there was more than cost to deal with, such as increased power consumption, more rack space and more storage to manage.
So it was an easy decision to do more research on compression technologies, which some of the organization’s technicians heard about from their peers, and VTLs.
As it turned out, VTLs, according to Blackburn, didn’t provide the value for the money that OCSD had hoped for. “As far as value goes, it was expensive and would be time-consuming to manage,” he said.
While the department never had any face-to-face meetings with VTL vendors, it did participate in an online Web seminar to learn about the technology and gathered information on several vendors’ products.
The next step involved contacting Data Domain, a vendor that was recommended by IT peers for its reputation and service.
Blackburn and his staff learned about Data Domain’s DDX Array Series that features the DD560 and DD460 controllers and is designed for enterprise data de-duplication and protection. The vendor’s in-line de-duplication technology reduces data down to its raw essentials by pooling redundant patterns within a file, across files, and even within a block, and stores only unique data segments. Local compression scans the unique data sequence across a local small window of comparison like a tape drive.
The county set up a pilot using the DD560 that ran for 15 days. “We installed it in our data center and pulled in backups and data,” said Blackburn. The DD560 attaches to the CommVault Media Agent and presents itself as a magnetic library, according to Blackburn.
The setup and configuration took about an hour and was done by a Data Domain technician.
According to Blackburn, the de-duplication appliance met the department’s data storage needs. In March 2007, the organization installed two DD560s, each with 5 Terabytes of capacity, one in the main data center and a second appliance for remote replication.
The original unit backs up 80 Terabytes of data, which remains on disk for 90 days at the OCSD. The department expanded the original configuration in November, adding another 5 TB module. Now OCSD can store 160 TB of data on disk.
Blackburn notes that from a cost and management perspective, the Data Domain solution made the most sense for the county. Day-to-day management is done by a senior systems engineer, who logs in through a console. Statistics and storage info can be access via the Web. “Basically, we set it up and forget about it,” he said. The appliance generates daily management reports.
Backup to disk has proved to be far more advantageous for OCSD than tape, allowing the IT department to respond to inquiries quickly. Still an Iron Mountain customer, the department only ships one set of data monthly after eliminating the weekly tape backups.