Download the authoritative guide: Enterprise Data Storage 2018: Optimizing Your Storage Infrastructure
Archiving is certainly one of the hottest areas of storage right now. It lies at the heart of a subject that is near, though perhaps not so dear, to the heart of every storage manager retaining a legion of records to comply with an ever-expanding roster of regulatory requirements.
"Compliant records data is presently estimated to grow over 60 percent per year, generating more than 1.6 PB of new storage capacity requirements in 2006," says Fred Moore of Horison Information Strategies. "This represents the single fastest-growing application segment of the storage industry."
Part of the appeal may lie in the sheer scope of the topic. Ask some vendors what archiving is and they wax lyrical about disks. Ask others and it's all about tape. Bring up the subject at a conference and you might hear about the problems of e-mail storage or the compliance demands of the modern enterprise. And yet other views may cover the importance of search capabilities, the role of instant messaging (IM) in archive policy and how archiving is a central element of any information lifecycle management (ILM) strategy.
That's why the vendor acquisition frenzy surrounding archiving may seem a little confusing at first glance: OTG acquired by Legato in 2003 and subsequently gobbled up by EMC; Educom picked up by Zantaz; KVS by Veritas and then Symantec; Computer Associates' acquisition of ilumin Software Services; Symantec purchasing IMlogic; and Quest purchased AfterMail, to name just a few. These buy-outs forward vendor roadmaps encompassing a wide range of archiving products and possibilities.
"All the various archiving options such as active archiving, ILM and tape have their place," says John Webster, senior analyst and partner at Data Mobility Group.
Finding its Centera
As the array king, it's no surprise that EMC is one of the major champions of disk-based archiving. It advocates a three-pronged attack on the archiving space: smart/fast storage and retrieval at the high end, general array-based archiving in the middle, and tape library usage at the bottom of the pile.
"Centera is the choice if online access is needed with assured content authenticity at a TCO lower than a tape library," says Mark Avery, senior director of EMC Centera. "If all that is needed is speed to information, then CLARiiON may be the appropriate solution. If neither speed to information nor assured content authenticity is needed, then ADIC tape technology is an adequate solution."
Via Centera, EMC is going all out with its concept of authentic active archiving archived information available on-line while assuring content authenticity. This comes under the broader heading of content addressed storage (CAS).
Centera is a purpose-built hardware and software storage platform for information archiving. CentraStar, Centera's operating environment, checks the authenticity of content, provides retention policy management capabilities and does single instancing, the storing of unique content only once. This platform has been strengthened with search and chargeback reporting capabilities based on technology OEM'ed from Norway-based Fast Search & Transfer ASA (FAST). CenteraSeek also allows fast cross-application advanced search of storage-based content metadata.
"Customers want to unlock the value in their long-term content, and that does not happen by storing it on tape or optical technology," says Avery. "With Centera, customers can afford to have disk that spins all the time, because it costs them no more and often less on a TCO basis than tape."
While EMC champions disk-based archiving, not everyone is convinced of its TCO claims.
"Long-term archival storage remains the realm of magnetic tape libraries," says Moore. "Today there is no truly cost-effective ILM strategy without a tape component."
Save Our Spam
Compliance is obviously a major driver of the market, perhaps more so in e-mail archiving than anywhere else.
In the past, storage managers were forced to restore and pull data from backup tapes in a costly and services-driven exercise that exposed firms to regulatory sanctions and unsuccessful litigation outcomes.
With new message management technology, there are now more cost-effective and efficient solutions to the problem. The market for messaging archiving continues to grow, driven in large part by the focus on regulatory and legal requirements to retain and discover e-mail.
"In 2005, we saw a large increase in demands for e-mail archiving as case law and regulatory pressures continued to force companies to retain e-mail as business records," says Mike Gundling, CA's vice president of product management. "Organizations are reeling from high-profile litigation and important legal priorities that are driving the need for e-mail discovery."
CA bought ilumin to fill out the archiving features of its BrightStor storage management line. Known as CA Message Manager, it offers advanced compliance and litigation support for enterprise e-mail archiving. Thus CA's strategy is to become a hardware agnostic one-stop shop for e-mail security (eTrust), backup (ARCserve) and archiving (CA Message Manager).
CA treads the middle ground between tape and disk. In its view, both play distinct roles. However, it reports that over 100 of its customers using optical jukeboxes have switched to disk-based WORM. Further, CA believes that active archiving is not a replacement for traditional backup.
"Organizations need to understand that active archiving is everything a backup solution cannot be," says Gundling. "But they need both archiving and backup."
Another vendor making a play in the e-mail arena is Mimosa Systems, based in Santa Clara, Calif. Mimosa NearPoint provides immediate mailbox and message recovery, disaster recovery, e-mail archiving and self-service search and access in one solution. NearPoint is disk-based, using SATA RAID and NAS appliances. It runs on a Windows 2003 server.
"Customers have realized that first generation e-mail archive products did not deliver value, so the e-mail archive market is now white hot," says Bob Spurzem, senior product manager at Mimosa.
The relevance of advanced e-mail functionality can be seen in the findings of a recent survey conducted by Osterman Research. E-mail management is front and center among enterprise concerns, with the top Exchange-based problems being the management of e-mail disaster recovery, the sheer size of the message stores, protecting e-mail databases and searching individual files for legal discovery.
"Enterprises are facing serious challenges across multiple areas of e-mail management," says Michael Osterman of Osterman Research. "The market is demanding solutions for holistic e-mail data management that go beyond a single feature or function."
Storing or Accessing?
The axis of attention in archiving, then, appears to have shifted. The traditional methodology dump all those old tapes in a closet or onto a rickety old array is under attack.
"Archiving is not about storage, it's about access," says Michael Howard, chairman and CEO of OuterBay Technologies, which was acquired earlier this week by HP.
Like EMC and CA, his company is talking up active archiving but with a Very Large Database (VLDB) slant. The generally bloated state of databases, it seems, has gone to a whole new level of late. OuterBay reports customers with multi-terabyte or even multi-petabyte databases. For example, one VLDB customer has DB tables of one terabyte. Another archives 3 TB of database transactions per year.
In response, the company has released OuterBay Enterprise Edition and Compliance Edition. The latter is a self-contained XML archive with added audit capabilities, data integrity features, data lineage and WORM storage device integration.
"We're seeing more and more customized applications in enterprises with large scale databases," says Ray Paquet, an analyst with Gartner. "Enterprises should look to database archiving applications to manage accelerating growth, increase enterprise application performance and meet compliance requirements."
Disk Head Debate
Many of the solutions referenced in this article advocate disk-based archiving. Yet the popular wisdom is that disks should not be used for long-term storage as the disk heads could stick. Copan, for one, claims a solution to the problem with its Disk Aerobics technology, which powers up disk drives once a month to keep them active and reduce failures.
"It is true that disk drives are not well suited to be unplugged and stored on the shelf," says Spurzem. "Long-term data storage is better suited for tape, and to optical if the capacities are a fit for the application."
He believes, though, that tape and disk can be combined into a cost-effective combination for backup and archiving. Disks supply the near-term storage and tape deals with more lengthy retention requirements.
The role of tape in the archiving hierarchy is being further strengthened by the latest feature sets. Tape technologies such as LTO and SDLT offer increased capacity and better security. Encryption, in particular, is seen as a major step forward in the usefulness of tape as an archiving medium.
"Encryption is making its way into the tape designs, positioning the tape industry for an explosion in compliance and archival applications," says Moore.
For more storage features, visit Enterprise Storage Forum Special Reports