Download the authoritative guide: Enterprise Data Storage 2018: Optimizing Your Storage Infrastructure
Open source storage has come a long way in the last few years. There are good open source offerings on the backup, mirroring, file system, NAS and storage virtualization side. It is possible to cobble together an awful lot of disks and run them at high performance without the need for state-of-the-art hardware. Even companies known for proprietary offerings like EMC (NYSE: EMC) are on board.
"EMC most often encounters open source in the form of a Linux-based host connected to our storage products," said Jay Krone, senior director of storage platforms at EMC. "Customers are purchasing Intel- or AMD-based servers and putting Linux on them to take best advantage of volume pricing on the hardware and minimal-to-no licensing costs on the software."
Krone said customers tend to add open source applications, like the Apache Web server, or proprietary products, like Oracle (NASDAQ: ORCL) databases, to those Linux-based servers to address a wide spectrum of business problems. To meet this trend, most EMC storage hardware and software products have been adapted to run in a Linux environment. For example, EMC's PowerPath family is available in Linux.
Despite the recognition by EMC and other data storage vendors, opinions differ on how far open source storage has come.https://o1.qnsr.com/log/p.gif?;n=203;c=204650394;s=9477;x=7936;f=201801171506010;u=j;z=TIMESTAMP;a=20392931;e=i
"I still wouldn't say that there were a lot of open source storage apps," said Jason Williams, CTO at Digitar of Boise, Idaho, a company that makes heavy use of Linux and Sun (NASDAQ: JAVA) open source software.
Greg Schulz, senior analyst and founder of StorageIO Group, is more upbeat about the state of open source storage offerings.
"There is a wide variety of open source storage solutions and applications from different sources, ranging from volume managers, iSCSI and NAS stacks, file systems, clustered file systems, object-based storage solutions, dedupe and compression, among others, not to mention all of the propriety or commercial solutions that may leverage open source technology embedded into turnkey solutions and products," said Schulz. "Of traditional server and storage vendors, Sun is probably the most notable and vocal around open source storage, along with many smaller startup vendors."
Sun's "Amber Road" project, now known as Unified Storage Systems (UFS) or the Sun Storage 7000 series, is built around preinstalled OpenSolaris and ZFS on x86 hardware. These units support both file and block data protocols, thin provisioning, replication, mirroring, snapshots, antivirus and analytics. An HPC version adds Linux to the mix too.
"Amber Road is essentially a NAS system that integrates inexpensive servers with open source software in an easy-to-use appliance," said David Trachy, a principal engineer at Sun. "The whole point is to get around the premium you have to pay for proprietary disk systems."
ZFS, in particular, is garnering good reviews. Offered free with OpenSolaris, it provides a high level of data integrity, as well as mirroring between sites. According to Trachy, it can be used as the basis for huge data repositories. It is already being picked up by partners like greenBytes and Nexenta Systems to build storage systems.
"Startups are using ZFS and combining it with JBODs to create different products and appliances," said Trachy. "Missing in Sun's open source lineup is FC [Fibre Channel] block-level storage and pNFS, but these will be added over time."
In addition, Trachy notes that ZFS integrates well with solid state drives (SSDs), which are beginning to gain traction in the storage world. Williams, for example, swapped SATA drives inside Sun X4500 servers for ZeusIOPS SSDs from STEC (NASDAQ: STEC) to function as a high capacity (up to 640 GB) memory cache. SATA remains his platform of choice for volume data storage.
Competition for ZFS comes from the likes of Red Hat's (NYSE: RHT) Global File System (GFS), the Linux Logical Volume Manager (LVM) and file systems like ext4 and BTRFS. GFS was first developed at the University of Minnesota as a means of offering high performance and data sharing capabilities for the Linux platform, as well as storage virtualization. While GFS is controlled by Red Hat, LVM comes in a wide range of versions in the open source community.
Open Source Storage Projects
None of the big data storage vendors are as committed to open source as Sun, so it is no wonder the rest of the field is rather dispersed among a wide range of players. In the backup arena, you have outfits like Zmanda Inc. of Sunnyvale, Calif., and Bacula Systems SA of Switzerland.
Amanda, the basis for Zmanda's backup offering, is billed as the most popular open source backup and recovery software in the world, with more than half a million servers and desktops running various versions of Linux, UNIX, BSD, Mac OS-X and Windows worldwide. Zmanda also has the Zmanda Recovery Manager (ZRM) for MySQL.
While Zmanda uses a business model similar to Red Hat, Bacula is the real deal in terms of frontier open source run by a team of devotees such as Kern Sibbald, who are now starting to offer professional services to Bacula fans. Bacula manages backup and recovery to and from tape or disk. What is endearing about these guys is the smart marketing a Dracula theme with a catch phrase that will appeal to backup veterans ("It comes by night and sucks the vital essence from your computers") and blunt honesty. The news page features the startling admission, "We recently found and corrected a serious bug in Bacula..." Oh, for such openness whenever a big IT vendor makes a snafu.
Cleversafe is another storage vendor pursuing an open source-based business model.
FreeNas.org is a free distribution that supports CIFS, NFS, FTP, iSCSI and provides RAID 0, 1 and 5. Another useful open source tool is DRBD by Linbit HA-Solutions GmbH of Austria. It is designed for mirroring of block-level data in high-availability clustering.
Open Source Storage Barriers
While the number of applications has certainly blossomed, widespread adoption of open source storage still faces many barriers, both real and imagined.
"Open source needs to be seen as more of a turnkey supported solution, even if that is what some vendors already provide, in order to overcome perceptions that open source is only for those looking to avoid costs, have the time and people to integrate, or is just one big computer science project," said Schulz.
He also believes that the very essence of open source being free gets in the way of broad acceptance.
"People tend to think that free means less value than what you might pay for, or less value and stability than for software that you might otherwise buy," he said. "Likewise, there can be a support concern or misperception that you might add a lot of cost and complexity by having to integrate the solution."
Others, Schulz said, avoid it because they are in the midst of heavy head-count reductions and have the idea that additional staff will be required to support open source. But the biggest barrier may be more fundamental. Schulz believes a philosophical shift is required for open source storage to make it to the next level it has to get past the simplistic "it's open source" value proposition and get more involved with the bigger picture.
"What I want to know is, what is the business, economic, functionality and support value proposition of open source compared to other solutions," Schulz said. "What I really want to hear is what the vendor is doing to leverage open source as part of a total solution and its overall value proposition."
Similarly, Chip Nickolett, owner of Comprehensive Consulting Solutions of Brookfield, Wisc., thinks we have yet to see the best of open source. To his mind, the big hurdle is convincing core storage professionals, who tend to regard storage as being so important from a performance, data integrity, backup and disaster recovery perspective that they are willing to spend the money on a SAN or other pricey storage hardware. They just aren't that interested in saving a few pennies on a potentially risky and to them unproven open source venture. Until that mindset shifts, he thinks open source will struggle around in the fringes of the storage universe.
"I really haven't seen much traction on the open source storage side of things," said Nickolett. "There are backup and disk management tools, and a few low-end NAS and SAN offerings, but nothing yet that has become 'viral' from a usage perspective."
So far, Sun and Zmanda, for example, report strong inroads for open source-based products, with a cost-savings message that's catching on in a tough spending environment. Sun's Open Storage program was its fastest-growing business in the March quarter. Even so, such offerings are still just a small part of the enterprise data storage market.
Follow Enterprise Storage Forum on Twitter