Download the authoritative guide: Enterprise Data Storage 2018: Optimizing Your Storage Infrastructure
The enterprise data storage market is growing rapidly, and at the center of this is flash storage. Flash storage market growth is robust. Flash is becoming increasingly integral to helping both data center operators and storage vendors weather an IT landscape that is being reshaped by next-generation workloads.
Typically packaged into solid-state drives (SSDs), flash started out as a high-performance cure for the performance ills that affected disk-based storage arrays and storage subsystems in servers. As an added bonus, they consume less electricity than hard disk drives (HDDs), helping data centers and organizations that are heavily reliant on large storage infrastructures to keep power consumption in check.
The market has come a long way since vendors started shipping storage systems in 2011 that used flash as a persistent data store rather than just an application-boosting caching layer. Declining prices have also changed up the economics of deploying flash-based storage arrays in storage-area networks (SANs).
Although they remain pricey compared to HDD-based systems on a per-gigabyte basis, SATA and SAS SSD-packed arrays have drifted much closer to affordability for a growing number of businesses that don't want their storage networks to hinder their ability to provide responsive application experiences.
How Flash Drives Today's Storage Market
Given that it's a lucrative market, it's no surprise that vendors compete aggressively for all flash storage market share. But in understanding the role of flash, there are other factors to consider beyond the actual price tag.
"As the prices have come down on flash, the total cost of ownership got to the point where it was a no-brainer to buy an all-flash array," Eric Burgener, research vice president for Storage at IDC, told Enterprise Storage Forum.
Indeed, the total cost of ownership (TCO) of flash storage often tears open the purse strings, which is evident in how the flash-enabled segments of the storage market are performing.
According to IDC's latest report on the enterprise storage market, the all-flash array category generated $2.1 billion in sales during the first quarter of 2018. That's a year-over-year increase of nearly 55 percent, outpacing the overall enterprise storage market (more on that shortly).
As indicated by the IDC chart above, the five top enterprise storage vendors compete in a market where "others" -- smaller vendors -- continue to have significant share. Flash comprises about a third of the overall storage market.
Hybrid flash arrays, which combine SSDs and HDDs to provide a suitable balance between performance and sheer storage capacity, fared well, too. The segment racked up $2.5 billion in revenue, an annual increase of nearly 24 percent.
For context, the entire enterprise storage market was up 34.4 percent, year-over-year in the first quarter of 2018, hitting $13 billion. For those keeping count, the all-flash and hybrid flash segments add up to $4.6 billion, more than a third of the overall market.
"Demand for public cloud resources and a global enterprise infrastructure refresh were two important drivers of new enterprise storage investments around the world," stated Burgener's colleague, Eric Sheppard, research vice president of Server and Storage Infrastructure at IDC. "Solutions most commonly sought after in today's enterprise storage systems are those that help drive new levels of datacenter efficiency, operational simplicity, and comprehensive support for next generation workloads."
As it happens, flash storage ticks many of those same boxes.
These figures from research firm Dell'Oro Group reveal that Dell EMC is the leader in the all-flash array market.
Today's Leading Flash Vendors
Many of yesterday's top storage vendors also happen to be today's leading flash storage providers, judging by Gartner's signature Magic Quadrant (MQ) report. This is due, in part, to a wave of market consolidation that saw flash startups get gobbled up by industry giants.
- Dell EMC
EMC, now a subsidiary of Dell Technologies after the 2016 mega merger, snapped up flash storage innovator XtremIO in 2012. Today, XtremIO's block-friendly storage technologies serve niche market segments while all-flash VMAX and Unity systems serve the block and file needs of enterprises. Even Isilon's file and object product line offers an all-flash option.
In late 2015, NetApp announced that it was acquiring all-flash storage systems maker SolidFire. Providing guaranteed application performance, SolidFire storage nodes are now part of NetApp storage portfolio that includes the EF-Series that runs the company's SANtricity software platform and the high-end FAS A-Series.
Nimble Predictive AF (All Flash) arrays have gifted Hewlett Packard Enterprise (HPE) with many satisfied customers, according to the research firm. Acquired by HPE in 2017, Nimble's predictive analytics technologies enable storage administrators to find and fix issues before they become application-crashing problems.
On the lower-end of the market, 3PAR arrays provide a good balance between capacity, IOPS (input/output operations per second), and of course, price. HPE, before its split from HP Inc., acquired 3PAR in 2010.
IBM's all-flash storage offerings run the gamut, from latency-busting FlashSystem V9000 arrays for enterprises with demanding performance requirements to the Storwize V7000F for smaller organizations looking for snappier applications experiences and storage services. All told, IBM has an all-flash storage solution for practically all types of environments, from financial firms that run mainframes to technology startups that need a storage system that keep pace with rapid innovation.
- Pure Storage Goes It Alone
Among top storage vendors, Pure Storage stands out because it only produces all-flash arrays and has yet to be snapped up by the Dell EMCs and HPEs of the world. And the company is not shy about its place in today's cloud-enabled data center.
The company's "FlashArray family of all-flash block storage arrays are deployed all the way from tier 1, mission-critical environments running business and transactional workloads such as Oracle, SQL and SAP, through to deeply consolidated and automated private and hybrid cloud environments," Jason Nadeau, vice president of product marketing at Pure Storage, told Enterprise Storage Forum's Drew Robb.
The All-Flash Storage Market's Future
What will the all-flash storage market look like in the next few years? Expect some major changes that will help push flash further into the enterprise mainstream.
Non-Volatile Memory Express (NVMe) is a game-changer, prompting storage vendors to shake up their product lines with alarming speed. Compared to traditional storage architectures, NVMe is catching on like wildfire. Looking back at 2011, the enterprise storage market was dominated by SCSI, reminded Burgener. Fast-forward to 2016 and 2017, vendors began exploring their NVME-based options. By the second quarter of 2018, most major enterprise storage vendors have announced all-NVMe versions of their arrays, he added.
The reason NVMe is driving such keen interest is the protocol's latency-lowering capabilities, enabling SSDs to reach their full potential. "NVME is much better suited to flash than SCSI was," Burgener said. "There are several performance vectors that NVMe addresses, including bandwidth, IOPs, and latency. For example, the maximum IOPs possible for Serial ATA was 200,000, whereas NVMe devices have already been demonstrated to exceed 1,000,000 IOPs," according to NVM Express, the industry group charged with advancing the technology.
NVM Express is backed by a who's who of leading data center technology suppliers and IT companies, including Cisco, Dell EMC, Intel, Micron, Microsoft, NetApp and Western Digital, to name a few." By supporting PCI Express and Fabrics such as RDMA [remote direct memory access] and Fibre Channel, NVM Express can support much higher bandwidths than SATA or SAS (e.g., a PCI Express Gen3 x4 delivers 4 GB/s)," added NVM Express.
With performance claims like those, it's no wonder that both all-flash array makers and their performance-conscious business customers are aggressively pursuing the technology. Looking ahead, Burgener said NVMe "is going to be required to support the next leap in storage," namely storage-class memory products like 3D XPoint from Intel and Micron. This type of persistent memory with performance characteristics that edge ever closer to DRAM.
Self-Managed Data Storage
Since many of all-flash arrays are based on traditional storage architectures they have inherited most of their shortcomings, mainly a high-maintenance approach to storage administration. The latest breed of flash systems adopt a software-defined storage (SDS) approach to storage management and are being "built under the assumption that we're not having specialized storage admin" pulling all the strings, said Burgener.
Rather, an IT generalist can log into a console, input the desired performance characteristics they want for their storage workloads and the array takes care of the rest, automating many of the configuration and tuning tasks that once used to take up a storage administrator's day. "We are seeing more all-flash arrays introduced that are built on that idea," said Burgener.
Developing Standard: Flash-Filled 2U24 Storage Arrays
The combination of Flash and SDS also helping to giving rise to a developing standard in enterprise storage arrays, 2U24. Buoyed by storage startups that are popularizing the SAN building block, 2U24 systems incorporate a dual-controller architecture and up to 24 2.5-inch SSDs into a 2U form-factor.
This approach has enabled innovators in the space to contract with ODMs like Foxconn, place their software on the system and compete against proprietary designs. Massachusetts-based Kaminario is known for its 2U24 all-flash systems. NetApp, too, is moving to 2U24 in its latest system, Burgener said.