Download the authoritative guide: Enterprise Data Storage 2018: Optimizing Your Storage Infrastructure
Software-defined storage defined: Software defined storage (SDS) decouples storage software from the underlying storage devices. It does this by creating a virtualized software management layer that operates above the storage hardware.
While that definition may reasonably straight forward, many players in the data storage industry debate the specifics and details of what, exactly, is the true definition of SDS.
Most people seem to agree that at first, SDS was little more than a marketing buzzword. It first came into vogue after the OpenFlow project introduced the idea of software defined networking (SDN) around 2011. As vendors like VMware began to embrace the idea of the software defined data center (SDDC), storage vendors saw an opportunity to gain traction for their products with the "software defined storage" label.
But while SDS may have originated as basically a marketing gimmick, the technology that underlies it truly is different than traditional storage hardware. More importantly, enterprises have come to realize that SDS offers substantial benefits over traditional SAN and NAS arrays.
In a press release, market research firm IDC wrote, "SDS is gaining traction because it meets the demands of the next-generation datacenter much better than legacy storage infrastructure. As a result, IDC forecasts the worldwide SDS market will see a compound annual growth rate (CAGR) of 13.5 percent over the 2017-2021 forecast period, with revenues of nearly $16.2 billion in 2021."
However, while the term software defined storage has been around for several years and a lot of organizations are investing in SDS solutions, many IT professionals still don't understand exactly what SDS is. Part of the reason for that confusion may the variety of competing SDS definitions.
As noted above, software defined storage can be defined relatively succinctly.
However, quite a few different organizations provide their own answer. For example, the Storage Networking Industry Association (SNIA) defines SDS as "Virtualized storage with a service management interface." It adds, "SDS includes pools of storage with data service characteristics that may be applied to meet the requirements specified through the service management interface."
Way back in 2013, the market researchers at IDC offered a slightly different SDS definition: "Any storage software stack that can be installed on commodity resources (x86 hardware, hypervisors, or cloud) and/or off-the-shelf computing hardware. Furthermore, in order to qualify, software-based storage stacks should offer a full suite of storage services and federation between the underlying persistent data placement resources to enable data mobility of its tenants between these resources."
According to vendor Dell EMC, "Storage defined storage implies storage software isn’t defined by the hardware it runs on — software is decoupled from hardware and can run on any industry standard hardware that can be procured. Software defined storage should be able to be consumed in choice of flexibility as a downloadable software or as an appliance-based model depending on the organization’s implementation standpoint."
And vendor NetAPP wrote, "Software defined storage (SDS) enables users and organizations to uncouple or abstract storage resources from the underlying hardware platform for greater flexibility and efficiency and faster scalability by making storage resources programmable. This approach enables storage resources to be an integral part of a larger software-designed data center (SDDC) architecture, in which resources can be more easily automated and orchestrated rather than residing in siloes."
While each of these definitions of software defined storage is slightly different, when you examine them all together a few key themes begin to emerge. These highlight the characteristics that separate SDS from traditional storage arrays:
- Abstraction. In SDS architecture, the software that manages the storage is separate from the storage hardware. In industry jargon, the software is said to be "abstracted," "decoupled" or "uncoupled" from the hardware. This arrangement is quite different from legacy storage arrays, which were sold as all-in-one solutions where the software was tightly integrated with the hardware.
- Virtualization. SDS architecture pools together storage resources and manages them as a cohesive unit. This is very similar to the server virtualization that it is now the norm in modern data centers, except that it is applied to storage instead of compute resources.
- Automation. It's possible to have storage virtualization without having true SDS. For true SDS, you also need storage management capabilities that use automation features to reduce the number of tasks that storage administrators must perform manually.
- Industry standards. SDS solutions rely on industry-standard hardware (typically x86 servers), as well as standard APIs for storage management.
- Scalability. Because SDS relies on virtualized storage and standards-based hardware, it is typically very easy to add (or remove) storage capacity and continue managing the storage as a cohesive whole.
- Flexibility. Most definitions of SDS also hold that true SDS solutions allow users to make their own choices about the hardware they use rather than tying customers to a particular vendor or type of infrastructure.
Lower Costs. For most SDS buyers, the key attraction of software defined storage is lower costs. Because they run on commodity hardware, SDS solutions have lower upfront costs than traditional SAN and NAS arrays. In addition, the automation capabilities of these products can also decrease operational expenses because they require less time from storage administrators. Also, some SDS software includes optimization capabilities that can reduce the amount of capacity needed to store data, further lowering expenses..
Flexible and Scalable. By design, SDS architecture is also more flexible and scalable than other types of storage. "For IT organizations undergoing digital transformation, SDS provides a good match for the capabilities needed – flexible IT agility; easier, more intuitive administration driven by the characteristics of autonomous storage management; and lower capital costs due to the use of commodity and off-the-shelf hardware," Eric Burgener, research director for Storage at IDC, stated in a press release. "As these features appear more on buyers' lists of purchase criteria, enterprise storage revenue will continue to shift toward SDS."
Avoid Vendor Lock-in. Also, because it can run on any hardware, SDS helps organizations avoid vendor lock-in. And in some cases, it also allows them to extend the life of their investment, preventing them from needing to upgrade for as long as ten years or more.
Eliminate Silos. Another key benefit is that it can help organizations eliminate silos. Often enterprises have added storage on an ad hoc basis as they needed it for various applications. That can result in a mishmash of heterogenous storage hardware from a wide variety of vendors. SDS offers the ability to unify management of these different storage devices, allowing IT to be more efficient.
Strategic Advantage. Last, some IT organizations choose to move towards software defined data centers (SDDCs) as a matter of strategy. For these groups, SDS is a necessary part of achieving the overall vision for their IT environment..
When it comes to purchasing SDS, customers have a lot of different options.
First, they'll need to consider how they want to purchase and deploy their SDS solution. Companies that have a lot of legacy storage hardware that they would like to integrate into their SDS solution may choose to buy a software-only product. Other companies also need to add storage capacity, and so they may choose to buy a solution with hardware and software bundled together. This option can simplify the deployment of SDS, but it might cost more than buying off-the-shelf server hardware.
An increasingly popular option is to deploy SDS on hyperconverged infrastructure (HCI). HCI bundles together virtualized compute, storage and networking infrastructure in a single appliance. Its emphasis on virtualization makes it a good fit for SDS, and IDC forecasts that HCI will be the fastest-growing segment of the SDS market with a compound annual growth rate (CAGR) of 26.6 percent through 2021.
Organizations will also need to evaluate whether they need file-, block- or object-based storage. And they also have the option to select either hypervisor-based or container-based SDS solutions.
This chart illustrating the characteristics of software defined storage highlights the many complexities in this emerging software technology.
Analysts and consultants say that SDS might not be the best option for every application. However, it does make sense for quite a few different use cases, including the following:
- Hybrid cloud. Organizations that want to share storage between public and private cloud environments can use SDS to simplify storage management.
- Storage management consolidation. Even if an organization isn't moving storage to the public cloud, SDS can help eliminate storage silos, which in turn can reduce costs and improve IT efficiency.
- Extending the life of existing infrastructure. Because SDS can run on (nearly) any hardware, organizations that want to eke a few more years out of their existing storage investments can use SDS to create a storage pool for simplified management.
- Infrastructure modernization. Organizations that are updating their hardware might also want to consider SDS, both because it offers a lower-cost option and because it can help them create an SDDC.
- VDI. Many vendors say that using SDS for virtualized desktop infrastructure (VDI) VDI offers better performance while lowering costs.
- Remote offices. Locating storage physically closer to the place where it is needed can reduce latency and improve performance. For this reason, many organizations choose to use SDS on HCI for remote offices or for field workers who need data storage on-site.
- High availability. If data needs to be available all the time, it makes sense to use a virtualized storage solution like SDS because it makes it somewhat easier to failover to new hardware — or even a new data center or cloud service — in the event of an outage.
- IoT. Many IoT applications create a lot of data — data that can be very expensive to store. SDS can lower the costs to store that data, making more information available for analytics.