Software Defined Storage: A Guide to Understanding & Utilizing SDS

Most experts agree that the data storage of the future will be software-defined. As enterprises pursue digital transformation, many are deploying software-defined storage (SDS) as a key part of their business strategy.

“For IT organizations undergoing digital transformation, SDS provides a good match for the capabilities needed – flexible IT agility; easier, more intuitive administration driven by the characteristics of autonomous storage management; and lower capital costs due to the use of commodity and off-the-shelf hardware,” stated Eric Burgener, research director for storage at market research firm IDC. “As these features appear more on buyers’ lists of purchase criteria, enterprise storage revenue will continue to shift toward SDS.”

The firm has predicted that spending on SDS will experience a compound annual growth rate (CAGR) of 13.5 percent through 2021, when revenues will approach $16.2 billion. And IDC further noted, “Enterprise storage spending has already begun to move away from hardware-defined, dual-controller array designs toward SDS.”

The researchers at Gartner have said that by 2019, approximately 30 percent of the global storage array capacity in enterprise data centers will be SDS or hyperconverged systems based on industry-standard x86. That’s a dramatic increase from 2016, when just 5 percent of enterprise on-premise data storage was on industry-standard servers.

Surveys seem to bear out those forecasts. DataCore Software’s sixth annual State of Software-Defined Storage, Hyperconverged and Cloud Storage report found that only 6 percent of enterprises surveyed were not considering or investigating SDS. For storage administrators and managers, the race is on to understand what software-defined storage has to offer and how it might benefit their companies. Experts say that those who successfully implement the new technology the fastest will be likely to gain a competitive advantage.

Jump to

Key Differences Between Traditional Storage and SDS

Many different organizations have attempted to come up with a definition of SDS. The short definition: Software defined storage (SDS) decouples storage software from the underlying storage devices. It does this by creating a virtualized software management layer that operates above the storage hardware.

But to go deeper, let’s add detail. First SDS is virtualized. Those virtualization capabilities allow users to create pools of storage capacity that can be managed as a single resource.

But storage virtualization alone isn’t enough to qualify a solution as software-defined storage. In SDS, the virtualized storage uses abstraction to separate the hardware from the storage management software. In a traditional storage array, the hardware is tightly integrated with the storage management software. But with SDS, the hardware and software are decoupled. In much the same way that virtual machines can run on any server hardware, SDS can run on any storage hardware, including inexpensive, industry-standard x86 servers.

SDS also needs a “service management interface.” That management interface provides automation capabilities that reduce or eliminate the need for storage administrators to handle many configuration and maintenance tasks manually.

The SNIA further adds that in order to be considered SDS, a solution must have the following characteristics:

• Automation – Simplified management that reduces the cost of maintaining the storage infrastructure.
• Standard Interfaces – APIs for the management, provisioning and maintenance of storage devices and services.
• Virtualized Data Path – Block, File and/or Object interfaces that support applications written to these interfaces.
• Scalability – Seamless ability to scale the storage infrastructure without disruption to the specified availability or performance.
• Transparency – The ability for storage consumers to monitor and manage their own storage consumption against available resources and costs.

These characteristics allow organizations to transform their storage infrastructure, and they enable the following benefits:

• Cost-effectiveness — Because SDS runs on industry-standard hardware instead of proprietary hardware, SDS solutions generally carry much lower upfront costs than standard storage arrays. Also, because organizations can scale their SDS pools on an ad hoc basis, they can spread out their purchases of additional capacity over a longer period of time. Given that storage costs per gigabyte decrease over time, this can lead to lower costs over the life of the solution.

• Simplified management — The management interface and automation capabilities of SDS can reduce the amount of time that storage administrators must spend maintaining the solution. In addition, SDS allows organizations to manage large pools of storage from a single interface. This further simplifies management and can reduce ongoing operational expenses.

• Scalability — Many traditional storage arrays can provide only limited storage capacity, but because SDS solutions can pool together multiple pieces of hardware — and in some cases, cloud resources as well — they can provide virtually unlimited scalability. In addition, SDS offers organizations the ability to add storage as it is needed rather than overprovisioning in the beginning.

• Flexibility and agility — SDS frees organizations from vendor lock-in and gives them the option of using nearly any hardware they choose. The software management interfaces also makes it easier to make changes to the system as business needs change.

Software defined storage adds a layer of abstraction to storage infrastructure, enabling greater flexiblity, scalability and automation.

SDS Challenges and Opportunities

As with any new technology, software-defined storage presents a number of challenges for organizations that want to experience its benefits. On the other hand, many of these challenges are also closely related to potential opportunities. Examples include the following:

DIY design and deployment:

Opportunities: One of the key benefits of traditional SAN and NAS appliances was the ease of deployment. Organizations didn’t have to worry about whether the hardware and software would work together because the vendor had already handled all the integration and testing for them.

With SDS architecture, by contrast, organizations are often left to their own devices when it comes to getting the SDS software and the hardware to work together. And while SDS software should theoretically work with any infrastructure, in the real world, it often takes some testing and tweaking to get the storage functioning as it should.

Challenges: On the other hand, the do-it-yourself approach also offers the opportunity for organizations to customize their storage for their exact needs. This could include extending the life of legacy storage infrastructure or utilizing faster storage hardware for applications with high-performance needs.

Organizations also have alternatives that can reduce the design and deployment challenges. Some vendors offer reference architectures or pre-configured SDS solutions that have already been tested and validated to work together. Because they use industry-standard hardware, these solutions are less expensive than traditional arrays, but organizations can deploy them quickly because they don’t require as much time for integration and testing.

Pre-configured hyperconverged system are becoming an increasingly popular way to deploy SDS. In fact, according to IDC, hyperconverged infrastructure is the fastest-growing segment of the SDS market with a CAGR of 26.6 percent, and revenues for this part of the market could total $7.15 billion by 2021.

Integration with existing infrastructure.

Opportunities: SDS also offers a great opportunity when it comes to integration. If organizations are willing to put in the time for configuration and testing, they often find they can integrate their existing hardware into the SDS architecture. That can reduce costs by delaying hardware upgrades and allowing organizations to purchase new hardware when it becomes less expensive in the future.

Few organizations want to rip-and-replace all their existing storage infrastructure in order to upgrade to SDS across the entire enterprise. Most wisely choose to begin with a small pilot project and expand their use of SDS over time.

Challenges: However, this approach introduces more complexity into the environment. Storage administrators now have two sets of storage resources to manage — their legacy arrays and their new software-defined infrastructure. And managing that new SDS architecture requires a new set of skills, as well as new processes and procedures.

Hybrid cloud:

Opportunities: If organizations choose an SDS solution that offers the ability to integrate with cloud storage, their SDS deployment may make it possible to manage all their storage infrastructure — both public cloud and on-premises — from a single management interface. This is particularly attractive for the many organizations that are pursuing a multi-cloud strategy.

Challenges: Many enterprises are expanding their use of public cloud services at the same time that they are considering deploying SDS. Using public cloud storage alongside on-premise SDS storage introduces more complexity and more management challenges to the environment.

Selecting an SDS vendor and solution:

Opportunity: the buying process offers a chance to learn more about how SDS works before deploying it in the data center. Plus, it gives organizations a change to reconsider their storage architecture and perhaps make new choices that can leave them better prepared for the future, particularly if that future includes moving toward a software-defined data center (SDDC).

Challenges: Nearly every storage vendor now offers an SDS product. But these products are not created equal. Some vendors put the SDS label on offerings that really aren’t that different than traditional arrays or virtualized storage, while others offer solutions with a much more unique and comprehensive set of capabilities.

In addition, SDS solutions come in a wide array of different types. There are SDS infrastructure solutions, where the focus is on the hardware, and SDS management solutions, where the focus is on the software. There are open source and proprietary offerings, and there are solutions designed for block, file or object storage. For enterprises, sorting through all those options can take a great deal of time and effort.

SDS Design: 6 Steps

In any SDS product, designing the solution is the most important — and the most complicated — part of the process. With so many different options to choose from, organizations often need help from vendors or third-party consultants to choose the best combination of software and hardware to meet their needs now and in the future.

Whether enterprises choose to get outside help or design their SDS solution on their own, experts suggest following several key steps:

1. Define your business goals. Before you even begin to look at SDS vendors and products, you should consider why you are deploying the technology. Are you hoping to reduce costs? To simplify management? To eliminate data silos and integrate your hybrid or multi-cloud environment? Do you plan to eventually move toward a software-defined data center? How concerned are you about scalability and flexibility? Your answers to those questions will clarify your next steps and bring focus to your initiative.
2. Determine your SDS deployment strategy. Once you know why you are undertaking your SDS project, you can develop a strategy that can help you meet those goals. For example, if your number one goal is to reduce capital expenses, then you may adopt a strategy that includes integrating your existing hardware into your SDS solution. But if your top goals are to simplify management and maximize scalability, you might choose instead to deploy new hyperconverged architecture that has been designed for SDS. Your strategy should also detail which applications will utilize SDS and the order for the roll-out. For instance, many organizations choose to start their SDS project with backup or block storage before moving on to other types of storage.
3. Specify your storage requirements.The next step is to determine what kind of capabilities your SDS infrastructure needs to provide. How much capacity do you need? What are your throughput requirements? Do you need SDS software with deduplication or compression capabilities? How about tiering and data protection? Knowing exactly what you need will greatly simplify the process of selecting a solution.
4. Select the product(s) that meet your needs. With your requirements in hand, you can approach vendors to see what they have to offer. Most experts recommend a healthy dose of skepticism when it comes to vendors’ claims. If their sales materials include benchmark test results or total cost of ownership (TCO) numbers, see if those figures come directly from the vendor or from an independent third party. If you can, talk to other storage professionals or consultants and learn from their SDS experiences.
5. Work with your hardware vendor to finalize your solution design. Even if you choose to use a reference architecture or a pre-configured appliance, you SDS infrastructure will likely need some minor modifications in order to fit into your environment. And if you are designing your SDS architecture from scratch, you’ll definitely need to put work into getting all the pieces and parts to work together. The hardware vendors can be an invaluable resource as most of them will have worked with other customers deploying SDS and can offer advice about what works well and what doesn’t.
6. Try before you buy. There’s really no substitute for taking the hardware and software for a test drive in your own environment. Before you commit to a large-scale deployment, take the complete solution for a test run.

Survey Data: Business Drivers for SDS

Source: DataCore Software’s State of Software-Defined Storage, Hyperconverged and Cloud Storage, 20

SDS Deployment Tips

When it comes time to implement your SDS architecture, following expert advice and best practices can help the process go more smoothly.

• Set up a testing lab. Because most SDS implementation require a lot of integration work, it’s best to have a dedicated space where you can tweak your configuration until you achieve the desired results.

• Start small. Begin with a pilot project so that you can learn from your mistakes before rolling out SDS more broadly.

• Get outside help. SDS is a new technology, which means that few storage professionals have experience deploying or maintaining it. Invest in training or consulting services to help your team get up to speed.

• Optimize over time. The fact that SDS includes automation capabilities doesn’t mean that the storage administrators no longer have work to do. To achieve maximum efficiency and cost savings, you’ll need to continue to optimize and maintain your SDS solution after deployment.

• Don’t forget about security. You’ll need to work with your security team to make sure that you have adequate measures in place to protect the data stored on your SDS infrastructure.

SDS Compliance and Security

Storage administrators should also be aware that SDS offers a very tantalizing target to hackers, cybercriminals and advanced persistent threats (APTs). Because SDS pools storage resources together, it offers attackers the potential to access a whole lot of data if they can break through the security measures protecting that data.

For this reason, it is more critical than ever that organizations deploying SDS follow best practices for data protection. That includes using firewalls, encryption, and intrusion prevention and detection systems. Organizations should also consider today’s more advanced security intelligence solutions that incorporate machine learning capabilities to identify and mitigate zero-day attacks.

The Future of SDS

What does the future hold for SDS? For starters, adoption is likely to continue growing.

However, while interest in SDS is high, the technology is still in its relative early stages of development. Look for vendors to rapidly incorporate new capabilities into their SDS solutions. While that is good news from the standpoint that products will be improving, it also means that enterprise storage managers will need to stay informed about the rapidly changing market. The vendor that offers the best fit for your needs today may very well not be your best option a couple of months from now.

Many analysts also predict that the SDS space is ripe for merger and acquisition activity. As the leading storage vendors look to beef up their SDS capabilities, some will likely opt to purchase startups with promising research or SDS products.

One thing is certain: SDS is changing rapidly, and storage professionals will need to continue to learn about the market and the technology if they want to keep their skills up-to-date.

Related Posts

What is true software defined storage?

The pros and cons of software defined storage

Software defined storage best practices