Enterprise Storage Forum content and product recommendations are editorially independent. We may make money when you click on links to our partners. Learn More.
Amid all of the current disruption and global happenings, some technical projects and trends quietly continued in the way they were expected to. One of those has been the ongoing rise of AI-driven storage solutions.
Take a look back at the AI data storage trends of last year, in fact, and you’ll see that AI-driven storage has been an exciting area of research and development for a few years now. This is due to a variety of factors. Storage infrastructure and AI systems have a natural confluence, because the vast amounts of data required to run AI systems mean that data storage architecture is important for AI.
But this also works in reverse — just as new approaches to data storage are important for AI, a new generation of AI tools is helping to improve the security, flexibility, and efficiency of storage solutions.
In this article, we’ll take a look at these new systems, the advantages they promise to bring, and the challenges that must still be overcome to realize them.
Devising Smarter Storage
First, let’s define what we mean by AI-driven storage. This is more difficult than it sounds, because AI tools are already being used in a wide variety of different, storage-focused applications. Here are just a few:
- AI can be used to automate data placement through checking the resources available, and placing data in the right place.
- AI-based analytical software also has value at the other end of the data storage pipeline, providing insights into how your storage infrastructure is performing and suggesting ways in which it can be made more responsive or more secure.
- In fact, security is one of the major advantages of AI tools in data storage. According to cybersecurity expert Barbara Ericson of Cloud Defense, “by leveraging AI tools, you can discover any holes in the proverbial ship and fix them before an actual hacker takes advantage of unseen weaknesses.”
- At a more practical level AI tools are helping systems administrators manage the increasingly hybridized (and increasingly complex) storage infrastructure now deployed. The average organization will now make use of a number of clouds — managed both in house and public clouds — and just keeping track of where data is stored can require automated tools.
Many of these approaches rely, in turn, on software abstractions. Most AI systems work best in software-based storage infrastructures, because this approach allows AI tools greater flexibility when it comes to managing the placement of data. Accordingly, we are seeing a positive feedback loop, in which a greater appetite for software-defined storage drives adoption of AI tools, and vice versa.
Also read: Software Defined Storage: A Guide to Understanding & Utilizing SDS
New Approaches to Storage
Despite these advantages, some challenges still need to be addressed if we are going to realize the true potential of AI tools in data storage. At the most fundamental level, many of the AI tools and systems now available focus on just one small subset of the tasks and processes required for effective data storage.
This means that the current landscape of AI-enhanced storage tools remains a fragmented one. This is primarily because many AI tools are still designed to work within one type of on-prem or cloud storage environment, and most organizations now store their data across a variety of hybrid clouds.
At a less fundamental level, there are more mundane issues. Market reports indicate that just two vendors, Dell and Hewlett Packard, dominate the AI-powered storage market, and that smaller more innovative companies are having difficulty breaking into it. There is hope, though. In recent years, we’ve seen the emergence of a new storage paradigm that aims to clear the way for the more widespread use of AI tools. One plank of this approach is software-defined storage, as we have already mentioned.
Another is the obliteration of the distinction between primary and secondary storage. Newer platforms have made the existing lines between primary and secondary data blurred, and make this distinction a purely software-defined one. Alongside this shift will come another — a reduction in the necessity of performing manual data segregation.
Also read: Best Storage Management Software
Performance and Security
This new storage model is likely to have far reaching effects on the way that organizations store, retrieve, and manage archival data. In fact, it’s conceivable that the adoption of AI tools will drive a revolution in the way that data is stored. This will be most visible in two ways — the increased performance of emerging systems, and in their security.
Let’s take performance first. AI-driven software-defined storage systems are built to be much faster and more flexible than the systems they are replacing. At the same time, however, AI tools will require ever faster access to data, and this will drive the adoption of ever faster drives and storage media.
Secondly, security. We’ve written about how some organizations are using AI to guard backups against ransomware. Over the next few years, this is likely to become even more popular, and AI is likely to see use in protecting firms against a broader range of attacks. AI tools are useful not just in spotting suspicious network activity, but can also be used to perform network forensics after a successful attack.
The Future of AI-based Storage
Ultimately, it’s probable that within a decade storage engineers will routinely work alongside AI assistants in order to manage their storage infrastructure. This is a trend that the markets have already picked up on. There are several reports that indicated the global software-based storage market could increase to a $7 billion overall value in the next three years.
That means that organizations should start preparing now. Getting started with AI-based data storage is not difficult, and could end up saving you a significant amount of time and money.
Read next: Top Storage as a Service Providers 2021
