8 Data Machine Learning and AI Storage Tips

It’s inevitable: machine Learning and artificial intelligence (AI) are coming. Most experts believe these technologies will severely disrupt the data storage world. So what should storage managers be doing about it? Here are some of the top tips:

1. Get Strategic

Organizations considering an investment in AI or machine learning first need to think strategically about the underlying storage infrastructure. That’s the opinion of Matt Kixmoeller, vice president of product, Pure Storage.

His logic is easy to follow. AI and machine learning are data-intensive, and organizations want to discern immediate value from that data. This means enterprises need a solution that is scalable and cost-effective, but that can also handle enormous data sets at high speed. That means deploying a hybrid of on-prem and cloud-based storage solutions. On-prem is there for performance and predictability of cost. The cloud, on the other hand, can quickly scale up and down in dev/test environments.

“There is no one-size-fits-all for every workload or application – they have massively variant requirements,” said Kixmoeller. “Successful organizations, particularly those in bleeding-edge tech and science fields, need to ensure infrastructure is a core component of their innovation strategy.”

2. Understand Differing Data Management Requirements

It would be so convenient if all data requirements were the same. In an ideal world, the differences between object, file and block would go away, and everything could be easily managed as one. But that is not the reality of modern storage. It’s the same with AI and machine learning.

“Users need to understand the differing data management requirements for business, human-generated and machine data, and invest in storage technologies that are built for the class of data that you are working to manage,” said Rich Rogers, senior vice president of IoT products and technologies, Hitachi Data Systems.

He pointed that out the needs of machine data, for example, are quite different from other kinds of data. For example, machine data will require an immediacy of processing at the edge and a scalable, shared repository at the core. So the type of storage that may have traditionally been deployed in a particular organization may not fit a machine-learning environment.

3. Think about I/O and CPU Needs

Greg Schulz, an analyst with StorageIO Group, agrees that it is important to understand fully the data being used or needed for the AI and machine learning tools. But he takes it further. He said that IT managers must also review the eventual data repository or database as well as key-value needs, along with I/O and CPU compute requirements.

4. Consider Overhead

For automated data and storage management using AI, Schulz added that organizations should take into account the amount of overhead being consumed by number crunching, analytics and the immense amount of data transportation incurred. In some cases, this might be a purely local concern affecting only one or two systems. But in other cases, the toll may be exerted across many systems and multiple sites.

5. Don’t Hedge

There is no hedging of your bets when it comes to machine learning. It’s a bit like the late nineties and early 2000s. Enterprises heard the mantra, “Go to the Web, young man.” Most paid it lip service at best and posted a one-dimensional website without any kind of e-commerce capability. A few years later, juggernauts like eBay and Amazon took out the rug from many industries.

Similarly, with machine learning, there are no half-measures. It is happening. Competitors are adopting it. So be prepared.

“AI and machine learning technology will become mainstream; it’s not a question of if, but when,” said Michael Tso, CEO, Cloudian.

Internet and retail search engines already complete searches based on what they think you might want. They knows your buying history data — what you have searched and what you eventually purchase. Five to ten years from now, companies that are not AI-ready will fall by the wayside, predicts Tso. It will be the new norm, feeding organizations critical business info about their consumers and how to reach them, he added.

6. Deploy Object Storage

While some encourage enterprises to adopt software-defined storage more deeply as a way to be prepared for machine learning, Tso takes a different tack. He believes object storage is the way.

“For those wanting to invest in AI or machine learning, don’t look at object storage as the ‘cheap and deep,’ but rather as the center of your differentiator in the future,” he said. “The learning that is going on today will feed the AI engine tomorrow. The IT world is shifting and the winners will keep data in a format that is AI friendly.”

7. Get Ready to Scale

Obviously, machine learning and AI are going to greatly increase storage requirements — as they have already. Expect storage demands to continue to surge. So scalability of infrastructure is a vital consideration when considering infrastructure upgrades or refreshes. But what kind of scalability?

“The larger the project, the more likely it is to need a storage system that can scale to large-capacity, high-density building blocks to reduce the number of devices and data center overhead as the project grows,” said Laura Shepard, senior director, product marketing, DataDirect Networks.

Her advice is to search carefully for scalable, high-performance storage that has intelligence built-in to manage flash and active archive. Look for something that can handle your medium and high potential outcomes in a system that can grow performance and capacity without needing side-by-side silos.

8. Keep Calm and Plan

With all new waves of IT, it is all too easy to get caught up in the frenzy and rush into an unwise deployment. Certainly, there are steep penalties for those who ignore this developing trend. But at the same time, too much haste can lead to mistakes — and these can be very expensive.

So even when upper management calls urgent meetings to develop an AI or machine learning strategy and gives tough deadlines for execution, stay calm and plan well.

“Have a good grasp of what it is that you are looking for the AI tool to accomplish as well as address,” said Schulz. “If you cannot articulate the problem or issue, you cannot correctly deploy storage or machine learning technology.”

In conclusion, because AI and machine learning are such nascent fields, more questions than answers remain for organizations looking to benefit from these technologies. Further, it is impossible to accurately predict the consequences for storage — both intended and unintended.

“Gain technical input from unbiased parties on the options for best-in-class storage deployment,” said Jeff Fochtman, vice president, global marketing at Seagate. “And have a future vision of where you want to go long-term and build to that.”

Drew Robb
Drew Robb
Drew Robb has been a full-time professional writer and editor for more than twenty years. He currently works freelance for a number of IT publications, including eSecurity Planet and CIO Insight. He is also the editor-in-chief of an international engineering magazine.
Get the Free Newsletter!
Subscribe to Cloud Insider for top news, trends & analysis
This email address is invalid.
Get the Free Newsletter!
Subscribe to Cloud Insider for top news, trends & analysis
This email address is invalid.

Latest Articles

What is Flash? NAND vs NOR

This article compares NAND and NOR types of flash storage and highlights which is better for different use cases.

Vulnerability Scanning, Vulnerability Assessments, and Vulnerability Management: What You Need to Know

Vulnerability scanning, vulnerability assessment, and vulnerability management offer different approaches to security. Learn how they work together.

Cloud Storage Pricing in 2023: Everything You Need to Know

Cloud storage pricing is complex. Learn the basics of cloud storage pricing and how to find the best deal for your business.