Enterprise Storage Forum content and product recommendations are editorially independent. We may make money when you click on links to our partners. Learn More.
Object storage emerged as a solution to the sudden appearance of massive amounts of unstructured data over the past decade.
Unstructured data is difficult to organize into traditional databases that favor a heavily structured approach. Social media, email, photos, videos, audio, and more modern data is largely unstructured.
Object-based storage is a smart way to archive, backup, or retain such data in the cloud or on-premises due to advantages, such as scalability and flexibility. Object opened the door to terabyte-scale and even petabyte-scale storage.
Here are some of the top trends companies and IT teams are seeing in the cloud object storage market:
1. Phenomenal growth
From nowhere a few years back, object storage has risen to prominence.
Enterprises are now storing roughly the same amount of data in the cloud as they do in house — close to 80 PB per year, according to IDC.
“Over the next five years, 80% of that data will be unstructured (i.e., file- and/or object-based) data,” said Eric Burgener, an analyst at IDC.
“Growth rates for data in the public cloud, fueled by both file and object storage growth, were higher than growth rates for on-premises storage.”
For object storage, the top workloads are archiving and content applications.
Some of the object storage features particularly appreciated by users include integration with accelerated compute platforms, quality of service, integration with many artificial intelligence (AI) tools and software stacks, and support for high-performance storage tiers that leverage flash media.
2. Analysis at scale
From the pull of data gravity to stricter data governance, the proliferation of data is continuing to surface challenges for organizations embarking on cloud transformation.
The ability to analyze data at scale, however, remains a core priority as enterprises strive to gain real-time insights and make better use of their data, said Rob Lee, CTO, Pure Storage.
As a result, organizations are increasingly deploying hybrid architectures across public and private clouds to bring the benefits of cloud-based data analytics to critical data assets across their IT landscape.
For example, to enhance the ability to harness the power of cloud analytics for on-premises data, Pure Storage has brought its FlashBlade object storage to Snowflake’s data cloud platform to provide enterprises more control and greater flexibility around their data workloads.
3. Storage optimization
We are about to enter the zettabyte age of storage.
Data stored globally will approach 6 ZB in 2022, according to IDC.
Just one ZB would require 55 million 18 TB hard disk drives (HDDs) or 55 million of the latest 18 TB LTO-9 tape cartridges.
With so much data in play, getting the right data, in the right place, at the right time, and at the right cost is one of the keys of competitive advantage. That’s where storage optimization comes in.
“Intelligent data management will be required, leveraging multiple tiers of storage, active archives, and innovative S3-compatible archive solutions for object storage,” said Rich Gadomski, tape evangelist, Fujifilm Recording Media USA.
“Nowhere will this be more apparent than in digital preservation and high-performance computing environments with a simple need to offload expensive object storage to cost-effective tape systems using an S3-compatible API.”
4. Facilitating hybrid cloud architectures
With ever-increasing pressure to save money and innovate faster, organizations are desperate for greater agility, flexibility, and efficiency in public cloud services.
As unstructured data is the largest, fastest-growing, and most difficult to manage dataset for IT, they have a strong desire to move this data to the cloud.
Unfortunately, data can’t move faster than the speed of light. To copy 1 PB of data over a 100 Mbps connection under ideal conditions would take three years. Even pushing that connection to 1 Gbps, it would take over four months to complete the migration.
Nevermind data access costs, latency, insufficient cloud expertise, and other speed bumps, the migration issue alone makes a complete cloud migration untenable for large datasets.
“Hybrid cloud storage is an attractive compromise that lets organizations keep their sensitive and frequently used data on-premises for privacy and ease of access, while moving their cold, non-proprietary to the cloud,” said Tanya Loughlin, director, Hitachi Content Platform Product Marketing.
“In many cases, customers would rather invest in an on-premise service rather than on-premise capital expense to take advantage of a pay-per-use model. For example, a customer has data (image, voice, video, audio) and the AI, ML, and recognition systems are too expensive and complex to host on-premise.”
These services are available in the cloud, and it is much more cost-effective to process large amounts of data via these services.
Instead of moving all their data to the cloud, customers can burst data to the cloud for processing, and once they have the results, cloud data can be deleted. Object storage is being harnessed to be able to store more data faster in hybrid cloud environments.
5. Cloud control
Since most data is neither sensitive nor frequently used, many organizations have chosen to offload their lower-value data to the cloud at an acceptable pace and low cost.
For important and new data, they can keep it on-premises in order to have fast writes, low-latency access, and complete control over security, compliance, and governance.
But the issue now becomes how to control what goes to the cloud and what stays on-premises. If all IT has to go on is the directory structure, file names, last access dates, and the like for their file data, they will struggle to make sure the right data is in the right place at the right time.
This is where an on-premises object store makes hybrid cloud storage a reality. Object storage puts cloud storage in customer data centers and provides the same capabilities that people look for in cloud storage: effortless data protection, reliable data availability, massive scale, easy to manage, and lower cost than traditional file services.
“The best object stores for a hybrid cloud storage architecture will have built-in or integrated tools, like content indexing, metadata extraction, storage tiering, that provide the intelligence necessary to automatically manage data across on-premises flash, disk, and removable media tiers as well as multiple cloud services — so that IT has complete visibility and control over their data, no matter where it lives,” said Loughlin with Hitachi Vantara.
