Enterprise Storage Forum content and product recommendations are editorially independent. We may make money when you click on links to our partners. Learn More.
Several companies like Dell Technologies are looking to wrangle unstructured data, minimize its storage costs, and extract its hidden value with its DataIQ software.
As unstructured data approaches 80% of enterprise data that is being produced, how does Dell’s DataIQ solution step up to the challenge?
Unstructured data challenges and DataIQ
Unstructured data comes in many shapes and sizes, but it is typically defined by its searchable, analyzable information. Video archives, for instance, don’t play nice with search functions, versus carefully structured databases containing names, addresses, and phone numbers of customers. DataIQ is designed to give intelligent insights into that unstructured data. Without a management tool, this can be difficult, not just because of the nature of unstructured data, but because information is often stored across a variety of platforms, whether on-premises or in the cloud.
Data consumers add further complication and cost, as their methods of interacting with data can often vary. Lack of transparency into metadata creates additional problems, as consumers often aren’t aware of just how old or out of date data might be or how costly those files are to store. How much is a company spending on obsolete information or even data that is stored in duplicate? What data can be moved into cheaper storage mediums, such as tapes, where they won’t incur regular energy costs? These are some of the issues which DataIQ attempts to address.
DataIQ key features
Duplication Visibility: DataIQ searches all available files across all platforms, identifies unnecessary duplicates, and can assist in their removal.
Platform-Agnostic Scanning, Indexing, and Management: DataIQ is deployable across all major storage providers and with on-premises hardware. All of these units of storage are managed from a single pane of glass.
Highly Scalable: DataIQ is targeted for enterprise use and is optimized for multi-petabyte databases.
Move Data: The UI gives its users the ability to move files into better optimized storage settings on the fly.
Self-Service: Users can add their own classifications to improve their searches.
Forecasting: DataIQ analyzes past trends of data accumulation to predict future capacity needs.
Warnings: Detects anomalous signals that prelude critical cluster events.
Searchability: Data can be searched using keywords, partial file names, or through metadata.
TCO Estimates: Each file can be displayed in terms of its total cost of ownership and the energy burden it places on the greater system.
Out of Band Data: DataIQ is not a barrier to data access, and data can continue to be used through previous means.
Intuitive Displays: Average age of data, frequency of access, and other pertinent information is viewable at a glance.
DataIQ benefits
Though the program is platform agnostic, it is designed to integrate best with Dell’s scale-out, NAS-based PowerScale storage solution, so users can enlist DataIQ to monitor PowerScale’s health and performance via a single tool. Moreover, Dell makes DataIQ freely available for users of PowerScale and other Dell storage platforms, making DataIQ a cost-effective choice for those already working within the Dell ecosystem.
Not all data needs to be hot or accessible. Using insights gained at a glance from DataIQ, a user can determine which sets of data are out of date or haven’t been accessed in years. Armed with this, plus estimates of total cost of ownership, the user is empowered to offload this data into colder, less expensive storage en masse. In the enterprise realm where petabytes of legacy data can sit for decades, this can equate large annual savings. IT can better tweak storage environments, using insights to identify performance bottlenecks, growth trends, and monitor which users or clients are consuming data and resources the most. DataIQ can generate comprehensive reports to summarize the results of these efforts, displaying the total cost of dormant data and the costs recovered through data’s optimized management.
Moving around or deleting huge volumes of data can make storage teams nervous. DataIQ has auditing capabilities to give IT and data managers a view of who has requested or instigated deletions, so some record is maintained of where critical data may have gone.
DataIQ’s auto-tagging feature is an attempt to troll through unstructured data and give it some semblance of structure, making it more searchable and more amenable to analytics. The auto-tagging feature can be configured to exclude files in the trash bin, search and purge improper characters from file names, define standards for folder-naming schemes, clean up non-conformance, and tidy up junk data that had been tagged for deletion.
Compliance gap
Where DataIQ is focused on finding opportunities to consolidate, optimize, and harness the value of data, competing tools have been developed with a greater focus on life cycle management and regulatory compliance. NetApp’s StorageGRID, for instance, is built with particular focus on bringing data into compliance with regulatory bodies, such as the U.S. Securities and Exchange Commission (SEC), Commodity Futures Trading Commission (CFTC), and Financial Regulatory Authority (FINRA) requirements for record keeping.
DataIQ case study
The digital animation studio Like a Photon Creative struggled to maintain storage capacity and computing performance during a time when the company was transitioning to remote work. Unlike many of its competitors, Like a Photon was poised to make this transition when it enlisted the services of Dell Technologies, including its DataIQ solution. The company’s CFO, Rachel Bauer, was excited about harnessing DataIQ’s data analytics capabilities for data deduplication and performance optimization.
“DataIQ helps us more accurately quote and forecast our needs and growth for the studio,” Bauer says, “which translates into real savings for us and our customers.”
Armed with DataIQ and other technologies, Like a Photon Creative was able to produce two major animated features two weeks ahead of schedule — amid the government-imposed lockdowns in its native Australia.
Conclusions
The insights into unstructured data gained from DataIQ, along with its data management tools far outweigh the cost of the tool, which for many is free. Dell does charge licensing fees to use DataIQ on third-party platforms, however. For many, it’s long past time to diversify their storage tools for unstructured data, and DataIQ is a tool for the job.
