Cloud Storage Management: Benefits and Challenges

Data growth is putting tremendous pressure on enterprise IT, and in many cases, IT managers are turning to cloud storage to help them deal with that data growth.

In fact, when the Interop ITX 2018 State of Infrastructure Report survey asked which factors were driving the most change in organization’s IT infrastructure environment, 55 percent of respondents chose “growth of storage/data,” making it the number one response, by far. How quickly is data growing? Sixty-two percent of those surveyed said their data was increasing by more than 10 percent every year.

According to 451 Research, “Data and capacity growth continue to be the top storage problems plaguing infrastructure professionals, and that is consistent across most geographies and industry segments. Meeting disaster recovery requirements and the high cost of storage (capex) were nearly tied for second place on the list of storage problems.” Data growth and backup needs are driving up costs related to storage hardware and storage management. In response, organizations are looking for options that can help them reduce costs — and frequently, that means looking to the cloud.

In Cloudian’s 2017 Hybrid Cloud Storage and Adoption Trends report, 87 percent of organizations surveyed said they were using the cloud to store some data. However, most of that cloud storage isn’t in enterprise-focused cloud storage services. The most popular way to store data in the cloud is in a file sync and share service like Dropbox or Box, and the second most popular is within software as a service (SaaS) applications. Only about one in five (21 percent) use enterprise cloud storage services for their primary data storage.

Image Source: Cloudian Hybrid Cloud Storage and Adoption Trends

But while enterprises aren’t using the cloud for their primary storage today, analysts believe that is likely to change in the near future. According to IDC, adoption by large enterprises is a major driver of infrastructure as a service (IaaS) cloud growth, and the market is likely to see increases greater than 20 percent per year through 2020, when IaaS will account for more than a third of enterprise storage spending.

That’s a sizable percentage, but it means that the majority of enterprise data will remain in on-premises storage — which adds to storage management complexity.

Public, Private or Hybrid?

When adopting a cloud storage strategy, enterprises have three basic options for cloud architecture: private cloud, public cloud or some form of hybrid cloud.

Today, most organizations run a private cloud in their data centers. According to the RightScale 2018 State of the Cloud report, 75 percent of organizations said that they have a private cloud, and that private cloud often includes storage.

But as mentioned, a growing number of organizations also use public cloud storage services. And in many cases, they are adopting a hybrid cloud storage strategy where they can easily move data between their on-premises infrastructure and the public cloud. In fact, the Cloudian survey found that 28 percent of organizations had deployed a hybrid cloud storage solution, and another 40 percent were planning to do so within a year.

Clearly, for the time being, the hybrid approach seems to be winning, with organizations adopting the public cloud for some use cases while keeping other workloads in-house.

Cloud Storage Use Cases

When it comes to public cloud storage, some use cases are more practical and popular than others. In general, the available cloud storage services fall one of the following categories:

• File Sync and Share Services that allow users to upload files and photos for easy sharing and collaboration have become a big hit with consumers, and many of them first found their way into the enterprise as shadow IT. Vendors followed up with enterprise-focused offerings, and these have become the most popular type of cloud storage among large organizations. As previously mentioned, the Cloudian survey found that 49 percent of organizations use services like Dropbox or Box.

• Backup and Disaster Recovery Many storage experts recommend keeping at least one copy of backup data off premises, and the cloud offers an easy and inexpensive way to accomplish that goal. As a result, cloud backup services are often one of the first ways that organizations begin using public cloud storage. In the Interop ITX survey, 37 percent of respondents said that they use cloud architecture for backup and recovery, and in the DataCore 2017 State of Software-Defined Storage, Hyperconverged and Cloud Storage report, a similar number (33 percent) said that they were using cloud storage for backup. In the Cloudian survey, 64 percent of respondents said that they either were using cloud services for backup or would like to do so.

• Archive The leading cloud storage vendors offer very inexpensive cloud storage for data that needs to be accessed infrequently, which is ideal for archived data. Often organizations set up a hybrid cloud storage strategy where they keep “hot” data that needs to be accessed frequently on premises while relying on cloud storage management software to move their “cold” data into cloud architecture. Thirty-one percent of those surveyed by Interop ITX said they use cloud storage services for archiving, and 35 percent of those surveyed by DataCore said the same thing.

• Application-Specific Storage According to the Interop ITX report, 30 percent of respondents said they use cloud services for application-specific storage. In particular, organizations are most interested in using cloud storage for their Web infrastructure (53 percent), dev and test (48 percent) and technical applications (43 percent), according to the Cloudian report. DataCore found that enterprise applications (33 percent), data analytics (22 percent) and databases (21 percent) were top candidates for a cloud storage strategy. And IDC reported that enterprises were more likely to use public cloud storage for their internally-facing IT applications.

• General Storage Using IaaS for primary storage is one of the least popular use cases for enterprise cloud storage. The Interop ITX report found that 29 percent of those surveyed were using the cloud for general storage purposes, and only 11 percent of those surveyed by DataCore said that were using public cloud services for primary storage.

Cloud Storage Benefits

The benefits of cloud storage are similar to the other benefits of cloud computing, and they include the following:

• Low costs Because of economies of scale, public cloud vendors can offer extremely low prices on storage. Public cloud storage services eliminate the need for organizations to buy and configure their own hardware, and they allow organizations to convert capital expenses (capex) into operational expenses (opex), which looks good on their financial reports.

• Simplified management When organizations use a cloud storage service, their IT personnel no longer need to configure, deploy and maintain the physical storage hardware. That can further reduce ongoing expenses.

• Speed of deployment Users can set up new cloud storage services and begin using them within just a few minutes. By comparison, it may take weeks or months to deploy new storage hardware in an enterprise data center.

• Scalability When they store data on-premises, organizations have to forecast their needs far into the future and purchase excess capacity in order to have enough space as their data continues to grow. But with cloud storage, more resources are automatically available as the organization needs them, with no need for overprovisioning.

• Availability While public cloud outages sometimes occur, in general, the leading public cloud vendors offer better uptime guarantees than most enterprises are able to achieve in their own data centers.

• Security People have different opinions about whether it’s safer for enterprises to keep their sensitive data in their own data centers or store it in a public cloud. However, there is certainly an argument to be made that the large cloud vendors with their big budgets and knowledgeable are better equipped to protect against constantly evolving threats.

Cloud Storage Management Challenges

While the benefits of migrating data to a cloud storage service can be substantial, cloud storage also poses some potential risks and pitfalls. And in many cases, the very benefits cloud storage offers are also possible challenges.

• Security While some people believe that the public cloud is more secure than in-house storage, many IT managers simply aren’t comfortable trusting sensitive data to someone else’s control. In the Cloudian survey, 62 percent of organizations listed security as one of their top cloud storage concerns, making it the most common cloud storage management challenge.

• Compliance In some cases, regulations require organizations to keep data in-house. In fact, 59 percent of survey respondents told Cloudian that they had data that cannot be migrated to the public cloud, and on average, 47 percent of their data must stay local.

• Cost While lower costs are a big driver for cloud migration, the cloud isn’t always cheaper than on-premises storage, especially if organizations already have legacy storage appliances that they could continue using for many more years. Also, many organizations that use the public cloud find it difficult to optimize costs. According to the Cloudian report, 55 percent of organizations expressed concerns about cloud storage costs.

• Cloud storage management complexity Forty percent of those surveyed by Cloudian expressed concerns about cloud storage management. While the public cloud eliminates the need to manage hardware, it can make it more difficult to make sure that the organization is enforcing its data management policies and following best practices.

• Interoperability With so many organizations adopting hybrid cloud storage strategies, interoperability with on-premise infrastructure is a key challenge for many enterprises. In fact, 40 percent of respondents in the Cloudian surveyed mentioned interoperability as a top concern.

• Vendor lock-in Once an organization begins using one cloud storage vendor, moving that data to a different vendor or bringing it back in-house becomes a costly and complicated operation. As a result, IDC found that nearly 20 percent of organizations had vendor lock-in concerns related to public cloud storage.

• Connectivity Inside their own data centers, organizations have very fast connections that allow them to access data quickly. But in the public cloud, they are often forced to rely on the public Internet, which is much slower and can result in noticeable latency for users. In the Cloudian survey, 33 percent of organizations said they felt they needed to keep some data on premises because of concerns about performance, and in the DataCore survey, 32 percent cited performance as an issue.

• Migration Connectivity is also a concern when it comes to cloud storage migration. Transferring several terabytes of data to a cloud storage vendor over standard Internet connections might take weeks. In addition, organizations going through a cloud migration almost invariably run into problems in getting their cloud storage to function the same way that their on-premise storage did.

Cloud Migration

Migration is typically one of the biggest headaches in any cloud storage initiative. In a Dimensional Research report sponsored by Velostrata, 59 percent of IT professionals surveyed said that their first cloud migration project was more difficult than expected. In addition, 64 percent said the migration didn’t meet their target deadlines and 55 percent said it exceeded its budget. When asked why their cloud migration projects hadn’t gone well, respondents pointed to time-consuming troubleshooting (47 percent), difficulty configuring security (46 percent) and slow data migration (44 percent) as the chief culprits.

In order to ease these concerns, the leading cloud vendors have rolled out an array of data migration products and services. Some are physical products onto which organizations can load their data before shipping the actual device to the cloud vendor, while others are services that ease and speed the migration process.

Experts also advise following cloud storage best practices in order to minimize the hassles involved with cloud migration while maximizing the benefits of public cloud storage.