Storage Outlook: What’s on the Storage Horizon?

Over the last year we’ve looked at different rates of increases for CPUs and storage, which show improvements in storage are clearly lagging behind those of processing power. IDC, Gartner Group, and all the industry pundits agree that storage needs will grow faster than the increase in CPU power. Even with the recession, storage is still growing as a percentage of the overall IT budget at most companies. It will likely continue to grow, and perhaps at an even greater rate than today given that:

1. Most companies are not supposed to delete data given regulations enforced by the U.S. Government, SEC, European Community, and others
   



2. Decision support such as POS, data mining, and other retail tracking tools
   



3. Every drug we take, plane we fly on, car we drive, and numerous other items that are computationally modeled have all the model runs saved both for reason number one above as well as to help with potential lawsuits
   



4. New areas have emerged with huge data volume requirements, including earth imaging, climate modeling, weather forecasting
   



5. U.S. Government applications such as Homeland Security, science and engineering, and other research have similar data storage requirements

This not just a phenomenon in the USA but is happening worldwide. So the questions are what are we going to do with all of this data and where are we going to put it?

The Recent Past

I am sure some of you heard about a company that was going to be the end all, be all of storage. It was called Terastor and was funded by some industry heavyweights to the tune of over half a billion dollars. In 1998 they were going to produce a 10 GB read/write optical disk that could write at 15 MB/sec and would soon follow with 20 GB and 100 GB disks by 2000, if my memory is correct.

As you can imagine, they failed, as did a number of other companies that were trying to develop optical technology, both tape and read/write CDs. DVD is a consumer product making some inroads to the computer storage market, but this is still a consumer product with different specifications than are required for long-term data storage.

Meanwhile, disk density has been steadily increasing every 18-24 months at about two times increase. I remember 4 GB drives in 1995, with 9 GB in 1996, then 18 GB in 1998, and now with Fibre Channel, 36 GB, 72 GB, 143 GB all spinning at 15K RPMs, and 180 GB spinning at 10K RPMs.

When will we see 283 GB drives or 360 GB drives? The trend for density doubling seems to be taking longer. Yes, every few months we hear claims of laboratories increasing disk density, but moving things from the laboratory to the production lines often takes a long time.

Another area that has yet to catch up is software protocols. The SCSI standard was designed around 32-bit processing, hence the “Small” in SCSI. SCSI is limited to 2 terabyte LUNs, which will become a problem with 283 GB drives, as for RAID-5 8+1, that would be 2.2 TB. And with 360 GB drives you are over the limit with a 6+1 RAID-5 configuration. The standard will change, but how soon will the server and storage vendors adopt the new standard, and how soon will you be able to upgrade your server software to support the change?

Page 2: What’s on the Near-Term Horizon (12-24 Months)

What’s on the Near-Term Horizon (12-24 Months)

Almost every storage pundit on the planet says we are just on the horizon for optical storage that will solve the world’s problems. The problem is that this has been said off and on for a number of years. Perhaps this time it’s true, but I do not see it happening within two years for the following reasons:

1. The cost for optical storage will be high initially, which limits its usage
   



2. If the cost is high, the only place it might be used is where organizations can afford the cost (large enterprises and/or the U.S. Government), which means it must be proven technology with a long shelf life and low bit error rates

This might be a boring prognostication, but what I see coming in the near-term is more of the same. I expect tape density to increase with LTO and enterprise tapes. In fact, if you look at the trend in tape density over the last few years, it is increasing at a faster rate than disk density (60 GB native for StorageTek 9940 tapes in 2000, and 200 GB native in 2002). This trend will continue given that some significant technology improvements have been made in the development of high-density tape and tape drives.

What’s also significant is the actual amount of data written on the tape with compression. The trend from some of the HPC sites I work with is significant. In one example the site was seeing 1.3 to 1 compression using the 60 GB tape drive. After moving the same data to the 200 GB tape drive, they saw 1.6 to 1 compression. That means the capacity of the tape increase was not 140 GB but rather 242 GB, or 4x over two years. Of course there are many applications that cannot be compressed, such as movies, audio, some pictures, and others, but a great deal of data on tape is compressible.

What is on the near-term horizon from a bunch of different companies is power management ATA storage. The idea is that disks running without power can be easily powered on when someone needs to write/read data. Serial ATA (SATA) drives have a power management interface that allows this, and laptop drives have had this power management interface for years.

What’s on the Mid-Term Horizon (24-36 Months)

This is a hard area to predict for several reasons:

1. Standards groups do not always follow schedules — I know of a number of products that were delayed because of the change from 1-Gig FC to 2-Gig FC. Whatever you say about a new storage technology, the interface will have to be something that follows the standards. Of course, there will be market leaders and followers, but still, this is not the 1970s.
   



2. What are going to be the tradeoffs between density and speed? Will two technologies develop that will allow each, or will one provide both? Some optical technologies are expected to provide both. Something that does not have high performance is going to be an issue given the time to migrate the data to the next product.

Of course, we will have changes in the market, and if SATA is successful in the enterprise, then the pricing difference per MB for SCSI and ATA disk drives could increase dramatically.

Page 3: What’s on the Long-Term Horizon (36-48 Months)

What’s on the Long-Term Horizon (36-48 Months)

I believe anyone who tells you they know what’s going to happen with storage beyond 48 months, unless they claim more of the same but faster and greater densities, spent way too much time partying in the 1960s. There are far too many technology hurdles out there for any prediction to come true with any degree of certainty.

Even if the technology works, getting it from the laboratory to the manufacturing floor is a big issue. TeraStor had a few working laboratory units, but they were never able to mass-produce products. Seymour Cray had a working Cray-4 in 1995 and Supercomputer Systems Inc (SSI) had the world’s fastest computer in 1993, but mass production proved insurmountable for both. The list goes on and on — it’s been happening since computing began. Remember the Babbage Calculator that could never be produced?

It would be nice if optical from companies like Inphase and others became a reality — proven technology that was:

1. Highly reliable
2. At least an order of magnitude higher in terms of density
3. Faster than tape for load, read, write, and unload

I just do not see optical becoming a reality in this timeframe, nor do I see bio-storage, nano-storage, and/or any other laboratory technology transition to production-ready technology, especially in terms of the quality needed for the enterprise.

The big question for my role as pundit predicting the future is consumer technology. If someone develops a consumer technology that is, say 100x DVD density, then I retract everything I’ve predicted. What could change the high end of the market is a viable consumer technology that can be morphed into a technology for the enterprise.

ATA disks might be a good example. IDE drives have been used in consumer and business PCs around the world for years; now given the cost per MB, every RAID vendor has or is looking at an ATA option as part of their RAID offerings for cost-effective storage. I see this as the morphing of high-end technology from consumer technology, and if another consumer technology becomes available during this period, we could see it happen again. A few important facts:

1. SATA drives in RAIDs are not fully accepted in the enterprise
   



2. The time to get the SATA drive integrated into the RAID storage has not been
   instantaneous
   



3. If SATA is successful, the SCSI and FC disk market will be hurt badly given the cost differences

Conclusions

Sites are just now feeling the pain of data migration given the decline in transfer rate-to-density ratio over the last 30 years, and the situation will not be changing anytime soon.

If transfer rates had kept up with density increases, we would be reading and writing tapes at well over 16 GB/sec. Moving to a new technology that is reliable, has good performance, and has great density will not solve all of the problems — you’ll still have the age-old interface problem. What if someone builds an optical platter that can last 100 years, but does so with a 2-Gig Fibre Channel interface? Try finding an HBA, driver, software, or anything that can read it in 10 years, much less 20 or even 50 years.

The bottom line is that it is not the storage that counts, but rather the data. Someone needs to solve the data problem, and the storage solution will come with it.

»


See All Articles by Columnist Henry Newman