Download the authoritative guide: Enterprise Data Storage 2018: Optimizing Your Storage Infrastructure
Another year has come and gone, and so it's time once again to look back on our storage predictions for 2006 and look ahead at what might be in store for 2007. In so doing, we'll examine where the storage market has been, and try to glean where it may be headed.
We'll start with a look back at the year that was — and how I called it.
I said that 1 TB tapes wouldn't happen before the end of 2006, so I got that one right. There were just far too many delays in 2005. Maybe we'll see them in 2007.
I predicted that object storage device (OSD) shared file systems and object storage managers (RAID) integration with HSM would be available or announced. I got that one right too — Sun Microsystems has stated publicly that they are doing this as part of their DARPA HPCS program, along with LSI, even though they were not chosen for Phase 3.
Another one I got right was predicting the appearance of PCI Express in large servers with more than 24 CPUs. This was a big issue, since the memory backplane needed to be redesigned, but they got it done.
For Fibre Channel, I said 4Gb director switches would become available in the first quarter, and I also said 8Gb Fibre Channel wouldn't happen this year and won't make it in the market. I'll take credit for both calls. We're not seeing any 8Gb HBAs, although 8Gb is being used a bit by a few for consolidation of ports. It is not really having any success in the market.
InfiniBand (IB) will continue its strong growth in the storage market, replacing FC for some large clusters and other HPC environments. IB growth will accelerate, given that I/O bus bandwidth now allows IB to work at full rate with the usage of PCI Express. I got that one right — IB is growing very fast, and I think it will replace 8Gb Fibre Channel.
Finally, a lay-up, but I'll take it: Drive density will continue to slow. This is a trend that has been happening for a number of years and will continue.
And now for the ones I got wrong.
I predicted more usage of RDMA-TCP, which could change the nature of storage devices, since we might live in a TCP-IP only world even for storage if this takes off. This is not moving as fast I as I had hoped, but is still needed, especially for the WAN.
I said holographic technology might finally come to fruition by late this year. While InPhase is predicting this, they aren't in the market yet.
I thought we were finally going to get some real security for file systems and storage in general, such as encrypted file systems with real user interfaces and management that run at device rates. This is not going to happen anytime soon in a real, broad way. There are too many competing interests and frameworks.
The way I score it, I got seven right and three wrong. Not too shabby.
Looking ahead to 2007 and 2008, I see a number of hardware and software developments worth noting.
Object storage will be available from the host side in 2007, and to storage devices, including object managers that replace RAID controllers, in 2008.This will be the first major change in storage technology end to end in many decades. This will affect areas such as HSM, write reconstruction and virtualization, to name just a few.
POSIX file system semantics will be changed in 2008 to allow for things like lazy metadata updates for shared file systems (see Solving the I/O Problem). This is a pretty easy prediction, but it still needs ratification. The standards are more than 25 years old at this point and need to be updated.
Metadata allocation and management for file systems will begin to be seen as a problem in 2007 and will be addressed in 2008. Areas such as databases and other additions for file systems will address the problems of ls –l and find and the effect on file system performance.
Storage management will not get any major standards, and managing thousands of disks and RAID controllers is not going to get any easier, much less standardized. The vendors cannot get together to support all of the layers of management and reporting. The global tools that exist today do not show all of the tunable parameters or provide all the details of various error conditions.
Disk drive densities will continue to slow both in 2007 and 2008. Gotta have an easy one.
2.5-inch drives will make their way into midrange and enterprise RAID controllers in 2008. Database index file and file system metadata need high numbers of IOPs, and 2.5-inch drives are the answer.
Vendors will announce support for tapes over 1 TB in late 2007 and release them in 2008. Tape densities and performance will continue to grow at a rate faster than disk for at least a few years.
RAID controllers from high-end enterprise and midrange vendors will support 10Gb Fibre Channel, IB and 10Gb Ethernet in 2008. The first two parts of the prediction are easy, but I am going out on a limb for 10Gb Ethernet in 2008. I believe that RDMA technology will become standard in the enterprise.
With NFSv4 and PNFS, you will see the convergence of NAS and SAN devices, and because of high-performance Ethernet connections such as 10Gb Ethernet. The management complexity of large SANs is not easy, nor is the management of huge numbers of NAS devices. Given the performance limitations with NFS, NAS was not scaling to the performance of SANs, and many NAS file systems were designed around NFS. If the NAS vendors rewrite and update their file systems and use NFSv4 and PNFS, they might achieve the performance of SANs. Combine this with object storage, which is a bit farther out, and the world as we know it might change.
Big switches for both FC and IB running at rate to all ports full duplex will be announced in late 2007 and shipped in 2008. Right now, large switches are limited to less than 300 ports or so. I think this number will at least triple, given the need for consolidation.
Enterprise drive vendors will begin discussion of other interfaces besides FC. This will not happen until 2008.The cost of FC interface per drive compared to commodity parts such as Ethernet might force drive vendors to consider other technologies.
PCI-E bus interface performance will grow faster than memory bandwidth. Right now it is reasonably standard to have 8 Lane PCI-E buses, which is about 2,000 MB/sec. Given that memory bandwidth on most single boards is 6,400 MB/sec, the ratio is 3.2 to 1. With 16 Lane buses becoming available, this might be an easy prediction. It is easier to build a bus that can move data at high rates than to be able to read or write it to and from memory.
That makes four software and nine hardware predictions for 2007 and 2008. A few of them are pretty easy, but some of them are a little risky. It could all change if we get another tech downturn, but I don't see any of it changing because of the appearance of some unforeseen technology breakthrough.
Happy Holidays, and we'll look back on this list again a year from now.
Henry Newman, a regular Enterprise Storage Forum contributor, is an industry consultant with 26 years experience in high-performance computing and storage.
See more articles by Henry Newman.