The Storage Horizon: Looking Back, Looking Ahead

Like any good pundit, at the end of October 2003 I made some predictions for 2004 and beyond. If you’re one of those people who likes to look back at old predictions to see how they turned out, then you’ll enjoy taking a look at last year’s article. I think I was more right than wrong, but I’ll let you make your own judgment.

Let’s start with a quick review of what I said last year, since much of it still holds true, and then I’ll offer my predictions on the changing storage landscape for the next couple of years.

In the one-to-two-year timeframe, I said I did not see optical storage coming — no change there, since the prediction seems to be holding.

I saw another year of the same old increases in tape density and speed. IBM came out with a new drive at 300 GB, and a drive with 40 GB native transfer and 120 MB/sec maximum data rate with compression. IBM also believes they have increased the compression of standard data to about three to one.

I mentioned that a bunch of different companies are looking at power management for SATA (serial ATA) storage. A number of companies have announced MAID (massive array of idle disks) products, such as Copan Systems.

Looking at last year’s two-to-three-year predictions, I said that this area was hard to predict, and it still is for the same reasons: standards groups do not always stick to schedules. I know of a number of products that were delayed because of the change from 1Gb/s FC to 2Gb/s FC, and there are always questions of tradeoffs between density and speed.

On the speed density front, look at what Seagate has done with its new Savvio product. And we are facing some big issues and infrastructure changes, such as 4Gb and 8Gb Fibre Channel, with an interface change required for 10Gb Fibre Channel.

In the three-to-four-year timeframe, I suggested that a consumer technology for storage might morph into an enterprise storage technology. I made the following points:

• SATA drives in RAIDs are not fully accepted in the enterprise.
• The time to get SATA drives integrated into RAID storage has been slow.
• If SATA is successful in the enterprise, the SCSI and Fibre Channel disk market will be hurt badly, given the cost differences.

I believe that we are beginning to see this problem emerge, with more and more RAIDs, even with enterprise controllers, supporting SATA drives. Ultra-SCSI drives may be going the way of the dinosaur soon, and the market impact on FC drives will not be a good thing for those of us who like their reliability and performance.

Since I was pretty general, my predictions from last year turned out pretty well. (Surely I was better than the yearly predictions you’ll read in the Weekly World News or the National Enquirer).

And Now For 2005…

As we enter into the first half of 2005, here are some things that we should see on the hardware side:

• 4Gb FC HBA availability will become commonplace.
• 4Gb FC RAIDs will become available.
• PCI Express will become commonplace in Linux clusters and on small Intel servers, replacing PCI-X.
• RAID vendors will announce bigger, faster midrange devices with 4Gb, and in some cases support for up to 10Gb. Note that 10Gb will require an infrastructure change.
• Disk sizes will get bigger, as we have been at 300 GB Fibre Channel drives for a long time. HDS has already announced 400 GB Fibre Channel drives, but we still need to see deliveries in quantity and shipment from the RAID vendors.
• Increases in tape density for high-end tapes: Imation announced in May that they have built a plant that will create 1 TB tapes, so this is a step in that direction, but will performance increase at a commensurate rate?
• Fibre Channel switch density will increase in port density, as will the bandwidth that is required for 4Gb FC.

What about software? Storage management will continue to improve, but storage management nirvana will not be reached. I define nirvana as where you can manage the servers, file system, HBAs/iSCSI NICs, switches, and RAIDs for a whole enterprise. This does not even include a single console for storage and network management.

I think we’ll see better support for host side failover for Linux and other operating systems, and support for HSM from a number of shared file system vendors that currently do not support HSM.

Page 2: Management, File Systems Lag

Continued From Page 1

Management, File Systems Lag

Two problems remain, however: management and file system technology are not where they need to be.

We are truly lacking an enterprise management framework. Yes, SNIA and others have APIs and specifications for management, and many RAID and switch vendors have followed them. Vendors that write management software are busy adding devices and features, but we are not even close to end-to-end management, which in my opinion not only includes simple things such as configuration of devices, but also includes error analysis and host and file system control. Some vendors might have a framework that works with their products, but even the error analysis is a hard issue to deal with.

One site I work with has more than 1,000 switch ports in a single SAN. Tracking HBA issues, SCSI retries, cyclic redundancy check (CRC) issues and performance issues is not simple. Maybe I should not be able to buy a commercial off-the-shelf package for a site of this size with multiple file system types, different switches, and host types, but I should be able to get closer than we are, and storage management software and network management frameworks are not getting closer fast enough.

With file system technology, I believe we will see incremental increases in scaling and performance at best. In the first half of next year, a number of companies will come out with hardware and software products that will provide security, storage performance management and remote replication. With the lack of a major increase in network performance and a major price drop, we will still have the same problems that we have today, with too much data to replicate given the data creation rate and the speed of the WAN.

Late 2005 and Beyond

The farther out in time we go, the greater the risk of making a prediction that turns out to be wrong. There are many potholes on the road to getting a product to market, including poor design architecture, ASIC design problems, software testing problems, firmware problems, manufacturing problems, supplier problems, products that are not released within the written specification, and many other possible problems. You may not be aware of it, but all of these problems happened to various vendors this past year to delay products, and they happen almost every year. So predicting which vendors will meet their schedule is difficult, but here is a list of what I see coming in the second half of 2005.

Hardware

10Gb Fibre Channel RAID devices: QLogic, McData and other vendors already have 10Gb switch-to-switch connections, so support for other 10Gb connections is not going to be a huge technology jump.

Terabyte tapes: Imation has already announced plans for 0.5 TB tapes and a roadmap to 2 TB tapes. If you can make the tapes, then companies are going to be able to make the drives.

Vendor products based on the T10 Object Storage Device standard should become available late in the year or early 2006. Now that the OSD specification has been out for about six months, we should start seeing products in a year or so. This is SOP in the industry.

We should see 600 GB ATA drives. Since HDS has already announced 400 GB SATA drives, 600 GB is not much of a stretch. This trend with larger and larger drives will continue, with SATA always being greater in size than FC/SCSI drives. This could have a long-term impact on the cost of FC/SCSI drives, since SATA drives are taking some of the market share away from FC/SCSI.

I also think we’ll see better methods for hardware integration of SAN to WAN to SAN for remote shared file systems.

On the software side, I think we’ll see: better management tools but not nirvana; better Linux cluster tools, but no silver bullet; better HBA drivers and file system options for Linux; and better shared file system integration for heterogeneous environments.

For 2006, my crystal ball starts getting a little blurry, but here are some best guesses anyway:

• 1 TB tapes, with either LTO standards, IBM or StorageTek.
• OSD shared file systems and object storage manager (RAID) integration with HSM will become available or announced.
• PCI Express in large servers with more than 24 CPUs. This is a big issue, since the memory backplane will need to be redesigned.
• More usage of RDMA-TCP. This could change the nature of storage devices, since we might live in a TCP-IP world even for storage if this takes off.
• Finally, and most importantly, we will finally get some real security for file systems, and for storage in general. This might include encrypted file systems with real user interfaces, and management that runs at device rates.

With all that said, the biggest news for next year, in my opinion, will be Object Storage Devices and the impact they will have on what we now know as file systems. Please see Let’s Bid Adieu to Block Devices and SCSI and A Storage Framework for the Future for more information on objects, block devices and SCSI.

Back To Enterprise Storage Forum