The Real Cost of Storage

This is the first in a two-part series that looks at the real cost of storage. The cost of storage is not just the cost of the drives and the RAID controller, but also includes the power costs for the drives and the cooling of the air because of the heat the drives create, among other costs. The impetus for this article came from a discussion I was having with a customer on the real cost of storage, since they were planning to move to an all SATAenvironment, but had significant performance requirements.

When I started writing this article, my premise was that the cost advantage of SATA drives over Fibre Channel (FC) drives was not that great in environments with performance requirements. Often when you write an article, you are trying to prove a point, but since this is more of an exploration, I will try to be as objective as possible to see if I am correct in my assumption. In the next article, we will see if I am correct when the facts are analyzed. Keep in mind that I am talking about SATA and FC drives in a performance environment, not where density is the primary consideration. Let’s see if my hunch was correct.

SATA Obsession

Configurations using SATA drives are all the rage, given the lower cost of these drives. I have seen SATA drives used in the enterprise initially for D2D backup, but now I am starting to hear about and see sites using SATA drives for all storage requirements because of the cost savings. SATA is starting to be used for mainline storage for large databases and for just about everything FC drives are typically used for, but that comes with some reliability concerns.

To respond to those valid concerns, RAID vendors had to develop controllers that supported RAID-6, allowing for multiple failures by having two drives for parity instead of one. Because of the additional drive, you will need more bandwidth between the controller and drives than with RAID-5. For example, using the information from the Seagate Web site, for 750 GB SATA drives the average transfer rate is 78 MB/sec. If you have RAID-5 8+1, you need 702 MB/sec to run the RAID set at rate, but with RAID-6 8+2 you need 780 MB/sec, or about 10% more bandwidth, to complete the rebuild at full rate. The same full rate issues are true for streaming I/O for video, audio or any application that requires bulk reads or writes. One area that people seem to forget requires bulk rates is database re-indexing.

Almost everyone is at least looking at SATA drives, but given the bit error rate compared to FC drives (see Restoring Data That Matters), and the fact that for IOPS they are much slower because of the lower RPM rate and longer seek time, sites are still buying FC drives, but the trend is changing and I am not sure this is a good idea.

Meeting Performance

Let’s get to the details of my conjecture. I assume that there are environments that require a certain level of performance for streaming I/O, documented in MB/sec or GB/sec. To achieve this performance you need the following hardware components:

• Memory bandwidth with enough performance to meet the I/O requirements
• PCI bus with enough performance to meet I/O requirements (see A Historic Moment for Storage I/O)
• Storage connections such as FC HBA (the most common connection in the enterprise by far), USCSI, SATA, HCAs, or a NIC
• Switch ports such as a FC switch (the most common), InfiniBand switches, or Ethernet switches
• RAID controllers which have either an FC, IB, SATA, or Ethernet interface
• Disk drives which today have an FC, SATA or up and coming SAS interface

You need enough of each of these components to meet the performance requirement, since any single component could become the bottleneck. For example, if you have 4 Gbit FC and a RAID controller capable of transferring data at 4 GB/sec, and you only have two HBAs, with two ports each, you could only transfer at 3.2 GB/sec, assuming full duplex transfers, since each HBA port is capable of running at 800 MB/sec (400 MB/sec each for read write). Assuming that enough disk drives were configured on the controller to run at 4 GB/sec, and all of the other hardware components were not a bottleneck, the HBAs will be the bottleneck.

So what is needed is a system that is balanced to meet the performance requirements. To have that type of system requires hardware at each of the areas that meets the requirements: memory, PCI bus bandwidth, HBAs, switch ports, RAID controllers and, of course, disk drives. Each of these components requires power, cooling and, which costs money, but this O&M cost is not going to be considered in my assessment since I do not have cost numbers for this area.

Drive Costs

To achieve a required storage performance requires a balanced architecture. Since SATA disk drives are slower and less reliable than FC or future SAS disk drives, more disk drives will be required to meet the bandwidth requirements, more RAID controllers will be required since more disk drives will be in reconstruction, more switch ports will be required since more HBAs will be needed to read and write, given the drive performance and reliability issues.

All in all, more hardware will be required. The question that has to be answered is, with the cost of this hardware and additional power and cooling costs, will SATA prices still be lower per MB/sec than FC or SAS drives, and if so, by how much?

Using froogle.com, the cheapest Seagate 15.5K Cheetah 4 Gbit 300 GB drive was $1,000. That means the cost per MB is .0033 per MB, but the cost per MB/sec is about $10.10. Using the same technique, the cost of a Seagate Barracuda 750 GB SATA drive is $269. So the cost for the SATA drive is 0.00035 per MB, or nearly 90 percent less. The cost per MB/sec is $3.45, or about one-third the cost of the FC drive. The cost difference for density is far greater than the cost difference for performance, by a factor of about three. As I said, cost of density is only one part of the cost equation.

In the next article, we will examine the whole cost equation and see what the real costs are over the long run. Until then, here is an interesting article on power costs to ponder.

Henry Newman, a regular Enterprise Storage Forum contributor, is an industry consultant with 26 years experience in high-performance computing and storage.
See more articles by Henry Newman.