Budgeting for Disk Storage, 2010 - 2013
Those of you that read my column regularly know I look at things a little differently than some of the big name industry analysts, such as Gartner or IDC, and try to look at the reasons behind the predictions. Most of the articles I read from these two groups and others talk about budgets in terms of how much total hardware will be sold, and much of that is based on the economy. I see the problem for budgeting much differently. In my view the problem is you need to buy terabytes or petabytes of storage capacity – if you are Google maybe tens of petabytes of - and I believe there are going to be some significant technology changes for disk storage that will, in turn, impact your budgets. The way I view the problem, is that you will need to buy a defined amount of storage and that has a cost, and too many bean counters, accountants, university researchers and big industry analysts, just draw straight lines based recent history on density and cost. They make the assumption that they have solved the problem and know what the cost will be. The last time I checked, technology growth, for the most part, does not happen on a straight line. The cost per GB for disk technology flattens compared to the old technology cost per GB when it is introduced. The cost quickly gets more dense and lower and then lines flatten again at the end of the technology lifecycle.
Here is a history of Seagate Enterprise disk drives:
|Year||Size for SCSI/FC/SAS disk in GB||Height of drive||Size in inches||Est. Max. Transfer Rate MB/sec||Seek in msec avg.||Latency (msec)||RPM|
If you graph this information over time, you will get some straight lines for density increases. Over longer periods of time, however, density increases are not linear and the slope of the line changes – cost per GB drops with density increases. If you go back in history and look at this chart in terms of technology you will find that with the introduction of new technology the rate of density increases goes up, and as that technology reaches the limit the density increases slows until a new technology is introduced. We are at that point right now. The current state of the art disk technology uses perpendicular recording and we reached the limit of linear increases back in 2008. The problem is that with the major economic downturn there was little to no investment in future technology and now we have to wait till 2013 or 2014 to get any major changes, according to this report.
The two technologies mentioned in the article – heat assisted magnetic recording (HAMR) and bit patterned media (BPM) – both will provide the potential for significant density increases, which will reduce the overall cost in the storage infrastructure.
These changes will impact both enterprise SAS drives and enterprise and consumer SATA drives. The underlying physical technology and design used in both drive types is the same. Look at the massive growth in SATA density from 2003 – 2008 and the much slower density growth since 2008. How does this impact planning budgets for storage?
Budget Planning and Density
Disk drive purchases are almost always based on how much storage you want, not bandwidth, as storage volume is the driving factor. In most budgetary planning processes, I have seen the cost of storage is based on historical costs and therefore historical density increases. If storage density is not going to grow very much over the next 3 – 4 years this it will have a dramatic impact on cost, given the storage increases that will be required in most environments. This is because:
Storage costs are not going to drop as they have in the past. There will not be large increases in areal density for drives. Some of the fixed costs for SAS/SATA ASICs and disk platters, will not change and therefore costs will not drop as much as they have in the past.
If storage growth continues at the same rate at your site, you will need more disk drives that require more power. In addition, you will require more disk controllers (SAS or SATA) which also require more power.
If storage growth continues at your site at the same rate it is likely that more RAID controllers will be needed for performance, given that controllers generally have a limit on the number of disk drives that each controller can house and address.