Early April may mean the end of March Madness and the start of the baseball season for the rest of the world, but for storage industry folks, it's a busy time for other reasons, with spring IT conferences and symposiums and vendor earnings announcements topping the list. It also seems to be survey and research time, when we get to read all the conflicting research statistics and market trends data, some of which could get you lost and driving in circles faster than trying to follow one of the popular GPS navigation systems during rush hour on a major freeway interchange.
Then there are the ongoing benchmark wars and debates about whose technology reigns supreme or why a particular benchmark is relevant or not and why you should or should not perform benchmarking (it might be good to start with this recent Enterprise Storage Forum article by Henry Newman).
After just a few weeks of this, my mind is buzzing with metrics, measurements and calculations, particularly after keynoting at a recent computer measurement group (CMG) session, a group that really knows metrics. So with metrics on my mind, I'll share a few thoughts with you, some relevant, some not so relevant pertaining to metrics and measurements, and some interesting or not so interesting sources for more on metrics.
From a computer, storage, I/O and networking performance and capacity planning standpoint, the folks at CMG are in a class by themselves. The common theme with these folks is computer measurement: performance engineering, capacity planning, benchmarking, simulation, forecasting and associated measurements and metrics. Granted there is a strong mainframe influence, which is not surprising from an organization that is several decades young, but there is also a good dose of networking, I/O interconnects, storage across open systems, Windows, Linux, Unix, and if you keep your eyes and ears open, even iPhones and millicomputing. The CMG folks have regional conferences and events throughout the year as well as an annual symposium.
Some other groups involved with metrics include the Storage Performance Council (SPC), which is looking to extend some of its benchmarks to other workloads and also look at power-related components. Check out what Craig Parris of Seagate had to say at the CMG international symposia in December on metrics and measurements.
SPC and other benchmarks (SPEC, TPC, ESRP, and so on) face their share of debates for and against, particularly the SPC. Some of the debate comes from vendors, usually low-end or startups, who appear to be afraid of how their stories will look if they ever have to show performance results in a benchmark, so instead they play the "we have more disk drives and thus performance has to be better" card and that benchmarks are bad and misleading or that their target market customers don't need performance.
EMC's Benchmark Dilemma
Then you have industry giant EMC, which on one hand downplays benchmarks and on the other hand publishes their own and participates in others such as Microsoft ESRP (Exchange Solution Reviewed Program), which means that for various reasons, there are some benchmarks that EMC doesn't like and others that they do. Ironically, one of those EMC doesn't like appears to be SPC, since it is a measure of how well a controller can use the back-end disk drives, a test that some vendors who claim a lot of disk drives and thus should have good performance don't want to test since it will reveal their issues.
In the case of EMC, theirs is a different debatable challenge, in my opinion, in that their flagship DMX is designed and architected around cache-centric workloads that leverage the efficiency and effectiveness of their caching algorithms (the same with the high-end Fujitsu Eternus, HDS USP, HP XP and IBM DS8000), so I don't expect to see the DMX in an SPC benchmark any time soon. Ironically, Fujitsu, IBM, Hitachi OEM HP (XP), HDS and their reseller Sun have all submitted SPC results, and if you want to see EMC DMX performance, go check out the Microsoft ESRP at technet.microsoft.com/en-us/exchange/bb412165.aspx.
It should also be pointed out that EMC midrange Clariion storage systems appear on the SPC results page courtesy of NetApp. As an aside, HDS likes to make noise about EMC's lack of participation in SPC, yet HDS has yet to make a submission for their own midrange and entry-level products, the Tagmastore AMS and WMS storage systems.
Given all the virtualization hype and debate about aggregation and pooling, including where the best place is for virtualization, in the storage system per HDS or in the fabric per others, if SPC is in fact a measure of how well a solution can use back-end disks without being a bottleneck, then what better way to see how, for example, an EMC Invista compares against other software appliances or storage system aggregation solutions. We'll never see it, at least publicly, but what a way to finally see which virtualization approach truly has the lowest latency, best performance, the least overhead and maximizes the use of back-end storage as opposed to becoming a bottleneck.
Speaking of measurements, if you have never seen or heard Willis Willington, the affable Scotsman from Seagate, and you are interested in disk drives and storage interfaces, be sure to take in one of his presentations at SNW or other events. Not only does Willis cover the usual and unusual storage-related metrics, he also covers some really extreme metrics like the disk drive "bogie wogie" and "jiggle wiggle," among others. The technology portion is great, the entertainment value even better; a must-see event.
Green Metrics
The Green Grid continues to chug along, expanding and revamping on the metrics they floated to the industry about a year ago, while the SNIA folks are busy working on metrics from a storage networking standpoint. Speaking of green IT and related topics, check out the metrics over at the Carbon Disclosure Protect. Many other groups are busy at work on metrics. Coming soon: EPA Energy Star for servers, and just behind that, the Energy Star for storage series.
The carbon disclosure project folks are taking a different approach by reporting on the footprint and what companies are doing, a gauge or metric of how a company is addressing their environmental footprint.
If that's not enough metrics for you, try combing through the CIA's annual world almanac and fact book or visit the APQC site to find more.
Speaking of green metrics, I have posted several at www.greendatastorage.com/metrics.htm, along with links to various power and energy-related calculators and sizing tools. I will also be discussing several metrics and related topics during speaking engagements at various industry events this spring; check out my events page to learn more.
Greg Schulz is founder and senior analyst of the StorageIO group and author of "Resilient Storage Networks" (Elsevier).
Back to Enterprise Storage Forum
IE 7
Firefox 2.0
