Download the authoritative guide: Enterprise Data Storage 2018: Optimizing Your Storage Infrastructure
After 12 years of writing a monthly column for Enterprise Storage Forum, it’s time to move on.
A monthly column may not seem like a very big commitment, but I have always tried to bring readers the most cutting-edge thoughts and technology from the worlds of storage and high-performance computing (HPC). That takes time and thought and commitment of time (with one exception—an article on the death of my best friend and mentor that I wrote during a 90-minute plane ride).
I think I’ve largely succeeded in giving you the best current thinking about our industry, as evidenced by the dozens of articles that have gone viral in places like Slashdot, Reddit and Linux Today, but it seems like it’s time to move on before I start to repeat myself and bore us all to tears. My editors at Enterprise Storage Forum want me to keep writing, so I won’t rule out an occasional column if inspiration strikes. However, this will be my last regular column.
I want to leave you with a single thought about our industry and how to consistently pick technology winners and losers. This is one of the biggest lessons I’ve learned in my 34 years in the IT industry: follow the money.https://o1.qnsr.com/log/p.gif?;n=203;c=204650394;s=9477;x=7936;f=201801171506010;u=j;z=TIMESTAMP;a=20392931;e=i
The IT industry is based on technology innovation, and that costs money—lots of it. This was just becoming true when I started back in 1981 because until then much of technology innovation was funded by the U.S. government. But by the end of the 1980s, they had almost no impact. From the census to Social Security, the U.S. government needed technology to solve their problems, but the cost of funding development and the pace of change caused industry to take charge.
We recently saw IBM get out of the foundry business because of the company’s lack of return on investment. Most industry pundits say it takes $8 billion to $10 billion to fabricate a new chip, and it also requires you to have to have the right people and management to make it happen. Although money is critical, it doesn’t ensure successful teams. New storage technology is no different, and when the tablet and phone revolution of the last decade needed storage, the industry responded with investment in NAND technology that benefited both consumer and enterprise applications. I remember using NAND in the 1990s for an embedded project, but it was not until the smartphone revolution that NAND took off in terms of both volume and reduced price.
The money trail has affected all storage technologies we are using today and all technologies that we are going to be using in the future, including non-volatile technologies such as PCM, Memristor and carbon nanotube, if any of them make it in the market.
Example 1:Tape’s declining fortune
As many of you know, I have been a big supporter of tape technologies for archiving for almost three decades. I have fought back against those who prematurely declared it dead, including EMC in the 1990s. Even back in 2006, when the late Jim Gray made a seminal statement at the Storage Guru Gong Show, Tape is Dead, Disk is Tape, Flash is Disk, I contacted Jim about this and went to discuss it with him. In spring 2007, I had dinner with Jim at a restaurant in Minneapolis near where he was giving a talk and tried to convince him that he was wrong about tape. Henry Newman vs. Jim Gray did not turn out so well for me, and I even bought dinner. The fact that Jim would even have dinner with me was one of the truly eventful things in my career.
When I started to write this piece and started following the money, I decided to look at things without emotion and without past prejudices and beliefs—really look at what drives the industry regardless of what I want to happen. So when I looked at LTO media sales a few weeks ago, I decided to look at tape as part of this follow-the-money analysis, and I have to admit that Jim is starting to look right. The Santa Clara Consulting Group puts out data on quarterly media sales for LTO tape, and has the data from 2008 to today on their website. The table below shows quarterly sales since 2008:
In six years, LTO media sales have been cut in half. The drop in tape media sales is both a short-term and long-term issue for LTO which most believe is a majority of the tape market. In my opinion, enterprise tape sales depend on the LTO ecosystem for market access, library, media, drive development and, last but not least, for R&D and technology improvements. Although the sales of tape media do not represent the entire market, they do represent a very significant portion of the market, as the ratio of high-cost drives to media usual ranges from 500 to 1 to 200 to 1 in large sites. Even at a cost of $30,000 per drive, media costs are still usually far more than half the cost of a tape storage system.
LTO manufacturers (media, drives and libraries) today are likely to face significant financial pressure to provide the cost recovery of the necessary R&D for future tape generations, and disk vendor density is growing faster than LTO density, at least currently. Also, remember the size of the disk market today is over $35 billion. The size of the entire tape market is unclear, as figures for enterprise tape are not published, but it was estimated to be $2.2 billion in 2011 based on data from IDC and others. Using the above figures, the estimated size of the market today is 16 percentless, or $1.85 billion, assuming the loss in the whole tape market is consistent with the LTO media market size drop.
Folks, this does not mean that tape is going away tomorrow or next year or even by 2020. I know people that still have 7-track tapes in storage. What it does mean is that innovation will slow and it already has. Look at LTO tape and nearline disk density changes:
- 2008: 800 GB LTO compared to 4 1 TB disk
- 2014: 2.5 TB LTO compared to 6 6 TB disk (note 8 TB announced)
If LTO-7 does come out in 2015 and does come out at 6.4 TB, disk will still have a commanding lead with plans for 2015 disk density improvements. But again, tape will not just disappear overnight as it has been in the market too long and has been embedded in workflows for decades.
Example 2: De-investment
Following the money also requires looking at the flip side and following the de-investment in a technology. If customers are reducing their purchases of a technology, how can companies justify increasing their spending on R&D? Companies do not throw good money after bad forever, and at some point they just stop investing.
That is my concern about the cloud. Whether or not a technology has characteristics as good as other competing technologies (such as better hard error rates, lower costs per byte, lower costs per byte for power) doesn’t matter, as the investment in R&D is what will drive long-term density and performance improvements. That is what the market cares about.
Back in the 1990s, the same thing happened to magneto-optical drives, and the list goes on and on. Watching what a company or industry does with their money and the technology bets that are made is important to understand who the winners will be long term. Most engineering problems can be solved with smart people and enough money, and the big technology companies have both.
Just because a technology does not have large investment money behind it today doesn’t mean you are going to wake up one morning and find it gone. If there was a long-term investment in the product or technology at one point, it will take years or decades before the technology is finally replaced. So, repeat after me, "don’t panic."
On the other hand, if a new technology with limited market presence no longer has a large investment behind it, then it is time to worry, and in buying that technology you are taking a big risk. Take, for example, the 12-inch glass platters that were produced in the late 1990s and early 2000s by one company, Plasmon. If I had purchased that technology, I would worry, as it was a single vendor and no one else jumped onto the bandwagon.
I think this follow-the-money approach can be used to evaluate all parts of the storage stack for today and the future. The future suggests that we are going to have non-volatile memories. For a hint of the future, just look at what Intel is doing.
The follow-the-money principle likely applies to far more than just IT storage technology. I was driving down the street this morning and saw a sign at a gas station that the price of unleaded regular was $2.89 per gallon while diesel was $4.19 at the same station. How many cars today use diesel and how many will in the future? I remember that when diesel and gas were closer in price, a number of cars and small trucks moved to diesel engines. But diesel cars and trucks never really caught on, and diesel vehicles never really challenged the market for gas vehicles no matter what the advantages were.
The same is true for almost all things that impact our lives, from food to technology and everything in between. The best technology, products or ideas do not always win. It is about how people invest in those ideas and market the products and technology.
Yahoo, Microsoft, Google, Amazon and Facebook could and do invest in technology, but they do not make hardware. If Yahoo, Microsoft, Google, Amazon and Facebook use some technology, it does not mean that manufacturers are going to invest in the same technologies for the broad market. Like it or not, even all of these five big players investing in something does not mean that they can create a market unless others start using the same technology. There are a number of storage examples to back this up. For example, Facebook has invested in optical, but it has not caught on in the broad market. It will be up to the optical vendors, not Facebook, to make optical useful in the broad market with the eco-system needed for the broad market. Cloud technology is not any different because services are basically products today, and clouds will depend on the same investments and profit requirements that every other technology faces.
My advice is to look at the investments today in storage technology for everything from the application interface to the storage devices and pick and choose the technologies that make the most sense in the short term while planning for the long term. Follow the money that is being invested, not your heart. If multiple vendors invest in a technology, it has a good chance of winning over the long haul. If multiple vendors have a technology they’re not investing in, it will eventually lose over time. Of course, over time market requirements can change. It is these interactions that I fear that are playing out in the tape market.
Last, but certainly not least, I want to thank Drew Byrd, who brought me to Internet.com; Forrest Stroud, who edited my columns in the early years; my long time editor and good friend Paul Shread; and James Maguire, my current editor, all of whom have made me look like a far better writer than I am over these many years. Thanks, Guys!
And thanks to my wife for reading my early columns and last column and allowing me the time to write. Thanks, Lisa!
Photo courtesy of Shutterstock.