Download the authoritative guide: Enterprise Data Storage 2018: Optimizing Your Storage Infrastructure
This might sound a little weird coming from a lifelong propeller head, but the last home video game I played was Pong back in the 1970s, and that was because one of my cousins got it for his birthday. Pong and other early games bear little resemblance to the games of today. Besides the technology and realism, today's games are different just in the volume sold. In the early 1970s, much of the market for the computer industry was created by both business and government desire for faster, bigger and cheaper systems. In the 1990s, the market was driven by the home PC, inspired by widespread Internet access. Everything in the industry became a commodity based on the PC and the Internet. Today I think it is quite likely that we will see a shift in the market where gaming devices become the driving force and the PC will become an add-on to the gaming device. I won't debate the societal value, positive or negative, behind this shift. The point is that we will all be affected by this change, and we need to begin to consider how.
It might be a good idea to review a few of the storage technologies and the effect they had. Let's look at just the last 10 to 15 years or so.
Most of the disk drives sold in the world today are for PCs. Of course, with the proliferation of portable music and iPods, this might be changing, but most of the storage sold today is still for PCs. SATA disk drives have a large cost differentiation over enterprise Fibre Channel (FC) drives and SAS drives. This cost differential is based not only on the complexity and interface, but sheer volume too. Although SATA drives are less complex and the interface costs less, I suspect much of the SATA advantage is volume. An FC chipset is far more expensive than an Ethernet chipset, whereas SATA interface technology is now at a price point where it can be placed on the motherboard just like low-end RAID controllers. The PC market drove this requirement.https://o1.qnsr.com/log/p.gif?;n=203;c=204650394;s=9477;x=7936;f=201801171506010;u=j;z=TIMESTAMP;a=20392931;e=i
The proliferation of NAS and low-performance shared storage, I believe, was also driven by the PC market. NAS was initially used to provide solutions for SMB PCs networked together to share storage, but today, with the cost of disk drives and home networks coming down, you are starting to see NAS attached to the home router as a backup device and shared storage. A technology that began in the enterprise and moved to the home.
There are just two examples of how commodity PCs affected the storage market. So if my assumption about the importance of gaming is correct, how will home gaming machines affect our future?
As always, it's all about requirements. What do these games require?
CPU and processing requirements:
- Graphic processing: Generally handled with special purpose hardware.
- High Performance CPU: Why would IBM, Toshiba and others develop the Cell Processor if they did not need high-performance CPUs?
- Fast memory.
Storage and I/O requirements:
- Streaming storage: Not only does the storage have to be fast, but the file system needs to be able to stream at device speeds (a high-performance file system), and the future might be a shared file system within a network.
- A high data rate channel for storage and output to your viewing device.
Let's assume that I'm right about all of these requirements. What will the affect be on your storage environment in a few years? Remember, it took about 10 years for the PC to affect enterprise storage for disk drives, but since markets now move faster, it is likely that the time could be halved.
I believe there are three potential affects: Physical storage density and performance; File systems, and power and cooling.
The future of gaming is shared gaming, in which you are playing not just against the machine, but against other people in the local environment and across the world. Today, some of the network games are limited by network performance, but equally important and often overlooked is storage performance. As the game environment grows, it becomes important to add and remove people and increase the complexity, which requires more storage and more processing. Since the local storage is likely faster than the network, sending small bits of information over the network to access the storage is always faster than sending all the information over the network.
What this means is that games are going to be required to store more information, with more and more games coming out, and to be able to read that information faster. Reading will be more important than writing, since much of the writing is loading the game and storage density must grow to meet the requirements.
Since games require higher performance storage, the file system must be able to scale to meet the performance of the network of players. Since storage needs to be faster, the file system must scale to meet the storage performance. If you take, say, an NTFS file system, over a few years you will find that your boot time gets greater and greater. If your system crashes and you need to reload your system, you will find that it amazingly boots much faster. The reason is that the file system data, metadata and virtual memory area have become fragmented. No amount of Microsoft defragging will help, but that is a different article. File system performance will need to scale to meet these performance requirements, and all of my complaining over the years about fragmentation will need to be addressed to ensure smooth operation of the gaming environment, and hopefully that will translate into fixing the problem for other systems and file systems.
Power and Cooling
We all know about power and cooling problems in computer rooms, both large and small, since for some environments this is becoming a serious problem. For the fun of it, I searched "cooling xbox" on Google, and lo and behold, there were a huge number of hits for extra cooling accessories, discussions on power and heat, and if my memory is correct, there were some issues with Xbox and overheating soon after it came out. This same problem is true for many home computers. I added an extra fan to my computer given the amount of heat generated. The amount of power that disk drives use is likely not going to change, since they are mechanical devices, so power and cooling and the future improvements in both will be dependent on the gaming industry.
It's All in the Game
I believe the gaming industry is going start driving storage technology just as it is driving computational and graphics technology. For example, the Cell Processor from the Sony-Toshiba-IBM consortium is being used in games, but it is also being used by Los Alamos Labs in their next-generation supercomputer project called RoadRunner.
I believe this same type of crossover technology is destined for the storage industry. For PCs, the proliferation of IDE and now SATA drives has moved into the mainstream for enterprise secondary operations such as D2D backup.
Move over PC technology, the gaming industry is about to take over. Combine this with the revolution in solid state and flash storage for cameras, music players and the like, and it will not be long until you have systems with levels of storage just like we have in the enterprise, except this time it will be flash and disk. Do a quick Google search and the laptop market is just about here and the game market is just behind. Maybe SSDs will make a comeback in the enterprise if vendors finally figure out how to truly have hierarchies of storage for the masses. One of the big problems with SSDs is that most people really do not know how to use them, what files are important at what time in terms of latency reduction. If we get levels of storage in games, it just a matter of time before operating systems and file systems start to understand levels of storage in a more complex way than is current available with HSM systems.
The world sure has changed. When I started, supercomputing drove the market and trickled down to the enterprise. Today, much of technology comes out of PCs, games and devices like iPods and iPhones and trickles up to supercomputers. Maybe the Mayans were correct and the world is going to flip upside down in 2012.
Henry Newman, a regular Enterprise Storage Forum contributor, is an industry consultant with 26 years experience in high-performance computing and storage.
See more articles by Henry Newman.