Download the authoritative guide: Enterprise Data Storage 2018: Optimizing Your Storage Infrastructure
To install M.2 high capacity SSDs into existing servers that lack dedicated M.2 sockets it's necessary to use a PCIe adapter card that sports M.2 sockets on each side. Some cards can support up to 16 M.2 sockets, and depending on the length of the M.2 high capacity SSDs used, it may be possible to pack 60 or more of these cards in a single 2U chassis, with a total directly attached storage capacity of 400 TB or more.
As M.2 SSDs get even higher capacity and server vendors get more used to the idea of building M.2 slots into their server offerings then we are likely to see much more storage available in a similar chassis size.
High capacity SSD costs
When it comes to cost, when these technologies are mature it's not clear yet whether one conventional high capacity SSD is going to be cheaper or more expensive than multiple smaller M.2 high capacity SSDs. In theory a single 60TB unit should be cheaper because it needs just one controller, while 30 2TB M.2 sticks would need 30 controllers – one for each M.2 stick.
But in practice it's not really quite that simple. That's because a 60TB high capacity SSD would have thousands of NAND chips to make up the 60TB of storage, which would make a single controller very slow, due to capacitive loading, Handy points out. So you would have to chop the controller up into bits and replicate it around the drive, with each one controlling say 20 address lines. That would increase the cost significantly – especially as these controllers would be manufactured in far smaller numbers than standard M.2 controllers.
Heat issues with high capacity SSDs
A possible drawback of using large arrays of tiny M.2 high capacity SSDs is the fact that heat becomes an issue when they are very densely packed. That's because unlike traditional SSDs, which are padded with thermal material to draw the heat away from the NAND chips and controller to dissipate through the exterior of their cases, M.2 high capacity SSDs are just bare boards with surface mounted components (like sticks of DRAM.) Pack them together too densely and you risk overheating or controller throttling, which may handle the temperature problem but only at the expense of performance.
Certainly heat is a critical issue, and Handy says that one of the original designers of the Violin flash storage platform has said that if the company could start again from scratch it would consider thermal issues first. (Violin storage modules are very little more than NAND chips on a DIMM, much like a M.2 stick.) "The question is, what's the most efficient way of getting rid of heat: tightly clustering NAND chips in a box (as is the case in a conventional SSD) or open to airflow like an M.2? I imagine that with M.2 the airflow would be better," he says.
So far the jury is out on which approach is going to be the most successful: one big high capacity SSD or lots of little M.2 ones. That's because it's too early to know what the cost, performance, reliability and even availability of servers with compatible sockets will be. The most likely scenario is that both approaches will have their place, but for the moment we'll have to wait and see how the high capacity SSD market evolves.