Download the authoritative guide: Enterprise Data Storage 2018: Optimizing Your Storage Infrastructure
Understanding how much cache is needed really comes down to either throwing darts blind folded or using tools that determine the amount of cache that is needed. Tools are the most difficult thing about this process.
I have seen tools from vendors in this area come and go. Since capacity management for open systems is not—and never has been—considered important considered important for open systems, many people just buy more hardware as that is cheaper than figuring out what is wrong.
Given where the disk drive vendors are going, things are about to get very interesting. Current hybrid disk drives have around 8 GB of cache per drive, but as of yet none of them are enterprise drives. I think that will be changing over the next few quarters based on announcements and market pressures.
Let’s say you have a file system with on hundred 4 TB drives and each of the drives has 16 GB of flash cache. That would be 1600 GB of flash cache or 1.6 TB for your 400 TB of storage. Let assume that you have RAID-6. That drops everything by 20 percent, so you have 320 TB of storage and 1.2TB of flash cache. Now, if we assume that the drive vendors are using similar algorithms to the caching vendors, with 320 disk drives and 1.28 TB of flash cache for the drives, you have 320*120 MiB/sec of bandwidth to and from flash. This yields 37.5 GiB/sec of bandwidth to/from flash.
Think about this: that is the equivalent of about twenty-three 1600 MiB/sec fibre channel connections or twenty-five 12 Gb/sec SAS connections (fifty 6 Gb/sec SAS connections). I think you see the points.
The big problem I see is that all these SSD flash caches are going to have nowhere near the bandwidth that the hybrid disk drives will have, unless you put a lot of money into bandwidth for connecting these types of external caching devices.
I think that the disk drive makers understand this, and the external caching marketing will be short-lived again. This is not to say that if you have caching requirements and your applications and users can benefit from them that you should not buy these technologies. But I am suggesting that over the next few years, external caching devices will likely be replaced with caching disk drives—if the market demand for these drives goes the way I expect them to go. But remember I have been wrong before.