Benchmarking Storage Systems, Part 1 Page 3
NAS and Tunables
Benchmarking NAS devices is even more complex than benchmarking RAIDs with file systems, as you have RAID and file systems, hardware, and software all in a single package. The only good news is that NAS performance does not and cannot get close to the performance of Fibre Channel-attached RAID. Additionally, as NAS cost per MB is generally lower than that of Fibre Channel RAID, NAS devices are benchmarked far less often. Fragmented file system performance and CPU performance can be big issues for some NAS systems.
You must ensure that you understand the effects of the I/O of your benchmark and utilize the cache of the NAS in a way that will emulate the behavior of the real system.
The server performance is often not considered, but definitely should be, especially for Fibre Channel RAID devices given their high performance. It might be sad, but I have observed the following to be true:
- Some servers have PCI and PCI-X buses that do not run at full rate
- Some servers have different performance for different PCI/PCI-X slots
- Some servers are not configured with enough memory bandwidth to run all of the PCI-X slots at full rate
- Many servers do not have the memory bandwidth to run the CPU(s) and PCI/PCI-X buses at the same time
Let me tell you about a test we recently conducted for a client. We were testing a RAID device and found that our 16 MB I/Os were getting broken down into requests from 4 KB to 128 KB on Linux. This reduced the performance of the RAID device by about 30% for reads and 40% for writes.
It was suggested that we load Windows 2000 and then retry our experiments. Once we did so, low and behold the performance was where it should be. I am not trying to bash Linux with this example, but rather point out that the operating system, even on exactly the same hardware, can make a huge difference.
HBAs and HBA tunables
In almost all cases, the vendor will understand and tune the HBA for maximum performance for the system. Therefore this should not a big issue. The only issue is that an HBA and drive can have a large performance difference for both transfer rate and system overhead to read/write the data with the HBA driver.