Super-Sizing Data Center SANs
Nearly all large enterprises have implemented shared storage based on Fibre Channel SAN technology. Typically, these installations began as fairly modest configurations, with a few dozen servers and one or more large SAN-attached storage arrays. But with SAN technology demonstrating its end-user value and ability to more efficiently manage storage data, data center SANs in many cases have quickly grown to encompass more servers and more storage capacity.
Although thankfully McDonald's has sworn off super-sizing fast food, data centers worldwide continue to super-size their SANs in an attempt to connect hundreds, and in some cases thousands, of servers and storage devices in a single storage network.
Standards vs. Reality
According to ANSI standards, a Fibre Channel SAN may have as many as 239 switches in a single fabric and support up to 15½ million devices. These standards limits are based on the number of bits that are available within a three-byte Fibre Channel address associated with unique Domain IDs for individual switches and device identifiers for attached nodes such as servers and storage targets. Although for competitive reasons all switch vendors proclaim their support for these impressive figures, none has been able to implement them in real product.
Supposing for a moment that a vendor could actually support these standards limits, the time required to stabilize a single SAN composed of hundreds of switches and the complexity of registering tens of thousands of devices would be unacceptable in any enterprise environment. Nonetheless, many customers are attempting to build large fabrics to support their business applications, either by connecting director-class fabrics or by building complex meshes of departmental switches.
Why Build Big SANs?
Storage managers are attempting to build large SAN fabrics for numerous reasons. Typically, the purpose of a large storage network is not to provide any-to-any connectivity; rather in most cases, a large multi-switch fabric results from an attempt to share one or more storage assets by hundreds of servers — a many-to-few connection strategy.
A large enterprise, for example, may have invested hundreds of thousands of dollars in a large robotic tape library. In order to share this single resource with hundreds of devices, it is necessary to attach multiple directors or switches by interswitch links (ISLs).
Likewise, sharing several large storage arrays between 500 or more servers may require multiple ISLs between ten or more director-class fabrics. From a customer standpoint, the business objective of more efficiently sharing storage assets is fairly straightforward. From a technology perspective, however, achieving this business goal can be very complex.