Download the authoritative guide: Enterprise Data Storage 2018: Optimizing Your Storage Infrastructure
The Limits of Switch Scalability
Although no real world customer (unlike certain performance analysis vendors) has proven crazy enough to attempt to connect 239 switches together, large enterprise customers often need to scale beyond 20, 30, or 40 switches. This has proven problematic and often impossible for several reasons.
First, Fibre Channel is a flat, layer two network, much like a bridged LAN. It is deliberately flat, since a layer two (link layer) network offers optimum performance, and optimum performance is what channels are all about. But because a fabric is flat, everything sees everything, and more importantly, any device can affect any other device (or all other devices) in the network.
As the number of switches in a single fabric increases, all the switches must engage in fabric-building processes, principal switch selection, exchange of Simple Name Server (SNS) information, routing tables, and state change registrations. This quickly engenders excessive fabric convergence times, often lasting tens of minutes or more. In most vendor implementations, the memory assigned to SNS tables is simply insufficient to support more than a dozen heavily populated switches in a single environment.
Second, supposing 10-20 switches can be brought up and stabilized into a single working fabric, the fabric is always vulnerable to disruptive fabric reconfigurations or state change notification broadcast storms. This can be highly disruptive to ongoing storage transactions and can incur SAN outages if the fabric has to undergo a compete rebuild.
SAN Routing Technology: Much Ado About Something
While some customers struggle to build very large fabrics with existing products, a more fundamental question is, why? What is the desired result that makes some customers feel that 239 switches in a single fabric is something they might want? Typically, large fabrics are not created so that any storage device or host can talk to any other storage device.
Networking provides any-to-any connectivity, but applications usually only need one-to-one or one-to-a-few (e.g., a server cluster to a single storage target). What customers that are trying to build large SAN fabrics are typically attempting to do is share a large and expensive storage asset such as a tape library between a larger number of storage devices — a many-to-one solution.
With native Fibre Channel, this many-to-one result can only be achieved by connecting all fabric switches together, and paying the penalty in terms of convergence time, SNS limitation, and exposure to fabric-wide disruptions.
New SAN routing technology, by contrast, can achieve many-to-one connections by allowing multiple SAN switches to exist as separate fabrics, with only authorized connections between them for assigned storage assets. First pioneered by the company formerly known as Nishan (which was recently acquired by McDATA), this solution filters fabric building protocols, eliminates SNS sizing issues, blocks fabric reconfigurations, and isolates faults. Additionally, it overcomes even the theoretical limit of 239 connected (in this case, routed) switches in a storage network.
With this new capability, exaggerated claims on data sheets are no longer required, nor are expensive test programs to expose them. Now, that’s much ado about something.