Enterprise Storage Forum content and product recommendations are editorially independent. We may make money when you click on links to our partners. Learn More.
It appears that a foot race may be going on among storage protocols. Fibre Channel (FC) has more or less standardized now on 4Gbps, with 8Gbps and 10Gbps FC waiting in the wings. Meanwhile, 10Gb Ethernet (10GbE) is threatening to give FC a run for its money. Also waiting in the wings is InfiniBand, which so far seems largely limited to high-performance computing environments.
But analysts give Fibre Channel the edge for two reasons: cost and installed base.
“Even though 10GbE has been out since 2001, extremely high prices have considerably slowed deployment,” says Seamus Crehan, senior director at analyst firm Dell’Oro Group. “10GbE is still a lot more expensive than 4Gb FC, so there is no good reason to change.”
The Ethernet switch market has evolved rapidly since the early days of the Internet. 10Mb was superseded by 100Mb Fast Ethernet in 1995, then by 1Gb in 1997 and 10Gb in 2001. Yet to this day, Fast Ethernet remains the switch you are most likely to bump into. They are typically used as the standard speed for connecting PCs to the network. 1Gb, on the other hand, is used to aggregate those switches at the back end, or in the data center as server connections.
Over time, however, desktop-level switches are being swapped out for 1Gb models, while 10Gb is gradually gaining ground on the aggregation and server connection side.
“The need for that kind of speed for direct connection to a server wasn’t really there before,” says Crehan. “It’s only lately that servers can deal with more bandwidth.”
Technologies such as dual-core processors and the faster bus speeds of PCI Express are driving the move to 10Gbps, but it is a slow transition. According to Dell’Oro numbers showing revenues and port shipments for each speed, 1Gb passed 100Mb in terms of volume during 2004. Since then, 100Mb Ethernet has been on a slow decline. But it still accounted for $4.2 billion last year. By the end of 2007, Dell’Oro predicts sales of $2.9 billion.
1Gb Ethernet, on the other hand, has gone from $6.7 billion in 2004 to $10.2 billion last year. It will experience a gain of 10 percent or so this year. And down in the basement, 10Gbps recorded modest sales of $384 million in 2004 and surpassed $1.3 billion in 2006. Dell’Oro expects this to leap to $1.9 billion this coming year — yet that’s still less than a fifth of the level of 1Gb after six years on the market. And in terms of total ports shipping, 10GbE is hardly even on the map — less than one percent of the total market.
This doesn’t compare well with SAN adoption rates. 1Gbps FC switches effectively disappeared in 2004, although they recorded a tiny amount of sales in 2005. Similarly, 2Gbps storage switches peaked in 2004 at $1.3 billion, followed by a slow decline to $1 billion in 2005 and $474 million in 2006. At the end of this year, no more than $90 million is expected. And while 4Gbps SAN switches scored only $5 million in 2004, they soared to dominance by 2006 with $1.15 billion. By the end of this year, they will reach $1.75 billion — that’s almost the entire market.
When you look at ports shipped, a similar picture emerges. For 2006, it was 3.1 million 4Gb and 1 million 2Gbps switches. By the end of this year, 4Gbps FC switches will account for 5.1 million of the 5.3 million ports.
“We see rapid transitions to the next speed in SAN, as there is typically a very small price premium in that market,” says Crehan.
For example, on a high-end switch, 1Gbps Ethernet over fiber is selling for about $600 a port — the same price as a 4Gbps Fibre Channel port that provides four times the bandwidth. For 10GbE, the price per port rockets to $4500.
Storage Conservatism
This rapid adoption rate in storage, however, is tempered by another factor — conservatism.
“SANs are often used to back up the crown jewels, so these companies can’t afford them going down,” says Tam Dell’Oro, president of Dell’Oro Group. “Users are naturally cautious, so they spend many months in testing before implementing any upgrades.”
Take the case of San Diego Supercomputer Center (SDSC) at the University of California, San Diego.
“Since we needed additional ports for an upgrade of our compute system, we decided to adopt 4Gbps storage technology,” says Bryan Banister, SDSC manager of storage systems and production servers. “We foresee a complete move to a 4G storage infrastructure in the near future.”
SDSC chose 4Gbps FC to forward its goal of providing computational high-performance resources for academic research. This is primarily achieved using a 15-teraflop IBM POWER4+ compute cluster and an IBM Blue Gene/L compute rack with 2,000 processors (5 teraflops). SDSC has two SAN fabrics that comprise a total of 1,500 FC ports. All compute systems and all nodes connect to the two SANs that are built around five Brocade 12000 directors (2Gb) and six Brocade 24000 directors (2Gb). Into the mix was added a 4Gbps Brocade SilkWorm 48000 director.
As Bannister acquires more 4Gbps switches, he adds 4Gbps-capable disk arrays and HBAs.
“4 Gbps will give us the major boost we need to write large amounts of simulation data,” says Banister. “We have applications that generate 100 TB in a single simulation run, and all of it is passing through the SAN.”
Copper Hopes
Crehan notes that the competitive picture might shift if 10GbE moves to copper rather than fiber. A copper standard for 10GbE has just been approved.
“Copper may help somewhat with the price gap between FC and 10GbE switches,” says Crehan. “However, there are still power and distance issues to be resolved with copper.”
In the meantime, don’t expect any FC sites to rip out their networks and replace them with Ethernet anytime soon.
“Once fiber cabling has been installed, Fibre Channel becomes a ‘no brainer’ choice for storage networking over alternatives such as iSCSI and Ethernet,” says Tam Dell’Oro.
By the same token, those who have implemented an IP SAN are probably unlikely to jump sides and embark upon the expensive business of establishing an FC SAN fabric.
“Expect both protocols to continue for several years before one or the other gives way,” says Mike Karp, an analyst with Enterprise Management Associates. “There is no reason why FC shops should migrate to iSCSI or vice versa. The important thing is that companies now have more choice, and that will continue to drive prices down.”
