Enterprise Storage Forum content and product recommendations are editorially independent. We may make money when you click on links to our partners. Learn More.
Fibre Channel over IP (FCIP) is a technology that links Fibre Channel-based storage area networks and extends them over large distances. FCIP comprises multiple transport technologies optimized for storage data movement, and uses the internet protocol (IP) to move data across a wide area network.
The technology is vital to businesses whose data is spread across multiple geographically distributed locations—for example, office buildings, data centers and disaster recovery data centers, and the cloud—because it allows for the speedy and efficient transfer of data with low latency.
Table of Contents
How Does Fibre Channel over IP (FCIP) Work?
FCIP is basically a tunneling protocol that uses widely available IP networks to connect geographically distributed Fibre Channel (FC) storage area networks (SANs). FCIP combines the capabilities of Fibre Channel with TCP/IP, effectively locating the entire FC stack within the TCP stack.
TCP/IP represents a set of rules that govern internet connections between computer systems. It determines how data is exchanged, how it is divided into packets, and how the various packets are to be addressed, transmitted, routed, and received as follows:
- Transmission Control Protocol (TCP)—Creates communications channels across the network and deals with disassembly of packets for transmission, and then reassembly at the receipt point
- Internet Protocol (IP)—Ensures packets are addressed properly so they arrive at the right destination (IP address)
This suite of communication protocols forms an abstraction layer between internet applications and the routing and switching fabric that underlies it, dictating which traffic goes where.
Transmission Control Protocol
TCP mechanisms provide FCIP with such capabilities as congestion control, error detection, and error recovery by binding FCIP links to TCP connections as a way to combine storage networking and networking over distance.
Internet Protocol
FCIP uses IP for transport over the internet while retaining vital management capabilities for remote SANs using SAN management tools. With IP gaining a vast bump in performance courtesy of the advent of Gigabit Ethernet, 10 Gigabit and emerging 400 Gigabit technologies, the amount of bandwidth available greatly adds to the value and cost-effectiveness of FCIP.
FCIP Tunneling
FCIP tunneling carries Fibre Channel traffic over IP networks by dividing FC data frames, putting them into IP packets within the tunnel, and reconstructing them as they depart. This point-to-point tunneling approach transparently connects two or more SAN fabrics across the IP networks that lie between them.
The Importance of Fibre Channel over IP
Establishing direct Fibre Channel connectivity using optic cables across vast distances is an expensive proposition—it’s far more cost effective to piggyback on the existing, nearly ubiquitous IP network. Owing to advances in IP technology in terms of throughput, it makes sense for FC to use IP for long distances.
This also opens the door to cheaper enterprise disaster recovery. It allows organizations to locate disaster recovery sites on opposite coasts, or in different states where the geography and weather patterns are different, for redundancy. If the grid goes down in one area, the other should be unaffected and ready immediately.
In some cases, transparent tunneling from one FC switch to another across IP networks extends SAN management capabilities across entire continents.
Fibre Channel over IP Use Cases
Fibre Channel is widely used and beneficial for a number of reasons. The following are some of the most common enterprise applications for it:
- Interconnecting SAN islands—Storage area networks can be spread out over large distances, anywhere from hundreds of kilometers to the entire continent.
- Disaster recovery data centers—FCIP enables the movement of large amounts of data cost effectively, reliably, and repeatability from one SAN to another, expanding disaster recovery locations.
- QoS improvements—FCIP can provide the higher Quality of Service levels demanded by SANs and many applications.
- Remote backup—The protocol allows for backup and replication at lower cost and over longer distances.
- Distance computing—FCIP’s low latency and cost effectiveness make it feasible to share resources and move data between geographies.
- Simplified SANs—Use of the protocol can simplify the management of a large, distributed network of SANs and reduce cost by leveraging existing IP networks.
FCIP also can be used to combine the capabilities of Fibre Channel and internet Small Computer Systems Interface (iSCSI). As complementary storage solutions, FCIP can connect FC islands over IP networks while iSCSI can help IP-based hosts access iSCSI or FC-connected storage.
However, they are generally used for different purposes—while SCSI commands support input/output disk access over an IP network. FCIP uses IP to transport SCSI and non-SCSI frames over IP networks.
Benefits of FCIP
Enterprise benefits of FCIP are numerous, and include the following:
- Lower latency—Operates efficiently over distance, and includes features such as windows scaling and selective acknowledgement to speed traditional TCP traffic.
- Packet coalescing—Further reduces latency by repackaging multiple smaller packets into a single larger one.
- Dynamic Path Control—Looks for and uses the fastest path to remote locations.
- Congestion reduction—Finds the best path for the highest performance across networks.
- Real-time reconstitution of lost packets—Eliminates delays common when multiple round-trip retransmissions occur.
- Resequences misordered packets—Automatically recognizes and resequences misordered packets.
- Adds flexibility—Allows organizations to use FC connectivity for low latency and mission critical storage, and IP for non-real time or lower cost storage and applications.
Challenges of FCIP
While the benefits are many, FCIP comes with a few widely acknowledged limitations.
Compromise over Pure Fibre Channel
The combination of FC and IP adds a variety of additional services and the ability to reach wider distances cost effectively for DR and data transportation purposes, but the downside is that it can’t match pure FC for performance, reliability, data integrity, or manageability. The amount of uptime of an FC SAN is higher than that of FCIP, as it uses IP networks.
Networking delays, packet loss, and latency are more likely in IP networks compared to wholly FC SANs. Traffic, applications, and data that use FCIP should be clearly defined, and critical traffic should remain within an FC SAN.
Lack of Security
FCIP does not come with built-in security. This is a common failing of many protocols and applications that were designed for functionality first, with security concerns manifesting later. Fortunately, the IPSEC protocol suite provides services for data confidentiality, authentication services, and key management.
Complexity
Troubleshooting and performance analysis of FCIP can be tricky. Within an FC SAN, a knowledgeable and experienced SAN administrator is needed to resolve issues, but with FCIP, both SAN and IP networking expertise is essentially to evaluate the entire data path that spans the FC fabric, an IP LAN, and an IP WAN.
Bottom Line
Both FCIP and FC SANs have their place in the enterprise world. FCIP offers the ability to use routers that link SANs across wide distances, transmitting data between SAN fabrics without the need for expensive FC cabling between sites. Existing LANs and WANs provide interconnectivity.
FCIP also represents a great way to keep DR costs down—it’s not cheap to mirror data centers across large distances using FC. Adding IP into the mix reduces expense and enables the movement of large amounts of data reliably and relatively fast.
Many organizations already have a sizable investment in both FC and IP and are using IP-based WANs for many functions. Piggybacking onto existing IP networks with FCIP can offer a simple, cost effective solution to expanding SANs.
Read 5 Types of Enterprise Data Storage to learn more about the different approaches businesses take to storing the vast amounts of data upon which they rely.
