As the Internet and related activities continue to expand, the amount of data that needs to be stored is also increasing. Enterprises and other organizations require effective ways to store and maintain this data.
In recent years, many enterprises have seen a significant increase in the volume of data produced. And, this amount of data continues to increase, particularly in Web-based and e-Commerce environments. A good example would be e-mail which impacts worldwide storage by producing more data than is generated by new Web pages. These types of traffic are typically multimedia intensive. E-mail and Internet-related enterprise/commercial transactions combined have caused a dramatic increase in storable data moving across Internet Protocol (IP) networks.
A new method is needed to bring improved storage capabilities to IP networks and reduce limitations associated with Fibre Channel SANs. The solution, as is widely known, is Internet Small Computer Systems Interface (iSCSI) or SCSI over IP. But what does this new technology mean to your storage environment? This article will answer the following questions:
- What will iSCSI mean to your storage network?
- What will the upcoming availability of iSCSI mean to customers who perhaps had considered storage networking to be too expensive?
- What, in simple terms, will users need to implement an iSCSI based SAN?
- How will an iSCSI based SAN compare in terms of performance and cost to Fibre Channel?
Internet SCSI (iSCSI) is a draft standard protocol for encapsulating SCSI command into Transmission Control Protocol/Internet Protocol (TCP/IP) packets and enabling I/O block data transport over IP networks. iSCSI can be used to build IP-based SANs. The simple, yet powerful technology can help provide a high-speed, low-cost, long-distance storage solution for Web sites, service providers, enterprises and other organizations.
An iSCSI Host Bus Adapter (HBA), or storage network interface card (NIC), connects storage resources over Ethernet. As a result, core transport layers can be managed using existing network management applications. High-level management activities of the iSCSI protocol (such as permissions, device information and configuration) can easily be layered over or built into these applications. For this reason, the deployment of interoperable, robust enterprise management solutions for iSCSI devices is expected to occur quickly.
First-generation iSCSI HBA performance is expected to be well suited for the workgroup or departmental storage requirements of medium- and large-sized enterprises. The availability of TCP/IP Offload Engines (TCP/IP Offload Engines (TOEs) are based on session-layer interface card (SLIC) technology, which can be used to improve the performance of servers, network-attached storage (NAS) and iSCSI storage devices.) will significantly improve the performance of iSCSI products. Performance comparable to Fibre Channel is expected when vendors begin shipping 10 Gigabit Ethernet iSCSI products in 2003.
Benefits of iSCSI
By combining SCSI, Ethernet and TCP/IP, iSCSI delivers the following key advantages:
- Builds on stable and familiar standards: Many IT staffs are familiar with the technologies.
- Creates a SAN with a reduced TCO: Installation and maintenance costs are low since the TCP/IP suite reduces the need for hiring specialized personnel.
- Ethernet transmissions can travel over the Global IP Network and therefore have no practical distance limitations.
- Provides a high degree of interoperability: Reduces disparate networks and cabling, and uses regular Ethernet switches instead of special Fibre Channel switches.
- Scales to 10 Gigabit : Comparable to OC-192 SONET (Synchronous Optical Network) rates in Metropolitan Area Networks (MANs) and Wide Area Networks (WANs).
Who Can Use iSCSI?
iSCSI SANs are most suitable for enterprises with a need for streaming data and/or large amounts of data to store and transmit over the network. This includes:
- Businesses and institutions with limited IT resources, infrastructure and budget. These organizations should look for iSCSI equipment that functions over standard Gigabit Ethernet Category-5 copper cabling already in place in most buildings today.
- For example, work team members who need the latest project data without waiting 24 hours for traditional replication/backup/reconciliation procedures.
- Geographically distributed organizations that require access to the same data on a real-time basis.
- Internet Service Providers (ISPs).
- Organizations that need remote data replication and disaster recovery. For example, a high-technology company in San Jose, California remains susceptible to disaster if it uses a Fibre Channel SAN. Original and backup data copies could be lost in the same earthquake due to distance limitations.
- Storage Service Providers (SSPs).
iSCSI Deployment Examples
Now, let's look at a few iSCSI deployment examples. The examples are as follows:
- Network Storage Services via iSCSI.
- Multiple Cards to Single iSCSI Router.
- iSCSI HBA and Fibre Channel Tape Backup.
Network Storage Services via iSCSI
Two iSCSI HBAs can be used in conjunction with standard Ethernet NICs through a Gigabit-capable switch connected to an iSCSI-capable Redundant Array of Inexpensive Disk (RAID) Array. This configuration is appropriate as either the next step in transitioning to an iSCSI-exclusive SAN or as an initial iSCSI SAN configuration.
Multiple Cards to Single iSCSI Router
Multiple HBAs in separate servers can be used in conjunction with a Gigabit capable switch connected to an iSCSI capable router with Fibre Channel ports. This is then connected directly to a native Fibre Channel RAID Array. This configuration is also appropriate as the next step in transitioning to an iSCSI front-end SAN with Fibre Channel storage devices.
iSCSI HBA and Fibre Channel Tape Backup
An iSCSI HBA can be used in conjunction with a Gigabit-capable switch connected to an iSCSI-capable router with Fibre Channel ports connected to a Fibre Channel tape drive. This configuration can be used as a means to perform backup and recovery, by using an existing Ethernet infrastructure.
Next, let's discuss what the availability of iSCSI will mean to customers who had considered storage networking too expensive. In other words, what are the advantages of iSCSI SANs?
As previously explained, iSCSI is an end-to-end protocol for transporting storage I/O block data over an IP network. The protocol is used on servers (initiators), storage devices (targets), and protocol transfer gateway devices. iSCSI uses standard Ethernet switches and routers to move the data from server to storage. It also enables IP and Ethernet infrastructure to be used for expanding access to SAN storage and extending SAN connectivity across any distance. The technology is based on SCSI commands used in storage traffic today and IP protocols for networking.
Leveraging the Best from Storage Networking
iSCSI builds on the two most widely used protocols from the storage and the networking worlds. From the storage side, iSCSI uses the SCSI command set, the core storage commands used throughout all storage configurations.
On the networking side, iSCSI uses IP and Ethernet, which are the basis for most enterprise networks, and uses in metropolitan and wide area networks is increasing as well. With almost 30 years of research, development and integration, IP networks provide the utmost in manageability, interoperability and cost-effectiveness.
10GbE Enabling iSCSI and Its Advantages
As the demand for bandwidth is increasing for storage and networking applications, Gigabit Ethernet technology provides the right path. However to make these applications mainstream, 10GbE is needed for the following reasons:
First of all, a 10 Gigabit Ethernet network will have the capabilities to provide solutions for unified storage and networking applications. With networking applications requiring gigabits of throughput and the terabits of storage transactions, existing gigabit networks will max out. However, 10GbE will be able to sustain lower latencies and high performance needed for these applications.
Second, with regards to interchangeability and Interoperability of equipment, the Fiber Channel model is not optimized for connectivity of multiple vendor devices. With 802.3 standards based products, Ethernet has continued to provide solutions that can connect systems for multiple vendors, thus allowing for a better cost model and a variety of vendors to chose from for the end user.
Third, would be the consolidation of the SAN and network attached storage (NAS) markets. The final reason why 10GbE is needed, is the ability to connect Fiber Channel SAN islands through IP. A link greater than the gigabit interfaces in the SAN islands is required.
So, with the preceding in mind, what would be the advantages of iSCSI SANs? The following advantages are as follows:
- Familiar network technology and management: Reduces training and staff costs.
- Proven transport infrastructure: Increases reliability.
- Transition from 1 Gigabit Ethernet to 10-Gigabit Ethernet and Beyond: Protects investment with simplified performance upgrades.
- Scalability over long distances: Enables remote data replication and disaster recovery.
- Brings Ethernet economics to storage: Enables lower total cost of ownership.
Building iSCSI SANs with 10GbE: A Data Center Approach
An iSCSI SAN is a perfect choice for a user interested in moving to networked storage. Using the same block level SCSI commands as direct attach storage, iSCSI provides compatibility with user applications such as file systems, databases, and web serving. Similarly, since iSCSI runs on ubiquitous and familiar IP networks, there is no need to learn a new networking infrastructure to realize SAN benefits. To build an iSCSI storage network in a data center, iSCSI host bus adapters can be used in servers, along with iSCSI storage devices and a combination of switches and routers.
An iSCSI SAN is an optimal choice for a user interested in moving to IP Storage. iSCSI is like one more application to the network protocol stack. So, iSCSI is not only compatible with the existing networking architecture, but also maintains the same block level SCSI commands. This capability allows the information technology (IT) staff to transition from the direct attached storage (DAS) model to the iSCSI SAN model. By adding the storage traffic to the existing network, the IT staff doesnt need any additional training to manage the networks for IP Storage.
In a typical data center, the servers updates/retrieves the data from the storage devices located remotely at gigabit speeds. Consolidated storage serves multiple servers at the same time. In the same environment, network traffic is processed at gigabit speeds. The IT staff has a challenging task to support the growing needs of storage and network requirements. Though gigabit networks are being deployed widely, it cannot solve all the problems. Storage networks have low latency and high bandwidth requirements.
iSCSI at 10 Gigabit Ethernet is the answer to these requirements. The 10GbE provides a smooth transition for the existing storage networking infrastructure to higher speeds. Applications like synchronous mirroring demand low latency, and file serving needs high bandwidth. By using a host bus adapter (HBA), which supports both the network protocols and the iSCSI protocols, both the SAN and NAS environments can be consolidated. The 10GbE networks facilitate the high bandwidth and low latency required in this environment, thereby resulting in improved application response time. This would consist of the following data center applications:
- Server and storage consolidation.
- Accelerated Backup Operations.
- Seamless Remote Site Access and Storage Outsourcing.
Server and Storage Consolidation
With a networked storage infrastructure, customers can link multiple storage devices to multiple servers. This allows for better resource utilization, ease of storage management, and simpler expansion of the storage infrastructure.
Accelerated Backup Operations
Backup operations previously restricted to operating across traditional IP LANs at the file level can now operate across IP Storage networks at the block level. This shift facilitates faster backup times, and provides customers the flexibility to use shared or dedicated IP networks for storage operations
Seamless Remote Site Access and Storage Outsourcing
With the storage network based on IP, customers can easily enable remote access to secondary sites across metropolitan or wide area IP networks. The remote sites can be used for off-site backup, clustering or mirroring replication. Additionally, customers can choose to link to storage service providers for storage outsourcing applications such as storage-on-demand.
Expanding iSCSI Storage Networks to Metro and Wide Area Networks
Enabling storage over long distances is essential to remote site backup or implementing robust disaster recovery applications. The rapid adoption and expansion of IP data on the Internet has proven the viability of using IP across long distance wide area networks. Although it is expected that initial deployments of iSCSI will use private networks, with the use of IP's security infrastructure (such as Secure Internet Protocol (IPSec) and Secure Sockets Layer (SSL) to provide authentication and privacy), it will be possible to use public networks for wide area iSCSI storage traffic as well.
The 10 Gigabit Ethernet provides the necessary network links, reliability, and bandwidth for carrying large amounts of storage data over both private and public IP networks covering varying distances. To meet the distance objectives of metropolitan area networks/wide area networks (MAN/WAN) storage deployments, the IEEE 802.3ae Task Force specifies the 1550nm serial transceiver to maintain the support of 40 Kilometer links over a single mode fiber that Gigabit Ethernet has already been successful in deploying. The 10GbE also specifies a variety of optical transceivers that support distances including 300 and 10,000 meters over single and multi-mode fiber. The 10GbE standard additionally specifies a WAN Physical Layer Device (PHY) to facilitate compatibility with the existing WAN network. The WAN PHY supports the same distances and optical transceivers as the LAN PHY specification. The 10 Gigabit Ethernet WAN PHY supports connections to existing and future installations of Synchronous Optical Network/Synchronous Digital Hierarchy (SONET/SDH) circuit-switched telephony access equipment by including a simplified SONET/SDH framer. The ubiquity of Ethernet and provisions made in the 10GigE Standard, make 10GbE the ideal choice for supporting iSCSI storage networks in metro- and wide area networks.
So, what in simple terms will people need to implement an iSCSI based SAN? Lets take a quick look.
Storage Area Networks (SANs) are growing rapidly because they solve a problem. That problem is the manageability of large and ever increasing amounts of disk storage. When storage is directly attached to the computer using it through a high-speed cable, the storage device can meet the needs of that computer very well. When computers are networked together (each computer with its own attached storage and total storage), resource management is difficult or impossible to achieve. The SAN approach offers many storage management advantages, including the ability to:
- Backup without degrading performance for the network users.
- Balance the load to each physical device.
- Easily grow the size of the storage pool.
- Easily share backup devices.
- Eliminate downtime.
- Manage your computing resource separately from your storage resource.
- Manage your storage resource easily and efficiently.
- Share a pool of storage.
As previously explained, the SCSI and Ethernet standards are each mature, stable, ubiquitous and interoperable. These qualities provide a solid foundation on which to implement a SAN. iSCSI is the emerging Internet Engineering Task Force (IETF) specification that defines how SCSI and Ethernet work together to perform SCSI data transfers across TCP/IP networks. It allows the tried and true block storage SCSI commands to be carried by the standard TCP/IP protocols over the Ethernet wire.
Until recently, Ethernet was too slow to compare to the performance of direct attach storage. With the advent of 1 Gigabit and 10 Gigabit Ethernet, an Ethernet network is now fast enough to use for storage applications. So, why would you want to have a different network infrastructure for SANs than you do for the LANs, MANs and WANs that make up the Internet and your company's own networking? Lets take a look.
Basically, iSCSI is composed of initiators and targets that will be used at both ends of the Ethernet wire. The iSCSI initiator connects the computer that initiates the request for data stored on the disk with the Ethernet wire. The iSCSI target connects the Ethernet wire with the storage device, whether it be an array of disks or perhaps a tape device. The initiator encapsulates the SCSI commands and data, so that they can be sent over TCP/IP; and, the target extracts the commands and data at the storage device end of the wire.
As with any networking protocol, the software that implements the iSCSI over the TCP/IP approach, is thought of as having layers. One or more of these software layers can be run on the hosting computer, on a coprocessor that lives on a plug-in board, or some of the software function may be implemented in a chip. The approach used determines the cost and performance of the SAN connections. The best approach may be application specific, and your options may depend on the operating system in use on your host computer. Nevertheless, while the potential benefits of a single network infrastructure for storage and networking are many, and will reduce the total cost of ownership of a SAN, there is still work to be done to make this option a practical reality.
Finally, how will an iSCSI based SAN compare in terms of performance and cost to Fibre Channel? Let's take a very brief look.
Fibre Channel Limitations
The differences in an iSCSI based SAN (as compared in terms of performance and cost) to Fibre Channel, is in the limitations of Fibre Channel itself. In other words, while Fibre Channel is a high performance transmission technology optimized for the same block storage format that storage devices use, it does have drawbacks:
Total Cost of Ownership (TCO)
The Total Cost of Ownership (TCO) for operating a Fibre Channel SAN, while lower than the DAS model, is still high. Since organizations vary widely in their storage needs, it is difficult to develop a set of assumptions for generating average cost figures. Still, Fibre Channel is a fairly new technology and many IT staffs have limited, Fibre Channel expertise. Finding the necessary specialized personnel can be challenging and training is often not readily available. As a result, installing and maintaining a Fibre Channel network is typically difficult and expensive.
Finally, although the theoretical limit for Fibre Channel is 10km, individual multi-mode fiber links used in Fibre Channel SANs may have a practical limitation of 250 to 500 meters. The storage ecosystem is evolving to where large organizations often have SANs located far from the LAN, to provide geographical redundancy as part of disaster planning. This means even 10km may be increasingly inadequate.
Summary And Conclusions
Organizations with changing data requirements, especially those requiring data storage security or disaster recovery, will benefit most from the introduction of IP storage and iSCSI. As performance increases and iSCSI SANs become ubiquitous, they will gradually be integrated into enterprise TCP/IP networks. Distributed intelligent services and automated allocation of storage resources via virtualization, will become an integral part of the future evolution of iSCSI SANs.
Furthermore, iSCSI represents a dramatic shift in the storage networking landscape. iSCSI will expand the market for networked storage by giving IT managers another alternative to direct attached storage that delivers the advantages of networked storage. Today, IP storage networks take advantage of IP networking knowledge in IT departments and use existing network management and tools for LANs, MANs and WANs. Finally, by riding the IP wave of technology development and enhancements (like the introduction of the 10 Gigabit Ethernet), iSCSI provides a logical unified infrastructure development path for enterprises and service providers alike.Further Reading