Data Center using Fibre Channel over Ethernet

I was in Almaty, Kazakhstan doing a training on FCoE and I’ve got some questions on the adoption of FCoE technology. Should we move? When? Why?

Fibre Channel over Ethernet is a technology that allow connection to a Fibre Channel SAN across an Ethernet network. The technology is quite new and has a lot of requirements.
The requirement on the server is a Converge Network Adapter (CNA). The CNA is a card that will have Fibre Channel and Ethernet capabilities. Most of the models, I’ve seen, have 2 10G Ethernet ports. Multiple types exist. For the high end server, you need a CNA that will process the Fibre Channel protocol on the hardware card. For the low end Servers, you could use card like the Intel Oplin 10G Ethernet card where you could process the Fibre Channel on the Operating System.

Converge Network Adapter

Currently not all 10 G Ethernet card have Fibre Channel capabilities. It may require upgrade of the driver or change of the network card.

On the Ethernet Network, you will need to support Data Center Bridging protocols. On the design side, it is recommended that the FCoE network is limited to a part of your Data Center. With the solutions I’ve seen, you don’t have the possibility to do FCoE over the WAN. Even, Multihop FcoE is really limited. All the equipments in your FCoE network need to support FCoE.

The advantage of FCoE is to limit the number of network equipments per row. Instead of having, 2 Ethernet switches and 2 Fibre Channel Fabric, you could have only 2 FCoE equipments. It will allow better integration of your SAN and LAN. And it will lower your cabling, your power and cooling requirements.

Currently, on the standard side, there is some weakness. Data Center Bridging standardization at the IEEE takes some time and interconnection of multiple vendor solution is said to be difficult. Cisco and EMC have tested some solutions with mixed equipments. May be the Fibre Channel Industry Association like the Wi-Fi group should label tested interoperabilities.

Is it time to move ? There is no simple answer. I would not recommend to one of my customer to change all his existing equipments to support FCoE. On the other hand, it could lower the TCO for new server deployment. If a new rack of server is required, I would push FCoE as there is no competition of that solution. FCoE is a good solution because it allows you to move to this technology at your pace. French Version.

This is an entry level post. I will try to do something more technical next time. Please comment and I will amend my post.


About Charles Perroquin

Networking all over the world, Africa, Middle East, Central Asia, Europe. Data Center, Telephony over IP, Security. Mainly with Cisco.
This entry was posted in Data Center, Marketing and tagged , , , , . Bookmark the permalink.

7 Responses to FCoE

  1. Hi there,

    Great first post. I thought you have a couple of items that are quite useful, but I thought I might help you with a couple of things.

    First, you mention that one of the downsides is that standardization takes some time. In fact, the IEEE DCB standards are complete and were published last year (though they have been technically stable for a long, long time before that).

    Second, you mention that multihop FCoE is limited, but it’s worth taking a closer look at what you mean by that. The technology is no more limited than FC – Cisco supports FCoE the same way as it supports FC, up to 7 hops and with the same fabric login limits. Customers can use FCoE in multihop scenarios using the same FC tools that they’re familiar with. In that regard, the technology is not limited.

    You do bring up interoperability. In order to do standards-based FCoE ISLs, you must be able to support VE_Ports. Currently there is only one vendor who has this (Cisco). If you wish to do a FIP-Snooping Bridge (FSB), there are some vendors who claim to support their switches to connect to a FCF (Juniper comes to mind). Interoperability is limited only by which company has the feature set you wish to use, and whether or not it’s supported.

    Third, you discuss a consortium. The Fibre Channel Industry Association (FCIA), which works on all things Fibre Channel, is just such a group to help promote the use of FC – regardless of whether it falls on ‘native’ FC links, Ethernet links, or some other type of physical media. You can check out their website at http://fibrechannel.org.

    Finally, the all-important question about whether or not to move to FCoE is not an easy answer. For some customers, when it comes time to refresh their servers or expand their networks, they like the fact that a converged network can reduce the number of management points (as you point out in your post). For others, the environments may be too small or they may not have FC experience and their expected growth patterns may not mandate a need for FC/FCoE.

    For others, though, there are many thousands of deployments of FCoE, ever since the first equipment began shipping in 2008. In some cases customers have saved millions in operational expenses by being able to consolidate their networks onto a common architecture. Ultimately, the answer of whether or not to advice a customer to deploy FCoE depends entirely upon their existing environment and future growth plans.

    After all, a protocol is a tool, and like any tool it can be used appropriately or inappropriately. Choosing not to use it when it is appropriate is equally as short-sighted as using it when you shouldn’t. 🙂

    Thanks for the post. Keep writing!

    J Metz

    (disclosure, I am a Product Manager for FCoE for Cisco)

    • cperroquin says:

      Mmm, opening a blog in the middle of the night because of bad sleep and jetlag could bring unexpected answer from unexpected people.

      The goal was really to do basic introduction on FCoE.

      On the consortium side, standarization and interoperability, ok for FCIA. But if we stick to the WiFi comparison, on the wifi, we’ve got complex interoperability (security, network, …) but a simple label to guide the customer.

      And of course, moving to FCoE, could not be decide in reading 3 lines on a blog… I was willing to say that it is a complex decision… I might need to rewrite this part.

      Anyway thank you for your support, I will try to improve this post base on your input.

    • cperroquin says:

      When this comment was published, it was 100% accurate. In the mean time, I slightly modify my post and it may seem less accurate. Sorry for that.

  2. Great start!
    What do you think about this article from Dell?

    • Comparasion of performance is always a problem. Specially when it is a vendor that provide the numbers. What are the conditions of the test ? Also, looking at the history, this article has been mainly written in 2009.

      The performance issues with iSCSI is when you’ve got packet drop. Unlike FCoE that works in lossless environment, iSCSI rely on TCP for flow control. When you lose a packet the speed of the TCP session will drop.

      Also with TCP due to the delay for the acknowledgement, if you increase the distance, you will lower significantly the throughput.

      On the price side, iSCSI is a seducing solution.

  3. Pingback: FCoE | Networking

  4. The FCoE protocol specification replaces the FC0 and FC1 layers of the Fiber Channel stack with Ethernet. By retaining the native Fiber Channel constructs, FCoE was meant to integrate with existing Fiber Channel networks and management software.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s