Blog Feed Post

Cables, Transceivers and 10GBASE-TX

In the past few weeks at Plexxi we spend probably an unreasonable amount of time talking about, discussing and even arguing over ethernet cables and connectors. As mundane as it may sound, the options, variations, restrictions and cost variations of something that is usually an afterthought is mind boggling. And as a buyer of ethernet networks, you have probably felt that the choices you make will significantly change the price you pay for the total solution.

During our quarterly Product Management get together, my colleague Andre Viera took 25GbE as a trigger to walk the rest of the team through all the variations of cables and transceivers. As a vendor it is a rather complicated topic and as a customer I can only imagine how the choices may put you in a bad mood.

Most of today’s 10GbE switches ship with SFP+ cages and a handful of QSFP cages. Now comes the hard part. What do I plug into these cages? There are lots of choices all with their own pros and cons.

Direct Attach Cable

The cheapest solution is a Direct Attach Cable or DAC. These are copper based cables that have SFP+ transceivers molded onto the cable. It comes a single entity, one end plugs into the switch, the other into the server of storage with an SFP cage. It’s cheap, the cheapest cabling solution around today. A three meter DAC cable should cost you less than $100 even when buying from your equipment vendor. It’s power friendly, consuming less than half a watt for the entire connection.The only downside is their limited distance (usually up to 10M), which may be even more limited depending on whether the Ethernet switch is built with PHY interface that can drive signals over a longer distance. Overall, this should probably be your first option to look at.

Active Optical Cable

Very similar to the DAC cable above, the Active Optical Cable is a single cable with transceivers on both ends attached. The cable in between them however is a fiber optic cable. This allows these AOCs to be driven across a much larger distance, you can buy 100 meter AOCs, probably longer distances if you search a bit further. That distance comes at a cost. Don’t be surprised to see these cables at 3-4x the price of a DAC cable from your favorite vendor.

SFP+ Transceivers

The most flexible option is to use individual SFP+ transceivers with fiber optic cables between them. But that flexibility is because there are so many options. Not exhausting:

  • SX or SR – uses 850nm wave length, targeted at multimode fiber installations, maximum distance of about 500 meters
  • MX or MR (or sometimes IR) – uses 1310nm wave length, single mode, distance of about 2km
  • LX or LR – uses 1310nm wave length, single mode only, distance up to 10km
  • EX or ER – uses 1310nm wave length, single mode only, up to 40km
  • ZX or ZR – uses 1550nm wave length, single mode only, up to 80km
  • EZX or EZR – 1550nm, single mode, up to 160km
  • and a bunch more obscure flavors (http://en.wikipedia.org/wiki/Ethernet_physical_layer)

There are really two decisions points in the choice of transceivers. What fiber you have installed or want to use is the first choice. Distance requirements obviously drives the final choice. Inside of a rack, SR with multimode fiber patches is no doubt the cheapest.

But, most vendors only allow you to use their branded transceivers. There are legitimate reasons behind this choice. I have supported networks long enough to see weird network problems caused by extremely questionable transceivers and most vendors want or need to provide some quality guarantees. But that should not mean they are expensive. There is no reason you should pay more than a few hundred bucks for a vendor supported quality SR SFP+ transceiver, similarly for a MR/IR single mode transceiver.

Once you get to the LX, EX, ZX and beyond, the price will go up, but those are really targeted at campus or metro based solutions. For specific DWDM channeled transceiver, don’t be shocked to see list prices near $10,000.

Enter 10GBASE-TX

A recent addition to 10GbE connectivity (it took a while to get the standards done because the desired distance posed some interesting physics challenges), 10GBASE-TX brings a Cat6 based twisted cabling using good old RJ45 connectors to 10GbE networking. Build for 100 meters, the actual Cat6 cabling is by far the cheapest out of all choices. But like everything else, it comes with a price tag in a different form. Today’s initial 10GBASE-T solutions will consume 4-6 Watts per port. Compare that to the 0.5-1 Watt for SFP+ based solutions. It also increases the entire switch latency by a factor of 5 or more. The overall cost may appear attractive, what could a stretch of a few meters of Cat6 cable cost, but there are additional costs in both the switch and the host NIC card which for a while to come will still make the overall solution more expensive than direct attach cables. Most of today’s 10GbE ToR switches are built without a component called a PHY. The PHY is a piece of hardware that drives a 10GbE port. Today’s ethernet chipsets allow you to create a switch without a PHY, but only for SFP+ and QSFP connections. To support 10GBASE-TX, the switch needs PHYs for each copper port, and those components have a price.

Interesting in all these choices is actually not what choice you make (assuming you are within the distance of various options), but who makes that choice. Is it the network folks that drive the type of connectivity they will demand from their server and storage attachments, or is it the server and storage folks that will be driving, simply by the virtue of server vendors putting certain types of connections natively on their system boards?

The first 10GBASE-TX only based servers have started to appear. It will be a while before this becomes true across the board. Broad adoption of 10GBASE-TX is probably another year away. And within that, certain market segments will rotate towards 10GBASE-TX, especially those customer, solutions and segments that will require a mix of their current 1GbE and up-and-coming 10GbE attached devices. Other segments are SFP+ or even QSFP (with breakout cables) and will stay there forever.

If I had a choice, I would pick SFP/QSFP switches with direct attach cables every time. It keeps my switches flexible. SFP+ or QSFP based switches can be used in just about any application I have. While 10GBASE-TX gives me 100 meter worth of distance, I would rather not use copper outside of a rack. Or deal with the bulk inside the rack like the picture above. And not having to pay for the extra 250 Watt per switch is a nice bonus. SFP+ and QSFP DACs should not cost me an arm and a leg. If it does, complain to your vendor.


[Today's fun fact: The word copper and chemical symbol for the element, Cu, come from the Latin word cuprum after the Greek word for the island of Cyprus. Maybe the fun part was missing from this one.]

The post Cables, Transceivers and 10GBASE-TX appeared first on Plexxi.

Read the original blog entry...

More Stories By Marten Terpstra

Marten Terpstra is a Product Management Director at Plexxi Inc. Marten has extensive knowledge of the architecture, design, deployment and management of enterprise and carrier networks.

Latest Stories
Is advanced scheduling in Kubernetes achievable? Yes, however, how do you properly accommodate every real-life scenario that a Kubernetes user might encounter? How do you leverage advanced scheduling techniques to shape and describe each scenario in easy-to-use rules and configurations? In his session at @DevOpsSummit at 21st Cloud Expo, Oleg Chunikhin, CTO at Kublr, will answer these questions and demonstrate techniques for implementing advanced scheduling. For example, using spot instances ...
DevOps is under attack because developers don’t want to mess with infrastructure. They will happily own their code into production, but want to use platforms instead of raw automation. That’s changing the landscape that we understand as DevOps with both architecture concepts (CloudNative) and process redefinition (SRE). Rob Hirschfeld’s recent work in Kubernetes operations has led to the conclusion that containers and related platforms have changed the way we should be thinking about DevOps and...
SYS-CON Events announced today that Taica will exhibit at the Japan External Trade Organization (JETRO) Pavilion at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Taica manufacturers Alpha-GEL brand silicone components and materials, which maintain outstanding performance over a wide temperature range -40C to +200C. For more information, visit http://www.taica.co.jp/english/.
When it comes to cloud computing, the ability to turn massive amounts of compute cores on and off on demand sounds attractive to IT staff, who need to manage peaks and valleys in user activity. With cloud bursting, the majority of the data can stay on premises while tapping into compute from public cloud providers, reducing risk and minimizing need to move large files. In his session at 18th Cloud Expo, Scott Jeschonek, Director of Product Management at Avere Systems, discussed the IT and busine...
SYS-CON Events announced today that SourceForge has been named “Media Sponsor” of SYS-CON's 21st International Cloud Expo, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. SourceForge is the largest, most trusted destination for Open Source Software development, collaboration, discovery and download on the web serving over 32 million viewers, 150 million downloads and over 460,000 active development projects each and every month.
Enterprises have taken advantage of IoT to achieve important revenue and cost advantages. What is less apparent is how incumbent enterprises operating at scale have, following success with IoT, built analytic, operations management and software development capabilities – ranging from autonomous vehicles to manageable robotics installations. They have embraced these capabilities as if they were Silicon Valley startups. As a result, many firms employ new business models that place enormous impor...
SYS-CON Events announced today that TidalScale will exhibit at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. TidalScale is the leading provider of Software-Defined Servers that bring flexibility to modern data centers by right-sizing servers on the fly to fit any data set or workload. TidalScale’s award-winning inverse hypervisor technology combines multiple commodity servers (including their ass...
As popularity of the smart home is growing and continues to go mainstream, technological factors play a greater role. The IoT protocol houses the interoperability battery consumption, security, and configuration of a smart home device, and it can be difficult for companies to choose the right kind for their product. For both DIY and professionally installed smart homes, developers need to consider each of these elements for their product to be successful in the market and current smart homes.
SYS-CON Events announced today that MIRAI Inc. will exhibit at the Japan External Trade Organization (JETRO) Pavilion at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. MIRAI Inc. are IT consultants from the public sector whose mission is to solve social issues by technology and innovation and to create a meaningful future for people.
In his general session at 21st Cloud Expo, Greg Dumas, Calligo’s Vice President and G.M. of US operations, will go over the new Global Data Protection Regulation and how Calligo can help business stay compliant in digitally globalized world. Greg Dumas is Calligo's Vice President and G.M. of US operations. Calligo is an established service provider that provides an innovative platform for trusted cloud solutions. Calligo’s customers are typically most concerned about GDPR compliance, applicatio...
Companies are harnessing data in ways we once associated with science fiction. Analysts have access to a plethora of visualization and reporting tools, but considering the vast amount of data businesses collect and limitations of CPUs, end users are forced to design their structures and systems with limitations. Until now. As the cloud toolkit to analyze data has evolved, GPUs have stepped in to massively parallel SQL, visualization and machine learning.
In his Opening Keynote at 21st Cloud Expo, John Considine, General Manager of IBM Cloud Infrastructure, will lead you through the exciting evolution of the cloud. He'll look at this major disruption from the perspective of technology, business models, and what this means for enterprises of all sizes. John Considine is General Manager of Cloud Infrastructure Services at IBM. In that role he is responsible for leading IBM’s public cloud infrastructure including strategy, development, and offering ...
As hybrid cloud becomes the de-facto standard mode of operation for most enterprises, new challenges arise on how to efficiently and economically share data across environments. In his session at 21st Cloud Expo, Dr. Allon Cohen, VP of Product at Elastifile, will explore new techniques and best practices that help enterprise IT benefit from the advantages of hybrid cloud environments by enabling data availability for both legacy enterprise and cloud-native mission critical applications. By rev...
SYS-CON Events announced today that Dasher Technologies will exhibit at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 - Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Dasher Technologies, Inc. ® is a premier IT solution provider that delivers expert technical resources along with trusted account executives to architect and deliver complete IT solutions and services to help our clients execute their goals, plans and objectives. Since 1999, we'v...
SYS-CON Events announced today that NetApp has been named “Bronze Sponsor” of SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. NetApp is the data authority for hybrid cloud. NetApp provides a full range of hybrid cloud data services that simplify management of applications and data across cloud and on-premises environments to accelerate digital transformation. Together with their partners, NetApp emp...