Cisco 64G Module: Enabling The Most Power Efficient SANs

The need for speed and sustainability

With the ever growing amount of data every organization manages, there is an associated need for a higher data retrieval speed, as demanded by Business Intelligence and Artificial Intelligence applications. Hence, the introduction of 64G Fibre Channel support on storage networking devices appears as a no brainer, especially when used in combination with the performance-optimized NVMe/FC protocol.

Read More: 352-001: CCDE Design Written Exam (CCDE)

At the same time, we are living in a context where sustainability efforts are mounting and pressing for power efficient solutions that can deliver more bandwidth at a reduced wattage.

Designing a power efficient SAN

Eight years ago Cisco launched the MDS 9700 family of mission critical directors with 16G switching modules. In 2017, green initiatives and power saving efforts became front and center, as reflected in the design of the 32G switching module and its major step forward in that direction. Continuing on the same path, the recently introduced 48 ports 64G switching module can really be described as a breakthrough technological epiphany. It provides 4 times the bandwidth of the original 16G switching module and shaves power consumption by approximately 40%, making it about 7 times more power efficient than its predecessor.

In combination with the highly efficient 80Plus platinum certified power supplies and the power-reduced chassis infrastructure elements, namely supervisor Sup-4 and crossbar fabric unit Fab-3, the new 64G switching module makes Cisco MDS 9700 directors the ideal choice for designing the most power efficient SANs.

New ASIC, new capabilities

This achievement is the result of a painstaking work to optimize both the physical footprint and power envelope of the new F64 switching ASIC, sitting inside the 64G switching module. This dual-die chipset can switch traffic for all the 48 ports at full line rate, freeing up space on the module motherboard.

The F64 ASIC incorporates numerous port counters and an efficient rate limiter, offering a combination of advanced congestion detection and mitigation techniques. This way the high-speed ports do not go underutilized due to roadblocks in the fabric.

Also, the entire switching module design was revisited to minimize latency, accommodate low energy components, and facilitate cooling.

Experiencing the full symphony

It is not unusual that the front-face of a high port count Fibre Channel switching module resembles an extra-long mouth-organ, but it is when you slide it inside an MDS 9700 chassis that you can appreciate the full symphony. Full line rate switching at 64G on all ports with no oversubscription, massive allocation of buffer to buffer credits for longer distances, traffic encryption for secure data transfer over ISLs, popular enterprise-class features like VSANs and PortChannels, hardware-assisted congestion prevention and mitigation solutions, including Dynamic Ingress Rate Limiting (DIRL) and Fabric Performance Impact Notification (FPIN) and Congestion Signals. All of that with a typical power consumption of only 300 Watts or 240W when operated at 64G or 32G respectively.

SAN Analytics on steroids

The low power consumption appears even more impressive when you consider that the 64G switching module comes with a dedicated onboard Network Processing Unit (NPU). It complements the ASIC-assisted metric computation and adds flexibility to the widely popular SAN Analytics capability of MDS 9000 switches.

The new 64G switching module raises the bar once again in terms of deep and granular traffic visibility. It uses an improved lens for frame headers inspection. It offers the capability to recognize VM-level identifiers. It provides a more refined analysis and correlation engine.

The presence of a dedicated 1G port on the switching module for streaming the telemetry data makes sure the scalability of the SAN Analytics feature can go beyond what is possible on the 32G modules. This is unrivalled, both in terms of self-discovered I/O Fibre Channel flows and computed metrics. Specific optimizations for a better handling of NVMe/FC traffic will also see the light when the SAN Analytics feature becomes available on this linecard.

Investment Protection

Investment protection is always good news for customers and the new 64G switching module excels in this area. It is supported inside Cisco MDS 9706, MDS 9710 and MDS 9718 mission critical directors. It can coexist and interoperate with previous generation of linecards, in any of these chassis. Existing MDS 9700 customers can decide to upgrade their install base to 64G speed without any service interruption. They can even migrate old SFPs to the new switching module, for those ports that do not need to be operated at 64G. This is what we call real investment protection.

At Cisco we are very proud of what our talented engineering team has been able to realize with this new 64G switching module. We hope you will feel the same when turning up the traffic volume in your SAN.