The networking industry is currently testing 400GbE products and is already looking forward to how the next generation will be built. The past few years were full of industry debate around the “right” pluggable form factor for 400GbE. However, that debate seems to have ended, resulting in nearly every Ethernet equipment manufacturer building QSFP-DD based solutions.
The motivations for sticking with the QSFP-based form factor are clear. There is always intense market pressure to quickly accelerate to a single form factor and for good reason. A single form factor drives significant cost reductions, and cost reduction is critical at high volumes. LightCounting estimates (gated content) that volume from the first five years of 400GbE modules will be 20x larger than the first five years of 100GbE, due to simultaneous adoption in multiple markets, including both Service Provider and Cloud Data Center.
This overlap of key markets all wanting to deploy 400GbE simultaneously means the industry cannot spend the next five years optimizing form factors like it did for 100GbE. With every system vendor now building 400GbE products based on QSFP-DD, 400GbE will be the first speed transition where the initial form factor will also be the high-volume, dense form factor that can support all reaches and media.
There were some critical lessons learned from the 100GbE journey that should be applied to 400GbE. Even though a dense form factor called CFP4 was defined and built, it lacked backwards compatibility with the dense 40GbE QSFP+ module and, as a result, was ignored. There is no reason to suggest something similar won’t happen in the 400GbE module market if we don’t learn from the past.
It’s important to consider how it was possible to extend the QSFP form factor from 40GbE to 400GbE, in order to better understand what’s possible beyond 400GbE.
As the co-chair of the QSFP-DD MSA, I had the unique experience to watch the collective motivation and intense collaboration that went on to bring QSFP-DD to market. With the goal of enabling the optical module costs to reduce as fast as possible, it was clear that a single form factor that supported all reaches was mandatory. History has shown this is the key to volume adoption in the networking industry. It was also agreed that supporting backwards compatibility had operational value to many and would further accelerate the desired volume adoption/cost reductions.
Given the success of QSFP28 for 100GbE, it was evident that developing a compatible solution would also be a success, if achievable. However, it wasn’t going to be easy and required overcoming technical risk. It would be far less difficult to relax the design goals and start from scratch, but our innovation experience told us the risk was manageable and it was possible to properly address all of the challenges.
Just within Cisco, we have developed nearly 300 unique linecard designs based on QSFP modules. This enormous body of experience informed us it was possible to innovate and address the thermal and high-speed electrical challenges. We knew from our close relationships with the leading component technology developers that they could see a path to fitting all the necessary components into the available space and again it was an acceptable risk. Advanced integration was enabling solutions all the way from 3m copper cables to Metro-reach coherent optical modules. The main concern was if the systems could drive and cool the modules.
Two years later, we are building and testing QSFP-DD systems and modules capable of driving 400 Gbps Ethernet electrical interfaces and being able to cool 20W modules with margin. The 20W target enables the 400ZR+ coherent DWDM optical modules capable of an impressive reach of greater than 1000km. The widely supported QSFP-DD MSA is now working to update its specs with these latest thermal capabilities. And all this without sacrificing backwards compatibility leading us to have confidence of broad industry adoption and market success.
Figure 1: QSFP-DD module showing integrated heatsink on nose that enables 20W system cooling. This is anticipated to support any 400ZR+ variant
The deep experience that Cisco and the industry has with QSFP-based designs has enabled this continual innovation. According to Dell’Oro, by the end of 2019, approximate 70 million QSFP (all rates) ports will have cumulatively shipped making it clear why supporting backwards compatibility supports many users in their network operations or investment protection goals.
While 400GbE deployments of QSFP-DD are at the start of their long deployment cycle, we’re already looking forward to what comes next. With the development of 100 Gb/s electrical SerDes happening in IEEE 802.3, we can expect future ASICs to be driving 100 Gbps signals towards these modules. QSFP modules again offer a clear advantage as we can use QSFP112 modules for 400 GbE interfaces (such as 400GBASE-DR4) and also QSFP112-DD for 800GbE capable modules (such as dual 400GbE). The interchange of these will be equally important as well as the backwards compatibility with the QSFP56-DD that we’ve been talking about for 400GbE.
The experience and innovation brought to bear in making this first generation of QSFP-DD feasible for 400GbE has opened a number of innovation opportunities that allow us to have confidence that supporting 800GbE will happen. Multiple system design approaches and configurations are already in the labs testing out support for 100
Gbps electrical SerDes and are looking very positive. The advances and innovation in thermal performances that we’ve seen already for the 400GbE work are not at the limits of what is possible as we further innovate and make progress. All the while, power is being reduced as chips move to 7nm from the current 16nm processes.
More than decade ago, when QSFP+ was in its early days of 40GbE development no one would have predicted that we’d be considering 800GbE variants. However, the market success of the approach and the flexibility of system design has driven a continual series of innovations to match the ongoing market needs. We’ve not seen the end of this and QSFP based modules have a strong and healthy future ahead. The networking industry is privileged to have so much technical, commercial and deployment experience with these modules that it continues to be foundational in everything we build.
The motivations for sticking with the QSFP-based form factor are clear. There is always intense market pressure to quickly accelerate to a single form factor and for good reason. A single form factor drives significant cost reductions, and cost reduction is critical at high volumes. LightCounting estimates (gated content) that volume from the first five years of 400GbE modules will be 20x larger than the first five years of 100GbE, due to simultaneous adoption in multiple markets, including both Service Provider and Cloud Data Center.
This overlap of key markets all wanting to deploy 400GbE simultaneously means the industry cannot spend the next five years optimizing form factors like it did for 100GbE. With every system vendor now building 400GbE products based on QSFP-DD, 400GbE will be the first speed transition where the initial form factor will also be the high-volume, dense form factor that can support all reaches and media.
There were some critical lessons learned from the 100GbE journey that should be applied to 400GbE. Even though a dense form factor called CFP4 was defined and built, it lacked backwards compatibility with the dense 40GbE QSFP+ module and, as a result, was ignored. There is no reason to suggest something similar won’t happen in the 400GbE module market if we don’t learn from the past.
It’s important to consider how it was possible to extend the QSFP form factor from 40GbE to 400GbE, in order to better understand what’s possible beyond 400GbE.
Necessity is the Mother of Invention
As the co-chair of the QSFP-DD MSA, I had the unique experience to watch the collective motivation and intense collaboration that went on to bring QSFP-DD to market. With the goal of enabling the optical module costs to reduce as fast as possible, it was clear that a single form factor that supported all reaches was mandatory. History has shown this is the key to volume adoption in the networking industry. It was also agreed that supporting backwards compatibility had operational value to many and would further accelerate the desired volume adoption/cost reductions.
Given the success of QSFP28 for 100GbE, it was evident that developing a compatible solution would also be a success, if achievable. However, it wasn’t going to be easy and required overcoming technical risk. It would be far less difficult to relax the design goals and start from scratch, but our innovation experience told us the risk was manageable and it was possible to properly address all of the challenges.
Just within Cisco, we have developed nearly 300 unique linecard designs based on QSFP modules. This enormous body of experience informed us it was possible to innovate and address the thermal and high-speed electrical challenges. We knew from our close relationships with the leading component technology developers that they could see a path to fitting all the necessary components into the available space and again it was an acceptable risk. Advanced integration was enabling solutions all the way from 3m copper cables to Metro-reach coherent optical modules. The main concern was if the systems could drive and cool the modules.
Two years later, we are building and testing QSFP-DD systems and modules capable of driving 400 Gbps Ethernet electrical interfaces and being able to cool 20W modules with margin. The 20W target enables the 400ZR+ coherent DWDM optical modules capable of an impressive reach of greater than 1000km. The widely supported QSFP-DD MSA is now working to update its specs with these latest thermal capabilities. And all this without sacrificing backwards compatibility leading us to have confidence of broad industry adoption and market success.
The deep experience that Cisco and the industry has with QSFP-based designs has enabled this continual innovation. According to Dell’Oro, by the end of 2019, approximate 70 million QSFP (all rates) ports will have cumulatively shipped making it clear why supporting backwards compatibility supports many users in their network operations or investment protection goals.
Beyond 400GbE to 800GbE
While 400GbE deployments of QSFP-DD are at the start of their long deployment cycle, we’re already looking forward to what comes next. With the development of 100 Gb/s electrical SerDes happening in IEEE 802.3, we can expect future ASICs to be driving 100 Gbps signals towards these modules. QSFP modules again offer a clear advantage as we can use QSFP112 modules for 400 GbE interfaces (such as 400GBASE-DR4) and also QSFP112-DD for 800GbE capable modules (such as dual 400GbE). The interchange of these will be equally important as well as the backwards compatibility with the QSFP56-DD that we’ve been talking about for 400GbE.
The experience and innovation brought to bear in making this first generation of QSFP-DD feasible for 400GbE has opened a number of innovation opportunities that allow us to have confidence that supporting 800GbE will happen. Multiple system design approaches and configurations are already in the labs testing out support for 100
Gbps electrical SerDes and are looking very positive. The advances and innovation in thermal performances that we’ve seen already for the 400GbE work are not at the limits of what is possible as we further innovate and make progress. All the while, power is being reduced as chips move to 7nm from the current 16nm processes.
In Praise of QSFP
More than decade ago, when QSFP+ was in its early days of 40GbE development no one would have predicted that we’d be considering 800GbE variants. However, the market success of the approach and the flexibility of system design has driven a continual series of innovations to match the ongoing market needs. We’ve not seen the end of this and QSFP based modules have a strong and healthy future ahead. The networking industry is privileged to have so much technical, commercial and deployment experience with these modules that it continues to be foundational in everything we build.
Posts
0 comments:
Post a Comment