Cisco DIRL offers solutions to SAN congestion without any dependency on hosts or storage.
We announced the release of Cisco® Dynamic Ingress Rate Limiting (DIRL), a major innovation to alleviate SAN congestion. Cisco DIRL is a reality today and promises to offer a practical solution to SAN congestion from a fabric perspective without any dependency on the host or storage. If you have not had a chance to review this new innovative technology, I would recommend reviewing the video, blog, presentation, solution overview or the Interfaces Configuration Guide.
In this writeup, I will cover the core of Cisco DIRL so that you can gain a solid understanding of this new innovative technology that does not require any changes to the host or storage. I will not cover how Cisco DIRL solves congestion, please refer to the web links listed above for that information.
First let’s cover the basics.
Fibre Channel (FC) Buffer to Buffer crediting
FC is built on the premise of offering a lossless fabric tailor made for storage protocols. Lossless fabric means that the probability of frame drop inside the switches and the interconnecting links is kept to a minimum. This guarantee is important to meet the performance requirements of storage protocols like SCSI/FICON/NVMe.
While FC switches implement various schemes to avoid frame drops within the switches, the way no frame drop is achieved on links interconnecting two FC ports is through a mechanism known as Buffer to Buffer (B2B) crediting. Let me break down the B2B crediting concept:
◉ During a FC link up, the number of receive buffers on a port is exchanged as B2B credits with its peer port (during FLOGI/ACC on a F-Port and ELP/ACC on a E-port). The transmitters at either ends set a TxBB counter = number of receive buffers on the peer port.
◉ An R_RDY primitive is used to indicate the availability of one buffer on the receive side of the port sending the R_RDY.
◉ As traffic starts flowing on the link, the R_RDY is used to constantly refresh the receive buffer levels to the transmitter. At the transmitter, every transmitted frame decreases TxBB counter by 1 and every R_RDY received from the peer increases the TxBB counter by 1. If TxBB counter drops to 0, no frame can be transmitted. Any frame needing to be sent has to wait till an R_RDY is received. At the receiver end, as and when incoming frames are processed and switched out, its ready to receive another frame and an R_RDY is sent back to the sender. This control loop constantly runs bidirectionally on every FC link. It ensures the transmit end of an FC port can transmit a frame only if the receiving port has a buffer to receive the frame.