Login
- Table of Contents
- Related Documents
-
| Title | Size | Download |
|---|---|---|
| 01-IPCC configuration | 144.20 KB |
Configuring CNP statistics collection
Configuring NAK packet statistics collection
Verifying and maintaining IPCC
Displaying and clearing packet statistics
Display and maintenance commands for IPCC
Configuring IPCC
About IPCC
Intelligent Proactive Congestion Control (IPCC) is an improvement of Explicit Congestion Notification (ECN). It intelligently identifies the interface congestion state and proactively sends Congestion Notification Packets (CNPs) to control the RDMA over Converged Ethernet version 2 (RoCEv2) packet sending rate, ensuring low latency and high throughput. For more information about ECN, see QoS configuration in ACL and QoS Configuration Guide.
RoCEv2 packets
IPCC takes effect only on RoCEv2 packets. RoCEv2 uses UDP encapsulation. Figure 1 shows the RoCEv2 packet format.
· Ethernet header—Contains source and destination MAC addresses.
· IP header—Contains source and destination IP addresses.
· UDP header—Contains source and destination port numbers. The destination port number of RoCEv2 packets is 4791.
· Payload—Message payload.
· CRC—Cyclic redundancy check.
· FCS—Frame check sequence.
· InfiniBand Base Transport Header—Contains key information about an RoCEv2 packet. IPCC is related to the following fields:
¡ Opcode—Indicates the RoCEv2 packet type, which can be one of the following:
- ConnectMsg—Communication management (CM) connection setup packet. The packet is used for RoCEv2 connection setup. The sender and receiver use the RoCEv2 connection to exchange data packets.
- Send—Send packet. The sender uses the packet to request data from the receiver, without specifying the address where the receiver stores data.
- Write—Write packet. The packet carries the address, key, and length of data to be written to the receiver.
- Read—Read packet. The packet carries the address, key, and length of data to be read from the receiver.
- ACK—ACK packet. The packet is a response message returned by the receiver.
The Send, Write, and Read RoCEv2 packets are RoCEv2 data packets.
¡ Dest QP—Destination Queue Pair (QP), identifying an RoCEv2 flow. This field is similar to the destination port number. It is a key value for setting up an RoCEv2 flow table.
IPCC mechanism
IPCC is usually deployed in data center networks. Enable IPCC on the interface that receives packets on the forwarding device to implement intelligent congestion control.
As shown in Figure 2, IPCC works as follows:
1. Set up an RoCEv2 flow table.
After you enable IPCC for an interface on the forwarding device, the interface replicates RoCEv2 data packets passing the interface and sends the packets to the CPU. The CPU processes the packets as follows:
a. Generates an RoCEv2 flow table based on the quadruple information (client IP, server IP, client QP, and server QP) in the RoCEv2 connection setup packets.
b. Associates the RoCEv2 flow table with the interface based on the RoCEv2 data packets.
By maintaining the RoCEv2 flow table, the forwarding device can obtain address and forwarding path information about each RoCEv2 flow.
2. Calculate the number of CNPs to be sent.
After you enable IPCC for a queue on the interface, the forwarding device monitors the length of the queue and calculates the number of CNPs to be sent as follows:
¡ When the length of the queue increases, the forwarding device needs to send a small number of CNPs if the cache usage of the queue is low. If the cache usage of the queue is high, the forwarding device needs to send a large number of CNPs to reduce queue congestion and forwarding latency.
¡ When the length of the queue decreases, the forwarding device does not need to send CNPs if the cache usage of the queue is low. If the cache usage of the queue is high, the forwarding device needs to send a small number of CNPs to reduce queue congestion while ensuring throughput and latency performance.
3. Construct and send CNPs.
The forwarding device constructs CNPs based on the calculated number and address information in the RoCEv2 flow table and sends the CNPs to the sender. Then, the sender reduces the RoCEv2 packet sending rate.
Figure 2 IPCC mechanism
IPCC tasks at a glance
To configure IPCC, perform the following tasks:
3. (Optional.) Specifying an IPCC mode
4. (Optional.) Configuring CNP statistics collection
5. (Optional.) Configuring NAK packet statistics collection
Enabling IPCC
Restrictions and guidelines
IPCC takes effect on only RoCEv2 packets.
For IPCC to take effect on an interface, you must also enable IPCC for a queue on the interface. To enable IPCC for a queue, execute the ipcc queue command.
IPCC performs RoCEv2 traffic analysis based on global bidirectional flows. If IPCC and NetAnalysis are both enabled at the same time, you must specify the bidirectional mode for NetAnalysis and cannot specify the acl keyword for the netanalysis rocev2 statistics command. For more information about NetAnalysis, see NetAnalysis configuration in Networking Management and Monitoring Configuration Guide.
Procedure
1. Enter system view.
system-view
2. Enter interface view.
interface interface-type interface-number
3. Enable IPCC.
ipcc enable
By default, IPCC is disabled.
Enabling IPCC for a queue
About this task
After you enable IPCC for a queue, the forwarding device monitors the length of the queue and calculates the number of CNPs to be proactively sent. Then, the forwarding device constructs CNPs based on the calculated number and address information in the RoCEv2 flow table and sends the CNPs to the sender. After receiving the CNPs, the sender reduces the RoCEv2 packet sending rate.
Restrictions and guidelines
For IPCC to take effect on a queue, you must also enable IPCC for the interface of the queue. To enable IPCC for an interface, execute the ipcc enable command.
Procedure
1. Enter system view.
system-view
2. Enable IPCC for the specified queue.
ipcc queue queue-id
By default, IPCC is disabled for a queue.
Specifying an IPCC mode
About this task
Configure IPCC high throughput mode for services with a high traffic volume and low latency sensitivity. Configure IPCC low latency mode for services with a low traffic volume and high latency sensitivity.
Procedure
1. Enter system view.
system-view
2. Specify an IPCC mode.
ipcc mode { high-throughput | low-latency }
By default, IPCC operates in high throughput mode.
Configuring CNP statistics collection
About this task
On an RoCEv2 network, if the destination server receives packets with ECN, it sends CNPs to the source server to notify the source server to reduce the rate at which it sends packets, thereby alleviating congestion.
This command allows the device to collect CNP statistics on interfaces. To view CNP statistics, execute display cnp statistics.
Restrictions and guidelines
If you configure this feature on both an aggregate interface and its member ports, the device collect packet statistics only on the member ports. If some member ports of an aggregate interface are not configured with this feature, the aggregate interface will collect packet statistics only from the member ports that are not configured with this feature.
CNP statistics collection is supported only in the incoming direction of an interface.
Procedure
Enter system view.
system-view
Enter interface view.
interface interface-type interface-number
Configure CNP statistics collection.
cnp statistics enable
By default, CNP statistics collection is disabled.
Configuring NAK packet statistics collection
About this task
On an RoCEv2 network, if the destination server detects an error packet, such as packet loss or packet corruption, it generates and sends a NAK packet to the source server. Upon receiving the NAK packet, the source server identifies the packet that requires retransmission based on NAK information, and then retransmits the packet to ensure data integrity and reliability.
This command allows the device to collect NAK packet statistics on interfaces. To view NAK packet statistics, execute display nak statistics.
Restrictions and guidelines
If you configure this feature on both an aggregate interface and its member ports, the device collect packet statistics only on the member ports. If some member ports of an aggregate interface are not configured with this feature, the aggregate interface will collect packet statistics only from the member ports that are not configured with this feature.
NAK statistics collection is supported only in the incoming direction of an interface.
Procedure
1. Enter system view.
system-view
2. Enter interface view.
interface interface-type interface-number
3. Configure NAK statistics collection.
nak statistics enable
By default, NAK statistics collection is disabled.
Verifying and maintaining IPCC
Displaying and clearing packet statistics
Displaying packet statistics
Perform display tasks in any view.
· Display statistics about CNPs that are proactively sent by the device.
display ipcc statistics [ interface interface-type interface-number ]
· Display CNP statistics.
display cnp statistics interface [ interface-type [ interface-number ] ]
· Display NAK packet statistics.
display nak statistics interface [ interface-type [ interface-number ] ]
Clearing packet statistics
Perform reset tasks in user view.
· Clear statistics about CNPs that are proactively sent by the device.
reset ipcc statistics [ interface interface-type interface-number ]
· Clear CNP statistics.
reset cnp statistics interface [ interface-type [ interface-number ] ]
· Clear NAK packet statistics.
reset nak statistics interface [ interface-type [ interface-number ] ]
Display and maintenance commands for IPCC
Execute display commands in any view.
|
Task |
Command |
|
Display statistics of CNPs that are proactively sent by the device. |
display ipcc statistics interface interface-type interface-number |
|
Clear statistics of CNPs that are proactively sent by the device. |
reset ipcc statistics [ interface interface-type interface-number ] |
IPCC configuration examples
Example: Configuring IPCC
Network configuration
As shown in Figure 3, the sender is deployed with storage services carried by RoCEv2. Enable IPCC on interface FourHundredGigE 1/0/2 on Device to control the RoCEv2 packet sending rate of the sender, ensuring low latency and high throughput for the RoCEv2 service.
Procedure
1. Configure IP addresses and subnet masks for the interfaces, as shown in Figure 3. (Details not shown.)
2. Configure IPCC:
# Enable IPCC.
<Device> system-view
[Device] interface fourhundredgige 1/0/2
[Device-FourHundredGigE1/0/2] ipcc enable
[Device-FourHundredGigE1/0/2] quit
# Enable IPCC for queue 4.
[Device] ipcc queue 4
# Configure IPCC to operate in low latency mode.
[Device] ipcc mode low-latency
Verifying the configuration
# View statistics of CNPs that are proactively sent by the device.
<Device> display ipcc statistics interface gigabitethernet 1/0/1
CNP number: 3
RoCEv2 flow entries active: 1
Active flow information:
---------------------------------------------
DIP SIP DQP
---------------------------------------------
12.110.2.2 11.110.2.2 2090
----------------------------------------------