Troubleshooting high availability

Troubleshooting BFD

BFD session establishment failure

Symptom

Execute the display bfd session command on the device. If no session information is displayed or the State field value in the output is not Up, the BFD session cannot be established.

Common causes

The following are the common causes of this type of issue:

·     The routing table has no routes to the destination address of the BFD session.

·     The path detected by BFD fails, and BFD packets cannot be correctly exchanged as a result.

Troubleshooting flow

Figure 1 shows the troubleshooting flowchart.

Figure 1 Flowchart for troubleshooting BFD session establishment failure

 

 

Solution

BFD uses a session established between two network devices to detect the bidirectional forwarding path between the devices for the upper-layer application. BFD itself does not have a discovery mechanism. It relies on the notifications from the associated upper-layer protocol to establish a session. After the upper-layer protocol establishes a new neighbor relationship, it sends the neighbor's parameters and detection parameters (including the destination address and source address) to BFD. BFD then establishes a session based on the received parameters. Once the session is established, BFD packets are quickly sent periodically. If BFD packets are not received within the detection interval, the bidirectional forwarding path is considered faulty, and fault information is sent to the upper-layer application, which takes corresponding actions. To accurately troubleshoot a BFD session establishment failure, make sure the upper-layer protocol operates correctly.

To resolve the issue:

1.     Execute the display bfd session command to identify whether BFD session information exists.

¡     If no BFD session information exists, proceed to step 2 and step 3.

¡     If BFD session information exists, but the State field value is Down, proceed to step 4.

2.     Identify whether an upper-layer protocol is associated with BFD.

Execute the display current-configuration command to identify whether an upper-layer protocol is associated with BFD. For example, the following output shows that OSPF is associated with BFD:

interface GigabitEthernet1/0/1

 ospf bfd enable

¡     If an upper-layer protocol is associated with BFD, proceed to step 3.

¡     If no upper-layer protocol is associated with BFD, execute the corresponding command to associate an upper-layer protocol with BFD and make sure the configuration is correct.

3.     Identify whether the number of BFD sessions exceeds the upper limit of the device.

Execute the display bfd session command to view the value of the Total sessions field. If the value reaches the device upper limit, new BFD sessions cannot be created. To solve this issue, delete some unnecessary BFD sessions by disabling BFD for some upper-layer protocols.

If the number of BFD sessions does not exceed the device upper limit, proceed to step 4.

4.     Identify whether the BFD routing or tunnel information is correct.

If BFD is used to detect the connectivity of an IP link, perform the following tasks to check the routing information:

a.     Execute the display bfd session command to view the IPv4 or IPv6 address displayed in the DestAddr field.

b.     Execute the display ip routing-table or display ipv6 routing-table command to identify whether a route destined for the address is available in the DestAddr field.

c.     If no routes exist, see the Layer 3—IP routing troubleshooting guide to troubleshoot the issue.

If a route exists but the BFD session cannot come up, proceed to step 5.

If BFD is used to detect the connectivity of an LSP, PW, VXLAN tunnel, MPLS TE tunnel, SRLSP, or SRv6 TE policy, see the troubleshooting guide for each module. If the tunnel state is abnormal, troubleshoot the tunnel issue. If the tunnel state is normal but the BFD session cannot come up, proceed to step 5.

5.     Identify whether BFD packet sending is correct.

Execute the display bfd session verbose command multiple times to view the value of the Tx count field, which indicates the number of sent packets. If the value remains 0, no BFD packets are sent. Perform the following tasks to check BFD packet sending:

a.     Execute the display current-configuration configuration bfd-static-session command to view the interface detected by the static BFD session. For example, the following output shows that GigabitEthernet 1/0/2 (after track-interface) is the interface detected by the static BFD session.

<Sysname> display current-configuration configuration bfd-static-session

#

bfd static chris peer-ipv6 1::2 source-ipv6 1::1 discriminator local 1000 remote 1010 track-interface GigabitEthernet1/0/2

#

b.     Execute the display interface interface-type interface-number command to view the running status of the interface. If the value of the Current state or Line protocol state field is not UP, troubleshoot the interface issue. If the interface is running correctly, proceed to step c.

c.     Execute the display bfd session command to view the value of the Init mode field, which indicates the BFD operating mode. A node operating in passive mode does not send BFD packets until it receives a BFD control packet from a node operating in active mode.

If the value of the Tx count field on the node operating in active mode keeps increasing, this node sends BFD packets correctly. In this case, proceed to step 6 to identify whether the node operating in passive mode can receive BFD packets correctly.

d.     If the BFD packet sending failure is caused by reasons other than those mentioned above, proceed to step 8.

6.     Identify whether BFD packet receiving is correct.

Execute the display bfd session verbose command multiple times on one end of the BFD session to view the value of the Rx count field, which indicates the number of packets received.

¡     If the value of the Rx count field remains 0, identify whether the peer end of the BFD session sends BFD packets correctly. If the peer end does not send BFD packets correctly, troubleshoot the packet sending failure on the peer end.

If the peer end sends BFD packets correctly, execute the display system internal bfd packet statistics command on the local end to view the data loss statistics in the The detailed discarded packet statistics area. If packet loss occurs, resolve the packet loss issue according to the reasons. If the issue cannot be resolved or no packet loss occurs, proceed to step 8.

¡     If the value of the Rx count field increases and the BFD session enters Init state, the local end can receive BFD packets. In this case, execute the display bfd session verbose command on the peer end of the BFD session to view the value of the Rx count field.

-     If the value of the Rx count field on the peer end remains 0 but the local end sends BFD packets correctly, the peer end of the BFD session does not receive packets correctly. This issue will cause the peer end to continually send BFD control packets in Down state, preventing the BFD session from coming up on the local end. Execute the display system internal bfd packet statistics command on the peer end to view the data loss statistics in the The detailed discarded packet statistics area. If packet loss occurs, resolve the packet loss issue according to the reasons. If the issue cannot be resolved or no packet loss occurs, proceed to step 8.

-     If the value of the Rx count field on the peer end remains 0 because the local end does not send BFD packets correctly, troubleshoot the packet sending failure on the local end.

If both ends send BFD packets correctly, but one end cannot receive BFD packets, proceed to step 7.

7.     Identify whether the link detected by the BFD session can forward packets correctly.

Execute the ping command to identify whether the link detected by the BFD session can forward packets correctly. Use different ping commands by link type, as shown in Table 1.

Table 1 Ping commands for different types of links

Link type	Ping command
IP links	ping ip ping ipv6
LSP tunnels	ping mpls ipv4
MPLS TE tunnels	ping mpls te
PWs	ping mpls pw
SRv6 TE policies	ping srv6-te policy

 

¡     If the other end cannot be pinged, see Ping and Tracert—Ping Failure Troubleshooting Guide, MPLS Troubleshooting Guide, and Segment Routing Troubleshooting Guide to troubleshoot tunnel issues.

¡     If the other end can be pinged, proceed to step 8.

8.     If the issue persists, collect the following information and contact Technical Support:

¡     Results of each step.

¡     The configuration file, log messages, and alarm messages.

Related alarm and log messages

Alarm messages

N/A

Log messages

N/A

BFD session flapping

Symptom

When the link is unstable, the CLI might frequently output logs about BFD session going down. For example:

%Jul 28 16:03:50:856 2022 H3C BFD/4/BFD_CHANGE_FSM: Sess[192.168.24.4/192.168.24.2, LD/RD:33793/33793, Interface:GE2/0/1, SessType:Ctrl, LinkType:INET], Ver:1, Sta: UP->DOWN, Diag: 7 (Administratively Down)

Common causes

The following are the common causes of this type of issue:

·     Physical link failures.

·     Upper-layer protocol failures.

·     Hardware failures.

Troubleshooting flow

The troubleshooting flow for this issue is as follows:

1.     Determine the reasons according to the BFD logs.

2.     Check the board hardware, physical link, upper-layer protocol state, route availability, and whether the tunnel is correctly established.

Figure 2 shows the troubleshooting flowchart.

Figure 2 Flowchart for troubleshooting BFD session flapping.

 

 

Solution

CAUTION: ·     Output of excessive debugging messages increases the CPU usage and affects the system operation. To guarantee system performance, enable debugging for only the specified modules. ·     Save the execution results of the following steps. If the issue cannot be resolved, contact Technical Support.

 

BFD uses a session established between two network devices to detect the bidirectional forwarding path between the devices for the upper-layer application. BFD itself does not have a discovery mechanism. It relies on the notifications from the associated upper-layer protocol to establish a session. After the upper-layer protocol establishes a new neighbor relationship, it sends the neighbor's parameters and detection parameters (including the destination address and source address) to BFD. BFD then establishes a session based on the received parameters. Once the session is established, BFD packets are quickly sent periodically. If BFD packets are not received within the detection interval, the bidirectional forwarding path is considered faulty, and fault information is sent to the upper-layer application, which takes corresponding actions. To accurately troubleshoot a BFD session establishment failure, make sure the upper-layer protocol operates correctly.

To resolve the issue:

1.     Identify whether the BFD session state changes from Init to Down.

The following log message outputted by the CLI indicates that the BFD session state changes from Init to Down:

BFD/4/BFD_CHANGE_FSM: Sess[20.0.4.2/20.0.4.1,LD/RD:533/532, Interface:Vlan204, SessType:Ctrl, LinkType:INET], Ver.1, Sta: INIT->DOWN, Diag: 1 (Control Detection Time Expired).

a.     Identify whether the link detected by the BFD session can forwards packets correctly.

Execute the ping command to identify whether the link detected by the BFD session can forward packets correctly. Use different ping commands for different types of links, as shown in Table 2.

Table 2 Ping commands for different types of links

Link type	Ping command
IP links	ping ip or ping ipv6
LSP tunnels	ping mpls ipv4
MPLS TE tunnels	ping mpls te
PWs	ping mpls pw
SRv6 TE policies	ping srv6-te policy

 

-     If the other end cannot be pinged successfully, see Ping and Tracert—Ping Failure Troubleshooting Guide, MPLS Troubleshooting Guide, and Segment Routing Troubleshooting Guide to troubleshoot tunnel issues.

-     If the other end can be pinged successfully, proceed to step b.

b.     Check the BFD packet receiving on the local end.

Execute the debugging bfd packet receive command to enable debugging for received BFD packets.

-     If the value of the Sta field in the debugging information is 1 or no debugging information is displayed, it indicates that the local end receives BFD packets in Down state or cannot receive BFD packets. In this case, proceed to step c.

-     If the value of the Sta field in the debugging information is 2 or 3, it indicates that the local end receives BFD packets in Init or Up state, but the BFD session cannot come up on the local end. In this case, proceed to step 4.

c.     Check the BFD packet receiving on the peer end as described in "BFD session establishment failure."

-     If the peer end cannot receive BFD packets, see "BFD session establishment failure" to troubleshoot the issue.

-     If the peer end can receive BFD packets, execute the display system internal bfd packet statistics command on the local end to view the data loss reasons in the The detailed discarded packet statistics area and resolve the packet loss issue according to the reasons. If the issue cannot be resolved or no packet loss occurs, proceed to step 4.

2.     Identify whether the BFD session state changes from Up to Down.

The following log message outputted by the CLI indicates that the BFD session state changes from UP to Down:

BFD/4/BFD_CHANGE_FSM: Sess[20.0.4.2/20.0.4.1,LD/RD:533/532, Interface:Vlan204, SessType:Ctrl, LinkType:INET], Ver.1, Sta: UP->DOWN, Diag: 1 (Control Detection Time Expired).

State change from Up to Down for the BFD is typically related to issues during the session negotiation phase. You can identify the reason based on the value of the Diag field in the BFD log message.

Table 3 Diagnostic information for different values of the Diag field

Value of the Diag field	Diagnostic information
1 (Control Detection Time Expired)	Local detection times out for a control-mode BFD session. The local end does not receive any packet from the peer end within the detection interval.
2 (Echo Function Failed)	Detection times out for an echo-mode BFD session. The local end does not receive any packet from the peer end within the detection interval.
3 (Neighbor Signaled Session Down)	The peer end notifies the local end of BFD session down.

 

If the value of the Diag field is 1, perform the following tasks:

a.     Disable BFD for the upper-layer protocol, and then select the appropriate tool to check link connectivity based on the type of the link.

-     If link flapping occurs, troubleshoot the link flapping issue.

-     If the link is operating correctly and the BFD session still flaps after you re-enable BFD for the upper-layer protocol, proceed to step 4.

b.     Check the BFD packet receiving on the local end.

Check the BFD packet receiving on the local end as described in "BFD session establishment failure."

-     If the local end can receive BFD packets but packet loss occurs, execute the display system internal bfd packet statistics command to view the data loss reasons in the The detailed discarded packet statistics area and resolve the packet loss issue according to the reasons. If the issue cannot be resolved or no packet loss occurs, proceed to step 4.

-     If the local end cannot receive BFD packets, proceed to step 4.

If the value of the Diag field is 2, perform the following tasks:

¡     If BFD is used to detect a single-hop IP link, use the ping command on the peer end to ping the source address of the echo-mode BFD session.

-     If the IP address cannot be pinged, it indicates a link failure. Troubleshoot the link failure.

-     If the IP address can be pinged, proceed to step 4.

¡     If BFD is used to detect an MPLS tunnel, the local end sends BFD echo packets through the MPLS tunnel and the peer end forwards the received BFD echo packets through an IP link. In this case, check the connectivity of the MPLS tunnel and the IP link.

-     If the MPLS tunnel or IP link fails, troubleshoot the MPLS tunnel failure or IP link failure.

-     If the MPLS tunnel and IP link operate correctly, proceed to step 4.

¡     If BFD is used to detect an SRv6 tunnel, the local end sends BFD echo packets through the SRv6 tunnel and the peer end forwards the received BFD echo packets through an IP link. In this case, check the connectivity of the SRv6 tunnel and the IP link.

-     If the SRv6 tunnel or IP link fails, troubleshoot the SRv6 tunnel failure or IP link failure.

-     If the SRv6 tunnel and IP link operate correctly, proceed to step 4.

¡     If uRPF is enabled for the device, it drops the echo packets forwarded back from the peer end. In this case, execute the display ip urpf command to identify whether an ACL is specified to permit the packets whose source IP address is the source IP address of the echo-mode BFD session. This configuration suppresses uRPF from dropping packets that match the ACL.

-     If no ACL is specified for packet drop suppression, use the ip urpf command to specify an ACL that permits the packets whose source IP address is the source IP address of the echo-mode BFD session.

-     If such an ACL is specified for packet drop suppression, proceed to step 4.

If the value of the Diag field is 3, perform the same tasks as those performed when the value of the Diag field is 1.

3.     Identify whether the BFD session state changes to Administratively Down.

The following log message outputted by the CLI indicates that the BFD session state changes to Administratively Down:

BFD/5/BFD_CHANGE_SESS: Sess[17.1.1.2/17.1.1.1, LD/RD:1537/1537, Interface:GE2/0/1, SessType:Ctrl, LinkType:INET], Ver:1, Sta: Deleted, Diag: 7 (Administratively Down)

The common cause of the issue is upper-layer protocol failure, which indirectly causes BFD flapping. First, disable BFD for the upper-layer protocol and identify whether the upper-layer protocol remains stable. If the upper-layer protocol flaps, see Layer 3—IP Routing Troubleshooting Guide, MPLS Troubleshooting Guide, and Segment Routing Troubleshooting Guide to troubleshoot the upper-layer protocol failure.

If the upper-layer protocol is stable but the BFD session cannot come up, proceed to step 4.

4.     If the issue persists, collect the following information and contact Technical Support:

¡     Results of each step.

¡     The configuration file, log messages, and alarm messages.

Related alarm and log messages

Alarm messages

Module name: HH3C-BFD-STD-MIB

·     hh3cBfdSessStateUp (1.3.6.1.4.1.25506.2.72.0.3)

·     hh3cBfdSessStateDown (1.3.6.1.4.1.25506.2.72.0.4)

Log messages

·     BFD/4/BFD_CHANGE_FSM

·     BFD/5/BFD_CHANGE_SESS