Troubleshooting VXLAN

Troubleshooting VXLAN issues

Unreachable centralized VXLAN IP gateway

Symptom

As shown in Figure 1, a VXLAN tunnel is established between the VTEP and the centralized VXLAN IP gateway, and a VSI interface on the centralized VXLAN IP gateway acts as a gateway interface. When a ping operation is executed on the server connected to the VTEP, the centralized VXLAN IP gateway is unreachable.

Figure 1 Network diagram

 

Common causes

The following are the common causes of this type of issue:

·     The status of the VXLAN tunnel is down.

·     The source or destination IP address of the VXLAN tunnel is incorrect.

·     The status of the VXLAN IP gateway interface is down.

·     No ARP entry for the ping operation exists on the device.

Troubleshooting flow

Figure 2 shows the troubleshooting flowchart.

Figure 2 Flowchart for troubleshooting an unreachable centralized VXLAN IP gateway

 

Solution

1.     View the VXLAN tunnel information of the VXLAN network to which the server belongs on the VTEP that is connected to the server.

a.     Execute the display l2vpn vsi verbose command to check the VXLAN ID of the VXLAN network to which the server belongs, and the name of the VXLAN tunnel associated with the VXLAN network (Tunnel Name field).

<Sysname> display l2vpn vsi verbose

VSI Name: vpna

  VSI Index               : 0

  VSI State               : Up

  MTU                     : 1500

  Bandwidth               : Unlimited

  Broadcast Restrain      : Unlimited

  Multicast Restrain      : Unlimited

  Unknown Unicast Restrain: Unlimited

  MAC Learning            : Enabled

  MAC Table Limit         : -

  MAC Learning rate       : -

  Drop Unknown            : -

  Flooding                : Enabled

  Statistics              : Disabled

  VXLAN ID                : 10

  Tunnels:

    Tunnel Name          Link ID    State  Type        Flood proxy

    Tunnel1              0x5000001  Up     Manual      Disabled

    Tunnel2              0x5000002  Up     Manual      Disabled

  ACs:

     AC                               Link ID    State    Type

     GE0/0/1 srv1000                  0          Up       Manual

b.     Execute the display interface tunnel command based on the name of the VXLAN tunnel and examine the current state, source IP address, and destination IP address of the VXLAN tunnel.

<Sysname> display interface tunnel 2

Tunnel2

Current state: UP

Line protocol state: UP

Description: Tunnel2 Interface

Bandwidth: 64 kbps

Maximum transmission unit: 1464

Internet protocol processing: Disabled

Last clearing of counters: Never

Tunnel source 2.2.2.2, destination 1.1.1.1

Tunnel protocol/transport UDP_VXLAN/IP

Last 300 seconds input rate: 0 bytes/sec, 0 bits/sec, 0 packets/sec

Last 300 seconds output rate: 0 bytes/sec, 0 bits/sec, 0 packets/sec

Input: 0 packets, 0 bytes, 0 drops

Output: 0 packets, 0 bytes, 0 drops

-     If the VXLAN tunnel is up, go to step 3.

-     If the VXLAN tunnel is down, go to step 2.

2.     Check on the VTEP to see if the source IP address of the VXLAN tunnel is a local IP address, and whether the destination IP address is reachable.

¡     Execute the display ip interface brief command to verify that the source IP address of the VXLAN tunnel is a local IP address. If not, use the source command to modify the source IP address of the VXLAN tunnel.

<Sysname> display ip interface brief

*down: administratively down

(s): spoofing  (l): loopback

Interface           Physical Protocol IP address      VPN instance Description

Loop1               up       up(s)    2.2.2.2         --           --

MGE0/0/0            up       up       192.168.1.61    --           --

MGE0/0/1            down     down     --              --           --

MTunnel0            down     down     --              aaa          --

Vlan1               *down    down     --              --           --

¡     Execute the display fib command to check if an entry for the destination IP address of the VXLAN tunnel is in the FIB table. If not, modify the routing configuration to ensure Layer 3 connectivity to the destination IP address of the VXLAN tunnel.

<Sysname> display fib

 

Destination count: 4 FIB entry count: 4

 

Flag:

  U:Useable   G:Gateway   H:Host   B:Blackhole   D:Dynamic   S:Static

  R:Relay     F:FRR

 

Destination/Mask   Nexthop         Flag     OutInterface/Token       Label

0.0.0.0/32         127.0.0.1       UH       InLoop0                  Null

2.2.2.2/32         127.0.0.1       UH       InLoop0                  Null

1.1.1.1/32         127.0.0.1       UH       InLoop0                  Null

127.0.0.0/32       127.0.0.1       UH       InLoop0                  Null

3.     Execute the display interface vsi-interface brief command on the VXLAN IP gateway to view information about the VXLAN IP gateway interface, including the gateway interface number (Interface field), gateway interface state (Link Protocol field), and the gateway address (Primary IP field).

<Sysname> display interface Vsi-interface brief

Brief information on interfaces in route mode:

Link: ADM - administratively down; Stby - standby

Protocol: (s) - spoofing

Interface            Link Protocol Primary IP      Description

Vsi1                 DOWN DOWN     192.168.1.1

¡     If the VXLAN IP gateway interface is down, check whether the shutdown command is configured for the VSI interface or whether the VSI bound to the VSI interface is up.

-     If the shutdown command is configured for the VSI interface, execute the undo shutdown command.

-     If the VSI bound to the VSI interface is down, execute the display l2vpn vsi command to check the AC status of VSI. If the AC status is down, verify that the AC configuration is correct and the AC-attached interface is up. If the AC configuration is incorrect or the AC-attached interface is down, modify the AC configuration or troubleshoot the interface issue.

¡     If the VXLAN IP gateway interface is up, execute the display arp command to check whether the ARP information for the gateway IP address has been learned.

<Sysname> display arp

  Type: S-Static   D-Dynamic   O-Openflow   R-Rule   M-Multiport  I-Invalid

IP address       MAC address    VLAN/VSI  Interface/Link ID        Aging Type

10.1.1.1         0001-0001-0001 0         Tunnel2                  17    D

10.1.1.11        0001-0001-0001 0         Tunnel2                  20    D

20.1.1.1         0002-0002-0002 1         Tunnel3                  17    D

20.1.1.12        0002-0002-0002 1         Tunnel3                  20    D

-     If yes, go to step 4.

-     If not, execute the display arp count command to check whether the number of learned entries has reached the maximum number of dynamic ARP entries for the device or interface. If yes, execute the  arp max-learning-num or arp max-learning-number command to increase the maximum number of dynamic ARP entries.

4.     If the issue persists, collect the following information and contact Technical Support:

¡     Results of each step.

¡     The configuration file, log messages, and alarm messages.

¡     Diagnostic information collected by using the display diagnostic-information command.

Related alarm and log messages

Alarm messages

N/A

Log messages

N/A

Disconnection between VSI interfaces on two VTEPs

Symptom

As shown in Figure 3, a VXLAN tunnel is manually set up between the VTEPs, and VSI interfaces are configured as gateway interfaces on the VTEPs. Two VSI interfaces cannot ping each other.

 

NOTE: This section introduces the troubleshooting methods for the ADWAN scenario.

 

Figure 3 ADWAN network diagram

 

Common causes

The following are the common causes of this type of issue:

·     A VSI interface has not been associated with a VSI.

·     A VSI interface is down.

·     The IP addresses of the VSI interfaces are not in the same subnet.

·     The VXLAN tunnel is down.

·     The source or destination IP address of the VXLAN tunnel is incorrect.

·     The VSI is down.

Troubleshooting flow

Figure 4 shows the troubleshooting flowchart.

Figure 4 Flowchart for troubleshooting disconnection between VSI interfaces on two VTEPs

 

Solution

1.     Execute the display ip interface brief command on the VTEPs to view a brief information about the interfaces and IP addresses. For the unreachable gateway IP address, identify the name and state of the owner VSI interface.

[Sysname] display ip interface brief

*down: administratively down

(s): spoofing  (l): loopback

Interface          Physical Protocol IP address/Mask    VPN instance Description

GE0/0/1            up       up       192.168.1.114/24   --           --

GE0/0/3            down     down     --                 --           --

RAGG1              down     down     --                 --           --

Vsi1               down     down     1.1.1.1/24         --           --

2.     Execute the display l2vpn vsi verbose command on the VTEPs to view the information of the gateway interface (Gateway Interface field) and VXLAN tunnel (Tunnel Name field) associated with VSI.

[Sysname] display l2vpn vsi verbose

VSI Name: aaa

  VSI Index               : 0

  VSI State               : Up

  MTU                     : 1500

  Bandwidth               : -

  Broadcast Restrain      : 5120 kbps

  Multicast Restrain      : 5120 kbps

  Unknown Unicast Restrain: 5120 kbps

  MAC Learning            : Enabled

  MAC Table Limit         : -

  MAC Learning rate       : Unlimited

  Drop Unknown            : Disabled

  PW Redundancy Mode      : Slave

  Flooding                : Enabled

  Statistics              : Disabled

  Gateway Interface       : VSI-interface 1

  VXLAN ID                : 100

  Tunnel Statistics       : Disabled

  Tunnels:

    Tunnel Name          Link ID    State    Type        Flood Proxy  Split horizon

    Tunnel1              0x5000001  UP       Manual      Disabled     Enabled

3.     Check the output from the display l2vpn vsi verbose command for the VSI associated with VSI-interface 1.

¡     If the VSI does not exist, use the gateway vsi-interface command to configure the VSI interface as the VSI's gateway interface.

¡     If the VSI exists, perform the following tasks for the VSI interface:

-     Check if the shutdown command has been executed on the VSI interface. If yes, use the undo shutdown command to bring up the VSI interface.

-     Verify that the IP addresses of the VSI interfaces on the two VTEPs are in the same subnet. If not, assign IP addresses from the same subnet to the VSI interfaces.

4.     Check the output from the display l2vpn vsi verbose command for VXLAN tunnels of the VSI.

¡     If no VXLAN tunnel is associated, create a VXLAN tunnel and use the tunnel command to associate it with the VSI.

¡     If a VXLAN tunnel is associated, follow step 2 to check the state, source IP address, and destination IP address of the VXLAN tunnel by using the display interface tunnel command.

<Sysname> display interface tunnel 2

Tunnel2

Current state: UP

Line protocol state: UP

Description: Tunnel2 Interface

Bandwidth: 64 kbps

Maximum transmission unit: 1464

Internet protocol processing: Disabled

Last clearing of counters: Never

Tunnel source 2.2.2.2, destination 1.1.1.1

Tunnel protocol/transport UDP_VXLAN/IP

Last 300 seconds input rate: 0 bytes/sec, 0 bits/sec, 0 packets/sec

Last 300 seconds output rate: 0 bytes/sec, 0 bits/sec, 0 packets/sec

Input: 0 packets, 0 bytes, 0 drops

Output: 0 packets, 0 bytes, 0 drops

5.     Check on the VTEPs whether the source IP address of the VXLAN tunnel is a local IP address, and whether the destination IP address is an address on the remote VTEP. Verify that the destination IP address is reachable.

¡     Execute the display ip interface brief command to check if the source IP address of the VXLAN tunnel is a local IP address. If not, modify the source IP address of the VXLAN tunnel by using the source command.

<Sysname> display ip interface brief

*down: administratively down

(s): spoofing  (l): loopback

Interface           Physical Protocol IP address      VPN instance Description

Loop1               up       up(s)    2.2.2.2         --           --

MGE0/0/0            up       up       192.168.1.61    --           --

MGE0/0/1            down     down     --              --           --

MTunnel0            down     down     --              aaa          --

Vlan1               *down    down     --              --           --

¡     Execute the display fib command to check if an entry for the destination IP address of the VXLAN tunnel is in the FIB table, and use the ping command to verify connectivity between the source and destination IP addresses of the VXLAN tunnel. If no FIB entry is found, modify the routing configuration to ensure Layer 3 connectivity to the destination IP address of the VXLAN tunnel.

<Sysname> display fib

 

Destination count: 4 FIB entry count: 4

 

Flag:

  U:Useable   G:Gateway   H:Host   B:Blackhole   D:Dynamic   S:Static

  R:Relay     F:FRR

 

Destination/Mask   Nexthop         Flag     OutInterface/Token       Label

0.0.0.0/32         127.0.0.1       UH       InLoop0                  Null

2.2.2.2/32         127.0.0.1       UH       InLoop0                  Null

1.1.1.1/32         127.0.0.1       UH       InLoop0                  Null

127.0.0.0/32       127.0.0.1       UH       InLoop0                  Null

6.     Execute the display l2vpn vsi verbose command on the VTEPs to check if the VSI is up.

¡     If the VSI is down, check whether the shutdown command has been configured on the VSI. If yes, execute the undo shutdown command.

¡     If the VSI is up, go to step 7.

7.     Perform steps 1 through 6.

8.     If the issue persists, collect the following information and contact Technical Support:

¡     Results of each step.

¡     The configuration file, log messages, and alarm messages.

¡     Diagnostic information collected by using the display diagnostic-information command.

Related alarm and log messages

Alarm messages

Module Name: HH3C-IF-EXT-MIB

·     hh3cIfPortUp (1.3.6.1.4.1.25506.2.40.3.0.5)

Log messages

·     IFNET/3/PHY_UPDOWN

·     IFNET/5/LINK_UPDOWN