Virtual eXtensible LAN (VXLAN) is a MAC-in-UDP technology that provides Layer 2 connectivity between distant network sites across an IP network. VXLAN is typically used in data centers for multitenant services.

VXLAN benefits

VXLAN provides the following benefits:

· Support for more virtual switched domains than VLANs—Each VXLAN is uniquely identified by a 24-bit VXLAN ID. The total number of VXLANs can reach 16777216 (224). This specification makes VXLAN a better choice than 802.1Q VLAN to isolate traffic for user terminals.

· Easy deployment and maintenance—VXLAN requires deployment only on the edge devices of the transport network. Devices in the transport network perform typical Layer 3 forwarding.

VXLAN network model

As shown in Figure 1, a VXLAN is a virtual Layer 2 network (known as the overlay network) built on top of an existing physical Layer 3 network (known as the underlay network). The overlay network encapsulates inter-site Layer 2 frames into VXLAN packets and forwards the packets to the destination along the Layer 3 forwarding paths provided by the underlay network. The underlay network is transparent to tenants, and geographically dispersed sites of a tenant are merged into a Layer 2 network.

The transport edge devices assign user terminals to different VXLANs, and then forward traffic between sites for user terminals by using VXLAN tunnels. Supported user terminals include PCs, wireless terminals, and VMs on servers.

NOTE:

This document uses VMs as examples to describe the mechanisms of VXLAN. The mechanisms do not differ between different kinds of user terminals.

The transport edge devices are VXLAN tunnel endpoints (VTEP). The VTEP implementation of the device uses ACs, VSIs, and VXLAN tunnels to provide VXLAN services.

· VSI—A virtual switch instance is a virtual Layer 2 switched domain. Each VSI provides switching services only for one VXLAN. VSIs learn MAC addresses and forward frames independently of one another. VMs in different sites have Layer 2 connectivity if they are in the same VXLAN.

· Attachment circuit (AC)—An AC is a physical or virtual link that connects a VTEP to a local site. Typically, ACs are site-facing Layer 3 interfaces that are associated with the VSI of a VXLAN. Traffic received from an AC is assigned to the VSI associated with the AC.

· VXLAN tunnel—Logical point-to-point tunnels between VTEPs over the transport network. Each VXLAN tunnel can trunk multiple VXLANs.

VTEPs encapsulate VXLAN traffic in the VXLAN, outer UDP, and outer IP headers. The devices in the transport network forward VXLAN traffic only based on the outer IP header.

Figure 1 VXLAN network model

VXLAN packet format

As shown in Figure 2, a VTEP encapsulates a frame in the following headers:

· 8-byte VXLAN header—VXLAN information for the frame.

¡ Flags—If the I bit is 1, the VXLAN ID is valid. If the I bit is 0, the VXLAN ID is invalid. All other bits are reserved and set to 0.

¡ 24-bit VXLAN ID—Identifies the VXLAN of the frame. It is also called the virtual network identifier (VNI).

· 8-byte outer UDP header for VXLAN—The default VXLAN destination UDP port number is 4789.

· 20-byte outer IPv4 or 40-byte outer IPv6 header—Valid addresses of VTEPs or VXLAN multicast groups on the transport network. Devices in the transport network forward VXLAN packets based on the outer IP header.

Figure 2 VXLAN packet format

VXLAN working mechanisms

Generic VXLAN network establishment and forwarding process

The VTEP uses the following process to establish the VXLAN network and forward an inter-site frame:

1. Discovers remote VTEPs, establishes VXLAN tunnels, and assigns the VXLAN tunnels to VXLANs.

2. Assigns the frame to its matching VXLAN if the frame is sent between sites.

3. Performs MAC learning on the VXLAN's VSI.

4. Forwards the frame through VXLAN tunnels.

This section describes this process in detail. For intra-site frames in a VSI, the system performs typical Layer 2 forwarding, and it processes 802.1Q VLAN tags as described in "Access modes of VSIs."

VXLAN tunnel establishment and assignment

To provide Layer 2 connectivity for a VXLAN between two sites, you must create a VXLAN tunnel between the sites and assign the tunnel to the VXLAN.

VXLAN tunnel establishment

VXLAN supports manual VXLAN tunnel establishment. You must manually create a VXLAN tunnel interface, and specify the tunnel source and destination IP addresses on the peer VTEPs.

VXLAN tunnel assignment

VXLAN supports manual VXLAN tunnel assignment. You must manually assign VXLAN tunnels to VXLANs.

Assignment of traffic to VXLANs

Traffic from the local site to a remote site

The VTEP uses Layer 3 interface-to-VSI mapping to assign customer frames to a VXLAN. This method maps a site-facing Layer 3 interface to a VSI. The VTEP assigns all frames received from the interface to the VXLAN of the VSI.

Traffic from a remote site to the local site

When a frame arrives at a VXLAN tunnel, the VTEP uses the VXLAN ID in the frame to identify its VXLAN.

MAC learning

The VTEP performs source MAC learning on the VSI as a Layer 2 switch.

· For traffic from the local site to the remote site, the VTEP learns the source MAC address before VXLAN encapsulation.

· For traffic from the remote site to the local site, the VTEP learns the source MAC address after removing the VXLAN header.

A VSI's MAC address table includes the following types of MAC address entries:

· Local MAC—MAC entries learned from the local site. The outgoing interfaces for the MAC address entries are site-facing interfaces.

¡ Static—Manually added MAC entries.

¡ Dynamic—Dynamically learned MAC entries.

· Remote MAC—MAC entries learned from a remote site. The outgoing interfaces for the MAC addresses are VXLAN tunnel interfaces.

¡ Static—Manually added MAC entries.

¡ Dynamic—MAC entries learned in the data plane from incoming traffic on VXLAN tunnels. The learned MAC addresses are contained in the inner Ethernet header.

The following shows the priority order of different types of remote MAC address entries:

a. Static MAC address entries.

b. Dynamic MAC address entries.

Unicast forwarding

Intra-site unicast forwarding

The VTEP uses the following process to forward a known unicast frame within a site:

1. Identifies the VSI of the frame.

2. Looks up the destination MAC address in the VSI's MAC address table for the outgoing interface.

3. Sends the frame out of the matching outgoing interface.

As shown in Figure 3, VTEP 1 forwards a frame from VM 1 to VM 4 within the local site in VLAN 10 as follows:

1. Identifies that the frame belongs to VSI A when the frame arrives at Interface A.

2. Looks up the destination MAC address (MAC 4) in the MAC address table of VSI A for the outgoing interface.

3. Sends the frame out of the matching outgoing interface (Interface B) to VM 4 in VLAN 10.

Figure 3 Intra-site unicast

Inter-site unicast forwarding

The following process (see Figure 4) applies to a known unicast frame between sites:

1. The source VTEP encapsulates the Ethernet frame in the VXLAN/UDP/IP header.

In the outer IP header, the source IP address is the source VTEP's VXLAN tunnel source IP address. The destination IP address is the VXLAN tunnel destination IP address.

2. The source VTEP forwards the encapsulated packet out of the outgoing VXLAN tunnel interface found in the VSI's MAC address table.

3. The intermediate transport devices (P devices) forward the frame to the destination VTEP by using the outer IP header.

4. The destination VTEP removes the headers on top of the inner Ethernet frame. It then performs MAC address table lookup in the VXLAN's VSI to forward the frame out of the matching outgoing interface.

Figure 4 Inter-site unicast

‌

Flood

The source VTEP floods a broadcast, multicast, or unknown unicast frame to all site-facing interfaces and VXLAN tunnels in the VXLAN, except for the incoming interface. Each destination VTEP floods the inner Ethernet frame to all site-facing interfaces in the VXLAN. To avoid loops, the destination VTEPs do not flood the frame back to VXLAN tunnels.

Unicast mode (head-end replication)

As shown in Figure 5, the source VTEP replicates the flood frame, and then sends one replica to the destination IP address of each VXLAN tunnel in the VXLAN.

Figure 5 Unicast mode

Access modes of VSIs

The access mode of a VSI determines how the VTEP processes the 802.1Q VLAN tags in the Ethernet frames.

VLAN access mode

In this mode, Ethernet frames received from or sent to the local site must contain 802.1Q VLAN tags.

· For an Ethernet frame received from the local site, the VTEP removes all its 802.1Q VLAN tags before forwarding the frame.

· For an Ethernet frame destined for the local site, the VTEP adds 802.1Q VLAN tags to the frame before forwarding the frame.

In VLAN access mode, VXLAN packets sent between sites do not contain 802.1Q VLAN tags. You can use different 802.1Q VLANs to provide the same service in different sites.

Ethernet access mode

The VTEP does not process the 802.1Q VLAN tags of Ethernet frames received from or sent to the local site.

· For an Ethernet frame received from the local site, the VTEP forwards the frame with the 802.1Q VLAN tags intact.

· For an Ethernet frame destined for the local site, the VTEP forwards the frame without adding 802.1Q VLAN tags.

In Ethernet access mode, VXLAN packets sent between VXLAN sites contain 802.1Q VLAN tags. You must use the same VLAN to provide the same service between sites.

ARP flood suppression

ARP flood suppression reduces ARP request broadcasts by enabling the VTEP to reply to ARP requests on behalf of VMs.

As shown in Figure 6, this feature snoops ARP packets to populate the ARP flood suppression table with local and remote MAC addresses. If an ARP request has a matching entry, the VTEP replies to the request on behalf of the VM. If no match is found, the VTEP floods the request to both local and remote sites.

Figure 6 ARP flood suppression

ARP flood suppression uses the following workflow:

1. VM 1 sends an ARP request to obtain the MAC address of VM 7.

2. VTEP 1 creates a suppression entry for VM 1, and floods the ARP request in the VXLAN.

3. VTEP 2 and VTEP 3 de-encapsulate the ARP request. The VTEPs create a suppression entry for VM 1, and broadcast the request in the local site.

4. VM 7 sends an ARP reply.

5. VTEP 2 creates a suppression entry for VM 7 and forwards the ARP reply to VTEP 1.

6. VTEP 1 de-encapsulates the ARP reply, creates a suppression entry for VM 7, and forwards the ARP reply to VM 1.

7. VM 4 sends an ARP request to obtain the MAC address of VM 1 or VM 7.

8. VTEP 1 creates a suppression entry for VM 4 and replies to the ARP request.

9. VM 10 sends an ARP request to obtain the MAC address of VM 1.

10. VTEP 3 creates a suppression entry for VM 10 and replies to the ARP request.

VXLAN IP gateways

A VXLAN IP gateway provides Layer 3 forwarding services for VMs in VXLANs. A VXLAN IP gateway can be an independent device or be collocated with a VTEP. For more information about VXLAN IP gateway placement, see "Configuring VXLAN IP gateways."

Protocols and standards

RFC 7348, Virtual eXtensible Local Area Network (VXLAN): A Framework for Overlaying Virtualized Layer 2 Networks over Layer 3 Networks

Configuring basic VXLAN features

VXLAN tasks at a glance

To configure basic VXLAN settings, perform the following tasks on VTEPs:

1. Creating a VXLAN on a VSI

2. Configuring a VXLAN tunnel

3. Manually assigning VXLAN tunnels to a VXLAN

4. Assigning customer frames to a VSI

5. (Optional.) Managing MAC address entries

¡ Configuring static MAC address entries

¡ Disabling remote-MAC address learning

¡ Enabling local-MAC logging

6. (Optional.) Configuring VXLAN packet parameters

¡ Setting the destination UDP port number of VXLAN packets

¡ Configuring VXLAN packet check

7. (Optional.) Reducing flood traffic in the transport network

¡ Confining unknown-unicast floods to the local site

¡ Enabling ARP flood suppression

8. (Optional.) Enabling VXLAN packet statistics

9. (Optional.) Enabling VXLAN fast forwarding

Prerequisites for VXLAN

Configure a routing protocol on the devices in the transport network to make sure the VTEPs can reach one another.

Creating a VXLAN on a VSI

1. Enter system view.

system-view

2. Enable L2VPN.

l2vpn enable

By default, L2VPN is disabled.

3. Create a VSI and enter VSI view.

vsi vsi-name

4. Enable the VSI.

undo shutdown

By default, a VSI is enabled.

5. Create a VXLAN and enter VXLAN view.

vxlan vxlan-id

You can create only one VXLAN on a VSI.

The VXLAN ID must be unique for each VSI.

6. (Optional.) Configure VSI parameters:

a. Return to VSI view.

quit

b. Configure a VSI description.

description text

By default, a VSI does not have a description.

c. Set the MTU for the VSI.

mtu mtu

The default MTU for a VSI is 1500 bytes.

The MTU set by using this command limits the maximum length of the packets that a VSI receives from ACs and forwards through VXLAN tunnels. The MTU does not limit the maximum length of other packets in the VXLAN VSI.

Configuring a VXLAN tunnel

Manually creating a VXLAN tunnel

About this task

When you manually create a VXLAN tunnel, specify addresses on the local VTEP and the remote VTEP as the tunnel source and destination addresses, respectively.

Restrictions and guidelines

As a best practice, do not configure multiple VXLAN tunnels to use the same source and destination IP addresses.

This task provides basic VXLAN tunnel configuration. For more information about tunnel configuration and commands, see VPN Configuration Guide and VPN Command Reference.

Procedure

1. Enter system view.

system-view

2. (Optional.) Specify a global source IP address for VXLAN tunnels.

tunnel global source-address ipv4-address

By default, no global source IP address is specified for VXLAN tunnels.

A VXLAN tunnel uses the global source address if you do not specify a source interface or source address for the tunnel.

3. Create a VXLAN tunnel interface and enter tunnel interface view.

interface tunnel tunnel-number mode vxlan

The endpoints of a tunnel must use the same tunnel mode.

4. Specify a source address for the tunnel. Choose one of the following methods:

¡ Specify a source IP address for the tunnel.

source { ipv4-address | ipv6-address }

The specified IP address is used in the outer IP header of tunneled VXLAN packets.

¡ Specify a source interface for the tunnel.

source interface-type interface-number

The primary IP address of the specified interface is used in the outer IP header of tunneled VXLAN packets.

By default, no source IP address or source interface is specified for a tunnel.

5. Specify a destination IP address for the tunnel.

destination { ipv4-address | ipv6-address }

By default, no destination IP address is specified for a tunnel.

Specify the remote VTEP's IP address. This IP address will be the destination IP address in the outer IP header of tunneled VXLAN packets.

Manually assigning VXLAN tunnels to a VXLAN

About this task

To provide Layer 2 connectivity for a VXLAN between two sites, you must assign the VXLAN tunnel between the sites to the VXLAN.

You can assign multiple VXLAN tunnels to a VXLAN, and configure a VXLAN tunnel to trunk multiple VXLANs. For a unicast-mode VXLAN, the system floods unknown unicast, multicast, and broadcast traffic to each tunnel associated with the VXLAN.

Restrictions and guidelines

For full Layer 2 connectivity in the VXLAN, make sure the VXLAN contains the VXLAN tunnel between each pair of sites in the VXLAN.

Procedure

1. Enter system view.

system-view

2. Enter VSI view.

vsi vsi-name

3. Enter VXLAN view.

vxlan vxlan-id

4. Assign VXLAN tunnels to the VXLAN.

tunnel tunnel-number

By default, a VXLAN does not contain any VXLAN tunnels.

Assigning customer frames to a VSI

Mapping a Layer 3 interface to a VSI

About this task

To assign the customer traffic on a Layer 3 interface to a VXLAN, map the interface to the VXLAN's VSI. The VSI uses its MAC address table to forward the customer traffic.

Procedure

1. Enter system view.

system-view

2. Enter Layer 3 interface view.

interface interface-type interface-number

3. Configure the VLAN tag processing rule for incoming traffic.

l2vpn rewrite inbound tag { nest { c-vid vlan-id | s-vid vlan-id [ c-vid vlan-id ] } | remark 1-to-2 s-vid vlan-id c-vid vlan-id } [ symmetric ]

By default, VLAN tags of incoming traffic are not processed.

Support for this feature varies by device model. For more information, see the command reference.

4. Map the Layer 3 interface to a VSI.

xconnect vsi vsi-name [ track track-entry-number&<1-3> ]

By default, a Layer 3 interface is not mapped to any VSI.

If the AC is a Layer 3 subinterface, you can specify the access mode. The default access mode is VLAN. If the AC is a Layer 3 interface, you cannot specify the access mode.

Managing MAC address entries

About MAC address entry management

Local-MAC address entries can be manually added or dynamically learned. You can log local MAC addresses and local-MAC changes.

Remote-MAC address entries have a variety of types, including manually added entries and dynamically learned entries.

Configuring static MAC address entries

1. Enter system view.

system-view

2. Add a static local-MAC address entry.

mac-address static mac-address { interface interface-type interface-number | interface interface-type interface-number service-instance instance-id } vsi vsi-name

For successful configuration, make sure the specified Layer 3 interface or Ethernet service instance has been mapped to the VSI.

3. Add a static remote-MAC address entry.

mac-address static mac-address interface tunnel tunnel-number vsi vsi-name

For the setting to take effect, make sure the VSI's VXLAN has been specified on the VXLAN tunnel.

Disabling remote-MAC address learning

About this task

When network attacks occur, disable remote-MAC address learning to prevent the device from learning incorrect remote MAC addresses. You can manually add static remote-MAC address entries.

Procedure

1. Enter system view.

system-view

2. Disable remote-MAC address learning.

vxlan tunnel mac-learning disable

By default, remote-MAC address learning is enabled.

Enabling local-MAC logging

About this task

When the local-MAC logging feature is enabled, the VXLAN module immediately sends a log message with its local MAC addresses to the information center. When a local MAC address is added or removed, a log message is also sent to the information center to notify the local-MAC change.

With the information center, you can set log message filtering and output rules, including output destinations. For more information about configuring the information center, see Network Management and Monitoring Configuration Guide.

Procedure

1. Enter system view.

system-view

2. Enable local-MAC logging.

vxlan local-mac report

By default, local-MAC logging is disabled.

Setting the destination UDP port number of VXLAN packets

1. Enter system view.

system-view

2. Set a destination UDP port for VXLAN packets.

vxlan udp-port port-number

By default, the destination UDP port number is 4789 for VXLAN packets.

You must configure the same destination UDP port number on all VTEPs in a VXLAN.

Setting the source UDP port number of VXLAN packets

About this task

Perform this task to enable a VXLAN tunnel interface to encapsulate different source UDP port numbers for traffic flows. This allows IPsec to identify the VXLAN packets to encrypt by the source UDP port number in the VXLAN encapsulation.

Restrictions and guidelines

The commands used for setting the source UDP port number of VXLAN packets take effect only on IPv4-based VXLAN. Only manually created VXLAN tunnel interfaces support these commands.

Procedure

1. Enter system view.

system-view

2. Enter VXLAN tunnel interface view.

interface tunnel tunnel-number mode vxlan [ ipv6 ]

3. Generate the source UDP port number in the VXLAN encapsulation based on the IP five-tuple of the inner Ethernet frame.

vxlan source udp-port five-tuple

By default, the source UDP port number in the VXLAN encapsulation is generated based on the source and destination MAC addresses of the inner Ethernet frame.

Configuring VXLAN packet check

About this task

The device always sets the UDP checksum of VXLAN packets to zero. For compatibility with third-party devices, a VXLAN packet can pass the check if its UDP checksum is zero or correct. If its UDP checksum is incorrect, the VXLAN packet fails the check and is dropped.

Procedure

1. Enter system view.

system-view

2. Enable the VTEP to drop VXLAN packets that fail UDP checksum check.

vxlan invalid-udp-checksum discard

By default, the VTEP does not check the UDP checksum of VXLAN packets.

Confining unknown-unicast floods to the local site

About this task

By default, the VTEP floods unknown unicast frames received from the local site to the following interfaces in the frame's VXLAN:

· All site-facing interfaces except for the incoming interface.

· All VXLAN tunnel interfaces.

To exclude a remote MAC address from the flood suppression done by using this feature, enable selective flood for the MAC address. The VTEP will flood the frames destined for the MAC address to remote sites.

Procedure

1. Enter system view.

system-view

2. Enter VSI view.

vsi vsi-name

3. Disable the VSI to flood unknown unicast traffic to VXLAN tunnel interfaces.

flooding disable

By default, unknown unicast traffic is flooded to all interfaces in the VXLAN, except for the incoming interface.

4. (Optional.) Enable selective flood for a MAC address.

selective-flooding mac-address mac-address

Enabling ARP flood suppression

Hardware and feature compatibility

F1000 series	Models	Feature compatibility
F1000-X-G5 series	F1000-A-G5, F1000-C-G5, F1000-C-G5-LI, F1000-E-G5, F1000-H-G5, F1000-S-G5	Yes
F1000-X-XI series	F1000-E-XI	Yes

F100 series	Models	Feature compatibility
F100-X-G5 series	F100-A-G5, F100-C-G5, F100-E-G5, F100-M-G5, F100-S-G5	Yes
F100-C-A series	F100-C-A2, F100-C-A1	No
F100-X-XI series	F100-C-XI, F100-S-XI	Yes
F100-X-XI series	F100-A-XI	No

‌

Restrictions and guidelines

The aging timer is fixed at 25 minutes for ARP flood suppression entries. If the suppression table is full, the VTEP stops learning new entries. For the VTEP to learn new entries, you must wait for old entries to age out, or use the reset arp suppression vsi command to clear the table.

If the flooding disable command is configured, set the MAC aging timer to a higher value than the aging timer for ARP flood suppression entries on all VTEPs. This setting prevents the traffic blackhole that occurs when a MAC address entry ages out before its ARP flood suppression entry ages out. To set the MAC aging timer, use the mac-address timer command.

When remote ARP learning is disabled for VXLANs, the device does not use ARP flood suppression entries to respond to ARP requests received on VXLAN tunnels.

Procedure

1. Enter system view.

system-view

2. Enter VSI view.

vsi vsi-name

3. Enable ARP flood suppression.

arp suppression enable

By default, ARP flood suppression is disabled.

Enabling VXLAN packet statistics

Hardware and feature compatibility

F1000 series	Models	Feature compatibility
F1000-X-G5 series	F1000-A-G5, F1000-C-G5, F1000-C-G5-LI, F1000-E-G5, F1000-H-G5, F1000-S-G5	Yes
F1000-X-XI series	F1000-E-XI	Yes

F100 series	Models	Feature compatibility
F100-X-G5 series	F100-A-G5, F100-E-G5	Yes
F100-X-G5 series	F100-C-G5, F100-M-G5, F100-S-G5	No
F100-C-A series	F100-C-A2, F100-C-A1	No
F100-X-XI series	F100-A-XI, F100-C-XI, F100-S-XI	No

‌

Enabling packet statistics for an AC

Enable packet statistics for all VXLAN tunnels of a VSI

About this task

VXLAN tunnels can be manually created. You can enable packet statistics for all VXLAN tunnels of a VSI.

If you enable packet statistics in VSI view, follow these guidelines:

· To display the packet statistics for VXLAN tunnels, use the display vxlan tunnel command in any view.

· To clear the packet statistics for VXLAN tunnels, use the reset l2vpn statistics tunnel command in user view.

Hardware and feature compatibility

F1000 series	Models	Feature compatibility
F1000-X-G5 series	F1000-A-G5, F1000-C-G5, F1000-C-G5-LI, F1000-E-G5, F1000-H-G5, F1000-S-G5	Yes
F1000-X-XI series	F1000-E-XI	Yes

F100 series	Models	Feature compatibility
F100-X-G5 series	F100-A-G5, F100-E-G5	Yes
F100-X-G5 series	F100-C-G5, F100-M-G5, F100-S-G5	No
F100-C-A series	F100-C-A2, F100-C-A1	No
F100-X-XI series	F100-A-XI, F100-C-XI, F100-S-XI	No

‌

Procedure

1. Enter system view.

system-view

2. Enter VSI view.

vsi vsi-name

3. Enable packet statistics for all VXLAN tunnels associated with the VSI.

tunnel statistics enable

By default, the packet statistics feature is disabled for the VXLAN tunnels associated with a VSI.

Setting the VXLAN statistics collection interval

1. Enter system view.

system-view

2. Set the VXLAN statistics collection interval.

l2vpn statistics interval interval

By default, the VXLAN statistics collection interval is 15 minutes.

Enabling VXLAN fast forwarding

About this task

VXLAN fast forwarding enables the device to bypass QoS and security services when the device forwards data traffic over VXLAN tunnels based on the software. As a best practice, enable this feature to improve forwarding speed only when QoS and security services are not configured on the following interfaces:

· VSI interfaces.

· Traffic outgoing interfaces for VXLAN tunnels.

Restrictions and guidelines

When VXLAN fast forwarding is enabled, a VXLAN tunnel cannot use ECMP routes to load share traffic. Instead, it selects one route from the ECMP routes to forward VXLAN packets.

Procedure

1. Enter system view.

system-view

2. Enable VXLAN fast forwarding.

vxlan fast-forwarding enable

By default, VXLAN fast forwarding is disabled.

Display and maintenance commands for VXLANs

Execute display commands in any view and reset commands in user view.

Task	Command
Display ARP flood suppression entries on VSIs.	display arp suppression vsi [ name vsi-name ] [ slot slot-number ] [ count ]
Display information about tunnel interfaces.	display interface [ tunnel [ number ] ] [ brief [ description \| down ] ]
Display L2VPN information for Layer 3 interfaces that are mapped to VSIs.	display l2vpn interface [ vsi vsi-name \| interface-type interface-number ] [ verbose ]
Display MAC address entries for VSIs.	display l2vpn mac-address [ vsi vsi-name ] [ dynamic ] [ count ]
Display information about VSIs.	display l2vpn vsi [ name vsi-name ] [ verbose ]
Display VXLAN tunnel information for VXLANs.	display vxlan tunnel [ vxlan vxlan-id [ tunnel tunnel-number ] ]
Clear ARP flood suppression entries on VSIs.	reset arp suppression vsi [ name vsi-name ]
Clear dynamic MAC address entries on VSIs.	reset l2vpn mac-address [ vsi vsi-name ]

NOTE:

For more information about the display interface tunnel command, see tunneling commands in Layer 3—IP Services Command Reference.

Configuring VXLAN IP gateways

About VXLAN IP gateways

The following are available IP gateway placement designs for VXLANs:

· VXLAN IP gateways separated from VTEPs—Use a VXLAN-unaware device as a gateway to the external network for VXLANs. On the gateway, you do not need to configure VXLAN settings.

· VXLAN IP gateways collocated with VTEPs—Include the following placement designs:

¡ Centralized VXLAN IP gateway deployment—Use one VTEP to provide Layer 3 forwarding for VXLANs. Typically, the gateway-collocated VTEP connects to other VTEPs and the external network. To use this design, make sure the IP gateway has sufficient bandwidth and processing capability. Centralized VXLAN IP gateways provide services only for IPv4 networks.

¡ Centralized VXLAN gateway group deployment—Use one VTEP group that contains redundant centralized VXLAN IP gateways to provide reliable gateway services for VXLANs.

¡ Distributed VXLAN IP gateway deployment—Deploy one VXLAN IP gateway on each VTEP to provide Layer 3 forwarding for VXLANs at their respective sites. This design distributes the Layer 3 traffic load across VTEPs. However, its configuration is more complex than the centralized VXLAN IP gateway design. Distributed gateways can provide services for both IPv4 and IPv6 networks.

In a collocation design, the VTEPs use virtual Layer 3 VSI interfaces as gateway interfaces to provide services for VXLANs.

‌

NOTE:

The following information describes traffic forwarding of VXLAN IP gateways in IPv4 networks. Traffic forwarding of VXLAN IP gateways in IPv6 networks is similar to that in IPv4 networks.

‌

VXLAN IP gateways separated from VTEPs

As shown in Figure 7, an independent VXLAN IP gateway connects a Layer 3 network to a VTEP. VMs send Layer 3 traffic in Layer 2 frames to the gateway through VXLAN tunnels. When the tunneled VXLAN packets arrive, the VTEP terminates the VXLANs and forwards the inner frames to the gateway. In this gateway placement design, the VTEP does not perform Layer 3 forwarding for VXLANs.

Figure 7 VXLAN IP gateway separated from VTEPs

Centralized VXLAN IP gateway deployment

As shown in Figure 8, a VTEP acts as a gateway for VMs in the VXLANs. The VTEP both terminates the VXLANs and performs Layer 3 forwarding for the VMs.

Figure 8 Centralized VXLAN IP gateway placement design

As shown in Figure 9, the network uses the following process to forward Layer 3 traffic from VM 10.1.1.11 to the Layer 3 network:

1. The VM sends an ARP request to obtain the MAC address of the gateway (VTEP 3) at 10.1.1.1.

2. VTEP 1 floods the ARP request to all remote VTEPs.

3. VTEP 3 de-encapsulates the ARP request, creates an ARP entry for the VM, and sends an ARP reply to the VM.

4. VTEP 1 forwards the ARP reply to the VM.

5. The VM learns the MAC address of the gateway, and sends the Layer 3 traffic to the gateway.

6. VTEP 3 removes the VXLAN encapsulation and inner Ethernet header for the traffic, and forwards the traffic to the destination node.

Inter-VXLAN forwarding is the same as this process except for the last step. At the last step of inter-VLAN forwarding, the gateway replaces the source-VXLAN encapsulation with the destination-VXLAN encapsulation, and then forwards the traffic.

Figure 9 Example of centralized VXLAN IP gateway deployment

Centralized VXLAN gateway group deployment

As shown in Figure 10, a VTEP group uses redundant centralized VXLAN IP gateways to provide reliable gateway services for VMs in the VXLANs. All member VTEPs in the group participate in Layer 3 forwarding and load share traffic between the Layer 3 network and the VXLANs. This design distributes processing among multiple VTEPs and prevents single points of failure.

Figure 10 Example of centralized VXLAN IP gateway group deployment

The VTEP group is a virtual gateway that provides services at a group IP address. Access layer VTEPs set up VXLAN tunnels to the group IP address for data traffic forwarding. Each VTEP in the group automatically uses its member IP address to set up tunnels to the other member VTEPs and access layer VTEPs. The tunnels are used to transmit protocol packets and synchronize ARP entries.

Distributed VXLAN IP gateway deployment

About distributed VXLAN IP gateway deployment

As shown in Figure 11, each site's VTEP acts as a gateway to perform Layer 3 forwarding for the VXLANs of the local site. A VTEP acts as a border gateway to the Layer 3 network for the VXLANs.

Figure 11 Distributed VXLAN IP gateway placement design

Figure 12 shows an example of distributed VXLAN IP gateway deployment. Create VSI interfaces on each distributed VXLAN IP gateway and the border gateway as gateway interfaces. Assign the same IP address to the same VSI interface on the distributed VXLAN IP gateways. You must enable local proxy ARP or local ND proxy on a distributed VXLAN IP gateway. The gateway performs Layer 3 forwarding based on ARP or ND entries. The following sections use distributed VXLAN IP gateways enabled with the local proxy ARP or local ND proxy feature to describe the forwarding processes for intra-VXLAN traffic, inter-VXLAN traffic, and traffic from a VXLAN to an external network.

A distributed VXLAN IP gateway can generate ARP or ND entries by a variety of methods. The following sections use dynamically learned ARP or ND entries to describe the forwarding processes.

Figure 12 Example of distributed VXLAN IP gateway deployment

Intra-VXLAN traffic forwarding between sites

As shown in Figure 12, the network uses the following process to forward traffic in a VXLAN between sites (for example, from VM 1 to VM 4 in VXLAN 10):

1. VM 1 sends an ARP request to obtain the MAC address of VM 4.

2. GW 1 performs the following operations:

a. Creates an ARP entry for VM 1 and replies with the MAC address of VSI-interface 10 (the gateway interface for VXLAN 10).

b. Replaces the sender MAC address of the ARP request with the MAC address of VSI-interface 10, and then floods the request to all sites in VXLAN 10.

3. VM 1 creates an ARP entry for VM 4. The MAC address in the entry is the MAC address of VSI-interface 10 on GW 1.

4. GW 2 (the VTEP for VM 4) performs the following operations:

a. De-encapsulates the ARP request and creates an ARP entry for VM 1. The entry contains VM 1's IP address (10.1.1.11), the MAC address of VSI-interface 10 on GW 1, and the incoming tunnel interface.

b. Replaces the sender MAC address of the request with the MAC address of VSI-interface 10 on GW 2, and then floods the request to the local site in VXLAN 10.

5. VM 4 creates an ARP entry for VM 1, and then sends a reply to GW 2. The MAC address in the ARP entry is the MAC address of VSI-interface 10 on GW 2.

6. GW 2 performs the following operations:

a. Creates an ARP entry for VM 4.

b. Replaces the sender MAC address of the reply with the MAC address of VSI-interface 10 on GW 2, and sends the reply to GW 1.

7. GW 1 de-encapsulates the ARP reply and creates an ARP entry for VM 4. The entry contains VM 4's IP address (10.1.1.12), the MAC address of VSI-interface 10 on GW 2, and the incoming tunnel interface.

8. For subsequent traffic between VM 1 and VM 4, GW 1 and GW 2 use their respective ARP tables to make the forwarding decision.

Inter-VXLAN traffic forwarding between sites

As shown in Figure 13, the network uses the following process to forward traffic between VXLANs (for example, from VM 1 in VXLAN 10 to VM 5 in VXLAN 20):

1. VM 1 sends an ARP request to obtain the MAC address of the gateway at 10.1.1.1.

2. GW 1 creates an ARP entry for VM 1 and replies with the MAC address of VSI-interface 10 (the gateway interface for VXLAN 10) so VM 1 will send the packets destined for VM 5 to GW 1.

3. GW 1 sends an ARP request to the local and remote sites in VXLAN 10. In the ARP request, the sender IP address is 10.1.1.11, and the sender MAC address is the MAC address of VSI-interface 10 on GW 1.

4. GW 2 performs the following operations:

a. De-encapsulates the ARP request and creates an ARP entry for VM 1. The entry contains IP address 10.1.1.11 and MAC address of VSI-interface 10 on GW 1, and the incoming tunnel interface.

b. Replaces the sender MAC address of the request with the MAC address of VSI-interface 10 on GW 2, and then floods the request to the local site in VXLAN 10.

c. Sends an ARP reply to GW 1. The reply contains IP address 10.1.1.1 and MAC address of VSI-interface 10 on GW 2).

5. When sending an ARP request in VXLAN 10, GW 1 also sends an ARP request to the local and remote sites in VXLAN 20 to obtain the MAC address of VM 5. In the ARP request, the sender IP address is 20.1.1.1, and the sender MAC address is the MAC address of VSI-interface 20 on GW 1.

6. GW 2 de-encapsulates the ARP request of VXLAN 20, replaces the sender MAC address of the request with the MAC address of VSI-interface 20 on GW 2, and then floods the request to the local site in VXLAN 20.

7. VM 5 creates an ARP entry for GW 2, and then sends a reply to GW 2. The entry contains IP address 20.1.1.1 and MAC address of VSI-interface 20 on GW 2.

8. GW 2 performs the following operations:

a. Creates an ARP entry for VM 5.

b. Sends a gratuitous ARP packet to the local and remote sites. In the packet, the sender IP address is 20.1.1.12, and the sender MAC address is the MAC address of VSI-interface 20 on GW 2.

9. GW 1 de-encapsulates the gratuitous ARP packet and creates an ARP entry for VM 5. The entry contains VM 5's IP address 20.1.1.12, the MAC address of VSI-interface 20 on GW 2, and the incoming tunnel interface.

10. For subsequent traffic between VM 1 and VM 5, GW 1 and GW 2 use their respective ARP tables to make the forwarding decision.

Figure 13 Inter-VXLAN traffic forwarding between sites

‌

VXLAN-to-external network traffic forwarding

As shown in Figure 12, the network uses the following process to forward traffic from a VXLAN to the Layer 3 network (for example, from VM 1 to the host at 50.1.1.1):

1. VM 1 sends an ARP request to obtain the MAC address of the gateway at 10.1.1.1.

2. GW 1 creates an ARP entry for VM 1 and replies with the MAC address of VSI-interface 10 (the gateway interface for VXLAN 10).

3. VM 1 sends a packet destined for the host to GW 1.

4. GW 1 performs the following operations:

a. Searches the IP routing policies or routing table for the next hop. In this example, the next hop for the packet is 10.1.1.2 (the border gateway).

b. Floods an ARP request to the local and remote sites in VXLAN 10 to obtain the MAC address of 10.1.1.2.

5. The border gateway de-encapsulates the ARP request, creates an ARP entry for GW 1, and tunnels a reply to GW 1.

6. GW 1 de-encapsulates the ARP reply and creates an ARP entry for 10.1.1.2.

7. GW 1 sends the packet destined for the host to the border gateway.

8. The border gateway de-encapsulates the packet and forwards it to the host.

Restrictions and guidelines: VXLAN IP gateway configuration

Do not configure both centralized VXLAN IP gateway settings and centralized VXLAN IP gateway group settings on a device.

As a best practice to avoid forwarding failure, set a large MTU on the traffic outgoing interfaces for VXLAN tunnels on VXLAN IP gateways.

VXLAN IP gateway tasks at a glance

To configure a VXLAN IP gateway, perform the following tasks:

1. Configure a VXLAN IP gateway

Choose one of the following tasks:

¡ Configuring a centralized VXLAN IP gateway

¡ Configuring a centralized VXLAN IP gateway group

¡ Configuring a distributed VXLAN IP gateway

2. (Optional.) Disabling remote ARP learning for VXLANs

3. (Optional.) Configuring a VSI interface

Prerequisites for VXLAN IP gateway configuration

Before you configure a centralized or distributed VXLAN IP gateway, you must perform the following tasks on VTEPs:

· Enable Layer 3 forwarding for VXLANs.

· Create VSIs and VXLANs.

Configuring a centralized VXLAN IP gateway

Restrictions and guidelines

Do not execute the local-proxy-arp enable command on the VSI interfaces of a centralized VXLAN IP gateway.

Configuring a gateway interface on a centralized VXLAN IP gateway

1. Enter system view.

system-view

2. Create a VSI interface and enter VSI interface view.

interface vsi-interface vsi-interface-id

3. Assign an IPv4 address to the VSI interface.

ip address ip-address { mask | mask-length }

By default, no IPv4 address is assigned to a VSI interface.

4. Return to system view.

quit

5. Enter VSI view.

vsi vsi-name

6. Specify a gateway interface for the VSI.

gateway vsi-interface vsi-interface-id

By default, no gateway interface is specified for a VSI.

Assigning a subnet to a VSI

About this task

Perform this task on VSIs that share a gateway interface. This task enables the VSI interface to identify the VSI of a packet.

You can assign a maximum of eight IPv4 subnets to a VSI. Make sure these subnets are on the same network as one of the IP addresses on the gateway interface.

For VSIs that share a gateway interface, the subnets must be unique.

If you remove the gateway interface from the VSI, the VSI's subnet settings are automatically deleted.

Procedure

1. Enter system view.

system-view

2. Enter VSI view.

vsi vsi-name

3. Assign a subnet to the VSI.

gateway subnet ipv4-address wildcard-mask

By default, no subnet exists on a VSI.

Configuring a centralized VXLAN IP gateway group

Configuring a VTEP group

Restrictions and guidelines

Make sure the member VTEPs use the same VXLAN settings.

Procedure

1. Enter system view.

system-view

2. Create a VSI interface and enter VSI interface view.

interface vsi-interface vsi-interface-id

This interface will be used as the gateway interface for the VSI.

3. Assign an IP address to the VSI interface.

ip address ip-address { mask | mask-length }

By default, no IP address is assigned to a VSI interface.

You must assign the same IP address to the VSI interface on each VTEP in the VTEP group.

4. Assign a MAC address to the VSI interface.

mac-address mac-address

By default, a VSI interface does not have a MAC address.

You must assign the same MAC address to the VSI interface on each VTEP in the VTEP group.

5. Return to system view.

quit

6. Enter VSI view.

vsi vsi-name

7. Specify the VSI interface as the gateway interface for the VSI.

gateway vsi-interface vsi-interface-id

By default, no gateway interface is specified for a VSI.

8. Return to system view.

quit

9. Assign the local VTEP to a VTEP group and specify a member IP address for the VTEP.

vtep group group-ip member local member-ip

By default, a VTEP is not assigned to any VTEP group.

The specified member IP address must already exist on the local VTEP and be unique in the VTEP group. You must configure a routing protocol to advertise the IP address to the transport network.

10. Specify the member IP address of all the other VTEPs in the VTEP group.

vtep group group-ip member remote member-ip&<1-8>

By default, the list of remote VTEPs is not configured.

Specifying a VTEP group as the gateway for an access layer VTEP

Prerequisites

Before you specify a VTEP group on an access layer VTEP, perform the following tasks on the VTEP:

· Enable Layer 2 forwarding for VXLANs.

· Configure VSIs and VXLANs.

· Set up VXLAN tunnels to remote sites and the VTEP group, and assign the tunnels to VXLANs.

Procedure

1. Enter system view.

system-view

2. Specify a VTEP group and all its member VTEPs.

vtep group group-ip member remote member-ip&<1-8>

By default, no VTEP group is specified.

Perform this task to specify all member VTEPs in the VTEP group.

Configuring a distributed VXLAN IP gateway

Restrictions and guidelines for distributed VXLAN IP gateway configuration

For a VXLAN that requires access to the external network, specify the VXLAN's VSI interface on the border gateway as the next hop by using one of the following methods:

· Configure a static route.

· Configure a routing policy, and apply the policy by using the apply default-next-hop command. For more information about configuring routing policies, see routing policy configuration in Layer 3—IP Routing Configuration Guide.

If both ARP flood suppression and local proxy ARP are enabled on a distributed VXLAN IP gateway, only local proxy ARP takes effect. As a best practice, do not use these features together on distributed VXLAN IP gateways. For more information about ARP flood suppression, see "Enabling ARP flood suppression."

Make sure a VSI interface uses the same MAC address to provide service on distributed VXLAN IP gateways connected to IPv4 sites. Make sure a VSI interface uses different link-local addresses to provide service on distributed VXLAN IP gateways connected to both IPv4 and IPv6 sites.

Configuring a gateway interface on a distributed VXLAN IP gateway

1. Enter system view.

system-view

2. Create a VSI interface and enter VSI interface view.

interface vsi-interface vsi-interface-id

3. Assign an IP address to the VSI interface.

IPv4:

ip address ip-address { mask | mask-length } [ sub ]

IPv6:

See IPv6 basics in Layer 3—IP Services Configuration Guide.

By default, no IP address is assigned to a VSI interface.

4. Specify the VSI interface as a distributed gateway.

distributed-gateway local

By default, a VSI interface is not a distributed gateway.

5. Enable local proxy ARP or local ND proxy.

IPv4:

local-proxy-arp enable [ ip-range startIP to endIP ]

By default, local proxy ARP is disabled.

For more information about this command, see proxy ARP commands in Layer 3—IP Services Command Reference.

IPv6:

local-proxy-nd enable

By default, local ND proxy is disabled.

For more information about this command, see IPv6 basics commands in Layer 3—IP Services Command Reference.

6. Bring up the VSI interface.

undo shutdown

By default, a VSI interface is up.

7. Return to system view.

quit

8. Enter VSI view.

vsi vsi-name

9. Specify the VSI interface as the gateway interface for the VSI.

gateway vsi-interface vsi-interface-id

By default, no gateway interface is specified for a VSI.

Enabling dynamic ARP entry synchronization for distributed VXLAN IP gateways

About this task

When local proxy ARP is enabled on distributed VXLAN IP gateways, enable this feature for all gateways to have the same ARP entries.

A controller can also synchronize ARP entries among distributed VXLAN IP gateways. When you use a controller, do not enable dynamic ARP entry synchronization.

Procedure

1. Enter system view.

system-view

2. Enable dynamic ARP entry synchronization for distributed VXLAN IP gateways.

arp distributed-gateway dynamic-entry synchronize

By default, dynamic ARP entry synchronization is disabled for distributed VXLAN IP gateways.

Assigning a subnet to a VSI

About this task

Perform this task on VSIs that share a gateway interface. This task enables the VSI interface to identify the VSI of a packet.

You can assign a maximum of eight IPv4 and IPv6 subnets to a VSI. Make sure these subnets are on the same network as one of the IP addresses on the gateway interface.

For VSIs that share a gateway interface, the subnets must be unique.

If you remove the gateway interface from the VSI, the VSI's subnet settings are automatically deleted.

Procedure

1. Enter system view.

system-view

2. Enter VSI view.

vsi vsi-name

3. Assign a subnet to the VSI.

gateway subnet { ipv4-address wildcard-mask | ipv6-address prefix-length }

By default, no subnet exists on a VSI.

Disabling remote ARP learning for VXLANs

About this task

By default, the device learns ARP information of remote user terminals from packets received on VXLAN tunnel interfaces. To save resources on VTEPs in an SDN transport network, you can temporarily disable remote ARP learning when the controller and VTEPs are synchronizing entries. After the entry synchronization is completed, enable remote ARP learning.

Restrictions and guidelines

As a best practice, disable remote ARP learning for VXLANs only when the controller and VTEPs are synchronizing entries.

Procedure

1. Enter system view.

system-view

2. Disable remote ARP learning for VXLANs.

vxlan tunnel arp-learning disable

By default, remote ARP learning is enabled for VXLANs.

Configuring a VSI interface

Configuring optional parameters for a VSI interface

1. Enter system view.

system-view

2. Enter VSI interface view.

interface vsi-interface vsi-interface-id

3. Assign a MAC address to the VSI interface.

mac-address mac-address

By default, the MAC address of a VSI interface is the bridge MAC address.

4. Configure the description of the VSI interface.

description text

The default description of a VSI interface is interface-name plus Interface (for example, Vsi-interface100 Interface).

5. Set the MTU for the VSI interface.

mtu mtu-value

The default MTU of a VSI interface is 1500 bytes.

6. Set the expected bandwidth for the VSI interface.

bandwidth bandwidth-value

The default expected bandwidth (in kbps) equals the interface baudrate divided by 1000.

The expected bandwidth is an informational parameter used only by higher-layer protocols for calculation. You cannot adjust the actual bandwidth of an interface by using this command.

Restoring the default settings of the VSI interface

Restrictions and guidelines

CAUTION:

This operation might interrupt ongoing network services. Make sure you are fully aware of the impact of this operation when you perform it on a live network.

This operation might fail to restore the default settings for some commands for reasons such as command dependencies or system restrictions. Use the display this command in interface view to identify these commands. Use their undo forms or follow the command reference to restore their default settings. If your restoration attempt still fails, follow the error message instructions to resolve the problem.

Procedure

1. Enter system view.

system-view

2. Enter VSI interface view.

interface vsi-interface vsi-interface-id

3. Restore the default settings of the VSI interface.

default

Applying a QoS policy to a VSI interface

About this task

To process the incoming and outgoing traffic of a VSI interface by using QoS features, apply a QoS policy to the VSI interface.

Procedure

1. Enter system view.

system-view

2. Enter VSI interface view.

interface vsi-interface vsi-interface-id

3. Apply a QoS policy to the VSI interface.

qos apply policy policy-name { inbound | outbound } [ enhancement ] [ extension ]

By default, no QoS policy is applied to a VSI interface.

For more information about this command and QoS policy configuration, see QoS configuration in ACL and QoS Configuration Guide.

Display and maintenance commands for VXLAN IP gateways

Execute display commands in any view and reset commands in user view.

Task	Command
Display information about VSI interfaces.	display interface [ vsi-interface [ vsi-interface-id ] ] [ brief [ description \| down ] ]
Clear statistics on VSI interfaces.	reset counters interface [ vsi-interface [ vsi-interface-id ] ]

‌

19-VXLAN Configuration Guide

VXLAN network model

VXLAN packet format

VXLAN tunnel establishment

VXLAN tunnel assignment

Traffic from the local site to a remote site

Traffic from a remote site to the local site

Intra-site unicast forwarding

Inter-site unicast forwarding

Unicast mode (head-end replication)

Access modes of VSIs

VLAN access mode

Ethernet access mode

VXLAN tasks at a glance

Configuring a VXLAN tunnel

About this task

Restrictions and guidelines

Procedure

Manually assigning VXLAN tunnels to a VXLAN

About this task

Restrictions and guidelines

Procedure

Assigning customer frames to a VSI

About this task

Procedure

Managing MAC address entries

About this task

Procedure

About this task

Procedure

About this task

Restrictions and guidelines

Procedure

About this task

Procedure

Confining unknown-unicast floods to the local site

About this task

Procedure

Hardware and feature compatibility

Restrictions and guidelines

Procedure

About this task

Hardware and feature compatibility

Procedure

About this task

Restrictions and guidelines

Procedure

Display and maintenance commands for VXLANs

Configuring VXLAN IP gateways

About VXLAN IP gateways

VXLAN IP gateways separated from VTEPs

About distributed VXLAN IP gateway deployment

Intra-VXLAN traffic forwarding between sites

Inter-VXLAN traffic forwarding between sites

VXLAN-to-external network traffic forwarding

About this task

Procedure

Restrictions and guidelines

Procedure

Prerequisites

Procedure

About this task

Procedure

About this task

Procedure

About this task

Restrictions and guidelines

Procedure

Restrictions and guidelines

Procedure

About this task

Procedure

Display and maintenance commands for VXLAN IP gateways

Intelligent Terminal Products

Product Support Services

Technical Service Solutions

Resource Center

Policy

Online Help

Become a Partner