M-LAG Technology White Paper

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Contents

Overview·· 1

Technical background· 1

Comparison between IRF and M-LAG·· 1

Benefits· 2

M-LAG implementation· 2

Concepts· 2

M-LAG network models· 4

M-LAG dual-homing network model 4

M-LAG single-homing network model 4

M-LAG system setup and operating mechanisms· 5

M-LAG system setup process· 5

DRCP· 6

Keepalive mechanism·· 6

MAD mechanism·· 7

M-LAG loop prevention mechanism·· 8

M-LAG forwarding entry synchronization· 9

M-LAG member device operating modes· 9

Configuration consistency check· 10

M-LAG dual-active gateways· 10

M-LAG sequence number check· 11

M-LAG packet authentication· 12

Traffic forwarding· 12

Unicast forwarding· 13

Unknown unicast and broadcast traffic forwarding· 16

Multicast traffic forwarding· 17

M-LAG failure handling mechanisms· 17

M-LAG interface failure handling mechanism·· 17

Peer link failure handling mechanism·· 18

Device failure handling mechanism·· 18

Uplink failure handling mechanism·· 19

Mechanisms to handle concurrent peer link and keepalive link failures· 19

STP in M-LAG network· 22

STP application scenarios· 22

Loop generated by the downlink access devices· 22

Loop generated between multi-level M-LAG systems· 23

Loop caused by initializing M-LAG configuration· 25

Loop generated by restarting devices with empty configuration· 26

Operation mechanism of STP in M-LAG·· 26

Layer 3 M-LAG gateway· 27

VLAN dual-active gateways· 27

VRRP gateways· 29

Loop detection in M-LAG network· 31

Introduction· 31

Operating mechanism·· 31

Loop detection mechanism of M-LAG member devices in a common network· 31

Loop detection mechanism on M-LAG member devices in a VXLAN network· 33

Support of DHCP/DHCPv6 for M-LAG·· 35

Overview· 35

Operating mechanism·· 35

M-LAG system setup· 35

User information entry sync mechanism·· 36

Traffic forwarding· 36

Bidirectional M-LAG on both the uplink and downlink directions· 36

Unidirectional M-LAG on only the downlink direction· 38

Unidirectional M-LAG on only the uplink direction· 39

M-LAG support in security mechanisms· 39

Introduction· 39

Operating mechanism·· 40

Port security support for M-LAG·· 40

Portal support for M-LAG·· 43

Processing of RADIUS protocol packets· 47

Layer 2 multicast support for M-LAG·· 1

Introduction· 1

Operating mechanisms· 1

Multicast source dual-homed to M-LAG·· 1

Multicast receiver dual-homed to M-LAG·· 3

Layer 3 multicast support for M-LAG·· 1

Introduction· 1

Operating mechanisms· 1

Multicast source dual-homed to M-LAG·· 1

Multicast receiver dual-homed to M-LAG·· 2

EVPN VXLAN support for M-LAG·· 1

Overview· 1

Typical networking· 1

Synchronizing the MAC address and ARP/ND information· 2

Establishing a VXLAN tunnel 2

User link backup· 2

User link backup mechanism with an Ethernet aggregate link as the peer link· 2

User link backup mechanism with a VXLAN tunnel as the peer link· 3

Traffic forwarding· 3

Unicast traffic forwarding through M-LAG interfaces· 3

Unicast traffic forwarding of single-attached interfaces· 5

BUM traffic forwarding· 7

Fault processing mechanism·· 9

Downlink fault processing mechanism·· 9

Uplink fault processing mechanism·· 11

Both peer link and keepalive link are faulty· 12

VXLAN support for M-LAG·· 13

Synchronizing the MAC address and ARP/ND information· 13

Establishing a VXLAN tunnel 14

Downlink BUM traffic on the network side· 14

EVPN data center interconnect support for M-LAG·· 14

MVXLAN support for M-LAG·· 1

Operating mechanism·· 1

Typical networking· 1

User-site backup mechanism·· 2

Multicast load sharing· 2

Layer 2 multicast support for M-LAG·· 3

Layer 3 multicast support for M-LAG·· 3

Layer 3 multicast DCI support for M-LAG·· 4

Application scenarios· 1

Single-tier M-LAG·· 1

Multi-tier M-LAG interconnect 1

Collaboration between M-LAG and STP· 2

Collaboration between M-LAG and QinQ·· 3

Collaboration between M-LAG and super VLAN· 4

Collaboration between M-LAG and private VLAN· 5

EVPN VXLAN with M-LAG·· 6

Related documentation· 7

Overview

Technical background

Multichassis Link Aggregation (M-LAG) is standardized in IEEE P802.1AX. It provides an open standard to virtualize two physical devices into one system through multichassis link aggregation to provide node redundancy in addition to link redundancy.

Figure 1 Comparison between link aggregation and M-LAG

 

Comparison between IRF and M-LAG

Table 1 shows the differences between Intelligent Resilient Framework (IRF) and M-LAG. For high availability and short service interruption during software upgrade, use M-LAG. You cannot use IRF and M-LAG in conjunction on the same device.

Table 1 Comparison between IRF and M-LAG

Item	IRF	M-LAG
Control plane	·     The IRF member devices have a unified control plane for central management. ·     The IRF member devices synchronize all forwarding entries.	·     The control plane of the M-LAG member devices is separate. ·     The M-LAG member devices synchronize entries such as MAC, ARP, and ND entries.
Device requirements	·     Hardware: The chips of the IRF member devices must have the same architecture, and typically the IRF member devices are from the same series. ·     Software: The IRF member devices must run the same software version.	·     Hardware: The M-LAG member devices can be different models. ·     Software: Some device models can run different software versions when they act as M-LAG member devices. Full support for different software versions will be implemented in the future.
Software upgrade	·     The IRF member devices are upgraded simultaneously or separately. A separate upgrade is complex. ·     Services are interrupted for about 2 seconds during an upgrade.	The M-LAG member devices are upgrade separately, and the service interruption time is shorter than 1 second during an upgrade. If the software supports graceful insertion and removal (GIR), an upgrade does not interrupt services.
Management	The IRF member devices are configured and managed in a unified manner. Single points of failure might occur when a controller manages the IRF member devices.	The M-LAG member devices are configured separately, and they can perform configuration consistency check for you to remove inconsistencies in the configuration that affects operation of the M-LAG system. You must ensure that service features also have consistent configuration. The M-LAG member devices are managed separately. No single point of failure will occur when a controller manages the M-LAG member devices.

 

	NOTE: GIR enables you to gracefully isolate a device from the network for device maintenance or upgrade. GIR minimizes service interruption by instructing the affected protocols (for example, routing protocols) to isolate the device and switch over to the redundant path. You do not need to configure graceful switchover protocol by protocol.

 

Benefits

In addition to the benefits of link aggregation, such as high bandwidth, link redundancy, and load sharing, M-LAG provides the following benefits:

·     Node redundancy. When one node fails, the other node can take over to forward traffic.

·     Loop-free Layer 2 topology without having to run a spanning tree protocol.

·     Dual-homing and load sharing.

·     Fast failover and service continuity in case of interface, link, or node failure.

·     Simplified network configuration and maintenance, thanks to the removal of spanning tree configuration.

·     Service continuity during software upgrade of one member in the multichassis link aggregation system.

M-LAG implementation

Concepts

As shown in Figure 2, M-LAG virtualizes two devices into an M-LAG system, which connects to the remote aggregation system through a multichassis aggregate link. Both Device A and Device B forward traffic to improve availability of the network.

Figure 2 M-LAG network model

 

M-LAG introduces the following concepts:

·     Primary M-LAG device—M-LAG member device assigned the primary role.

·     Secondary M-LAG device—M-LAG member device assigned the secondary role.

·     Peer link—Link used for the interaction of M-LAG protocol packets and transmission of data flows between M-LAG member devices. A peer link can be an aggregation link or a tunnel. When you use an aggregation link as a peer link, aggregate multiple links. An M-LAG system only has one peer link.

·     Peer-link interface—Interface used to set up the peer link, which could be an aggregate interface or a tunnel interface. Each M-LAG member device only has one peer-link interface.

·     Keepalive link—A Layer 3 link established between the M-LAG member devices for detecting multi-active collision with keepalive packets if the peer link fails.

·     M-LAG group—Identifies the M-LAG interfaces connected to the remote aggregation system. M-LAG interfaces connected to the same remote aggregation system belong to one M-LAG group.

·     M-LAG interface—Layer 2 aggregate interface connected to the remote aggregation system. For high availability, use dynamic link aggregation. M-LAG interfaces in an M-LAG group form a multichassis aggregate link.

 

NOTE: Typically, both the primary and secondary member devices perform traffic forwarding, with no difference in their behavior. However, in failure scenarios, their behavior differs.

 

M-LAG network models

M-LAG dual-homing network model

As shown in Figure 3, Device D and the server are both dual-homed to the M-LAG system formed by Device A and Device B. Device A and Device B perform traffic load sharing. If one member device fails, traffic will fail over to the other member device, ensuring service continuity.

Figure 3 M-LAG dual-homing network model

 

M-LAG single-homing network model

Single-homing refers to that a device is connected only to one of the M-LAG member devices in the M-LAG system. This device is known as a single-homed device.

Single-homing schemes are as follows:

·     M-LAG interface single-homing—A single-homed device is attached to an M-LAG interface on the M-LAG system.

·     Non-M-LAG interface single-homing—A single-homed device is attached to a non-M-LAG interface on the M-LAG system.

As shown in Figure 4, Device D is connected to the M-LAG system with an M-LAG interface, and Device E is connected to the M-LAG system with a non-M-LAG interface. Both Device D and Device E are connected to the M-LAG system via single-homing. The MAC address, ARP, ND, and other entries for Device D and Device E are backed up between the M-LAG member devices to provide a backup path for north-south traffic, and enhance reliability.

Figure 4 M-LAG single-homing network model

 

M-LAG system setup and operating mechanisms

M-LAG member devices exchange distributed relay control protocol data units (DRCPDUs) and keepalive packets to set up the M-LAG system and ensure correct operation of the M-LAG system. When the M-LAG system is operating correctly, the primary and secondary member devices share the load for traffic forwarding. If an interface, link or device failure occurs, the M-LAG system can ensure that service operations are not affected.

M-LAG system setup process

As shown in Figure 5, two devices perform the following operations to form an M-LAG system:

1.     Send DRCPDUs over the peer link to each other and compare the DRCPDUs to determine the M-LAG system stackability and device roles:

a.     Compare the M-LAG system settings. The devices can form an M-LAG system if they have consistent M-LAG system settings.

b.     Determine the device roles based on the initial M-LAG role, M-LAG MAD DOWN status, health condition, M-LAG role priority, and the bridge MAC address.

c.     Perform configuration consistency check. For more information, see "Configuration consistency check."

2.     Send keepalive packets over the keepalive link after primary M-LAG member election to verify that the peer system is operating correctly.

3.     Synchronize configuration data by sending DRCPDUs over the peer link. The configuration data includes MAC address entries and ARP entries.

Figure 5 M-LAG system setup process

 

DRCP

M-LAG uses Distributed Relay Control Protocol (DRCP) for multichassis link aggregation. DRCP runs on the peer link and uses DRCPDUs to advertise the M-LAG configuration.

DRCP operating mechanism

M-LAG-enabled devices exchange DRCPDUs over the peer link to determine whether they can form an M-LAG system. The devices can form an M-LAG system if they have consistent M-LAG system settings.

DRCP timeout timers

DRCP uses a timeout mechanism to specify the amount of time that a peer-link interface must wait to receive DRCPDUs before it determines that the peer interface is down. This timeout mechanism provides the following timer options:

·     Short DRCP timeout timer, which is fixed at 3 seconds. If this timer is used, the peer interface sends one DRCPDU every second.

·     Long DRCP timeout timer, which is fixed at 90 seconds. If this timer is used, the peer interface sends one DRCPDU every 30 seconds.

Keepalive mechanism

M-LAG uses a keepalive mechanism to detect multi-active collision in case of peer link failure.

Keepalive timers

The following timers are used for keepalive detection.

Table 2 Keepalive timers

Keepalive timer	Description	Default value
Keepalive interval	The time interval at which keepalive packets are sent.	1 second.
Keepalive hold timeout	Time that the device must wait to detect the peer link failure cause after the peer link goes down.	3 seconds.
Keepalive timeout	Keepalive packet timeout time.	5 seconds.

 

Keepalive operating mechanism

If the local M-LAG member device receives a keepalive packet sent by the peer M-LAG member device within the keepalive timeout period time, the following rules apply:

·     If the peer link is down, the device starts the keepalive hold timeout timer.

¡     If a DRCPDU is received before the timer expires, the peer link state is restored to up, and the M-LAG system operates correctly.

¡     If no DRCPDU is received before the timer expires, the local and remote M-LAG member devices elect a primary member device based on the received keepalive packet. This ensures that only one M-LAG member device in the M-LAG system forwards traffic and avoids that both M-LAG member devices are primary devices.

·     If the peer link is up, the M-LAG system is operating correctly.

If the local M-LAG member device does not receive a keepalive packet sent by the peer M-LAG member device within the keepalive timeout period, the following rules apply:

·     If the peer link state is down, the remote M-LAG member device is considered as down. The keepalive hold timeout timer is set. After this timer expires, the following rules apply:

¡     If the local device is the primary member device and it has up M-LAG interfaces, it remains the primary role. Otherwise, the role of the local device changes to none.

¡     If the local device is the secondary member device and it has up M-LAG interfaces, it takes over the primary role. Otherwise, the role of the local device changes to none.

With the none role, a device cannot send or receive keepalive packets, and the keepalive link is down.

·     If the peer link is up, the keepalive link is down. The primary and secondary member devices are working correctly and output a log message to remind users to check the keepalive link status.

MAD mechanism

A multi-active collision occurs if the peer link goes down while the keepalive link is up. To avoid network issues, M-LAG MAD takes one of the following actions on the secondary M-LAG member device when the M-LAG system splits:

·     M-LAG MAD DOWN—Shut down all network interfaces except the interfaces excluded manually or by the system.

·     NONE—Do not shut down any network interfaces except the interfaces configured manually or by the system to be shut down by M-LAG MAD.

To retain only special-purpose interfaces in up state on the secondary M-LAG member device when the M-LAG system splits, configure M-LAG MAD to take the M-LAG MAD DOWN action. M-LAG MAD will shut down all network interfaces except the following:

·     Interfaces automatically or manually excluded from being shut down by M-LAG MAD.

·     Network interfaces forced to stay in up state.

The following interfaces are automatically excluded from being shut down by M-LAG MAD:

·     Peer-link interface.

·     Aggregation member interfaces if a Layer 2 aggregate interface is used as the peer-link interface.

·     M-LAG interfaces.

·     Management interfaces.

To prevent package loss after the recovery of the peer link, the secondary member device tries to finish the synchronization of forwarding entries such as MAC address and ARP entries within the recovery delay recovery time. Then, the device brings up the interfaces in M-LAG MAD DOWN state.

To have the secondary M-LAG member device retain a large number of interfaces in up state and shut down the remaining interfaces, configure M-LAG MAD to take the NONE action. M-LAG MAD will shut down only the network interfaces that are configured manually or by the system to be shut down by M-LAG MAD. One applicable scenario of this method is the EVPN environment in which you use a VXLAN tunnel as the peer link. In this scenario, you must retain a large number of logical interfaces (for example, tunnel and loopback interfaces) in up state.

M-LAG loop prevention mechanism

M-LAG has a loop prevention mechanism that can create a loop-free network. As shown in Figure 6, local forwarding is prioritized for the unicast traffic arriving at the M-LAG member devices from the access device or network side, and the peer link is typically not used to forward data traffic. When the traffic is forwarded to the peer M-LAG member device over the peer link, unidirectional traffic isolation is performed between the peer link and the M-LAG interface. The traffic received by the peer-link interface will not be forwarded out of the M-LAG interfaced.

Figure 6 M-LAG loop prevention mechanism

 

M-LAG forwarding entry synchronization

As shown in Figure 7, the M-LAG member devices synchronize MAC address entries, ARP entries, DHCP entries, ND entries, and other entries in real time between them. In single-homing scenarios, the entries for a single-homing M-LAG interface will be synchronized to the peer-link interface of the M-LAG peer device, allowing downstream traffic to fail over via the peer link and be forwarded to Device F.

Figure 7 M-LAG forwarding entry synchronization

 

M-LAG member device operating modes

An M-LAG member device operates in the following modes:

·     M-LAG system mode—The device acts as a member device of the M-LAG system and participates in packet forwarding. In the LACPDUs exchanged between the access device and the M-LAG member device, the LACP system ID is composed of the M-LAG system MAC address and M-LAG system priority.

·     M-LAG standalone mode—The device operates independently, disconnected from the M-LAG system, and forwards packets on its own. In the LACPDUs exchanged with the M-LAG member device, the LACP system ID is composed of the LACP system MAC address and LACP system priority.

The secondary member device changes to M-LAG standalone mode immediately or after a delay when it detects that both the peer link and the keepalive link are down.

In M-LAG standalone mode, the LACPDUs sent out of an M-LAG interface by each M-LAG member device contain the interface-specific LACP system MAC address and LACP system priority.

M-LAG standalone mode helps avoid traffic forwarding issues in a multi-active situation by allowing only the member ports in the M-LAG interfaces on one member device to forward traffic. The Selected state of the member ports in the M-LAG interfaces in an M-LAG group depends on their LACP system MAC address and LACP system priority. If an M-LAG interface has a lower LACP system priority value or LACP system MAC address, the member ports in that M-LAG interface become Selected to forward traffic.

Configuration consistency check

During M-LAG system setup, M-LAG member devices exchange the configuration over the peer link and perform configuration consistency check to verify their consistency in the following configurations:

·     Type 1 configuration—Settings that affect traffic forwarding of the M-LAG system. If an inconsistency in type 1 configuration is detected, the secondary M-LAG member device shuts down its M-LAG interfaces. The link status is normal, but packet loss might persist.

·     Type 2 configuration—Settings that affect only service features. If an inconsistency in type 2 configuration is detected, the secondary M-LAG member device does not shut down its M-LAG interfaces.

M-LAG dual-active gateways

When a Layer 3 network is dual-homed to an M-LAG system, each M-LAG member device uses a logical interface as the gateway interface for the Layer 3 network. Both logical interfaces are active to perform Layer 3 forwarding, and they use the same IP address and MAC address. When both links to the M-LAG system are operating correctly, the links load share traffic to balance bandwidth usage. When one of the links fails, the M-LAG system switches the traffic on the failed link to the other link to ensure high availability.

Typically, dual-active gateways are deployed if downstream devices run a dynamic routing protocol to communicate with the M-LAG system. As shown in Figure 8, IGP or BGP runs between Device C and the M-LAG system. To configure dual-active VLAN interfaces, perform the following tasks on Device A and Device B:

1.     Create VLAN interfaces for the same VLAN, VLAN-interface 100 for example, assign the same IPv4, IPv6, and MAC addresses to the VLAN interfaces, and assign the M-LAG interfaces to that VLAN.

2.     Assign different M-LAG VIPs to the VLAN interfaces.

Device A and Device B use their M-LAG VIPs to set up IGP or BGP peer relationships with Device C and with each other.

Figure 8 Layer 3 access to an M-LAG system through dynamic routing protocols

 

M-LAG sequence number check

M-LAG sequence number check protects M-LAG member devices from replay attacks.

As shown in Figure 9, with this feature enabled, the M-LAG member devices insert a sequence number into each outgoing DRCPDU or keepalive packet and the sequence number increases by 1 for each sent packet. When receiving a DRCPDU or keepalive packet, the M-LAG member devices check its sequence number and drop the packet if the check result is either of the following:

·     The sequence number of the packet is the same as that of a previously received packet.

·     The sequence number of the packet is smaller than that of the most recently received packet.

Figure 9 M-LAG sequence number check

 

M-LAG packet authentication

M-LAG packet authentication prevents DRCPDU and keepalive packet tampering from causing link flapping.

As shown in Figure 10, with this feature enabled, the M-LAG member devices compute a message digest by using an authentication key for each outgoing DRCPDU or keepalive packet and insert the message digest into the packet. When receiving a DRCPDU or keepalive packet, an M-LAG member device computes a message digest and compares it with the message digest in the packet. If the message digests match, the packet passes authentication. If the message digests do not match, the device drops the packet.

Figure 10 M-LAG packet authentication

 

Traffic forwarding

If the M-LAG system is successfully established, the primary and secondary M-LAG member devices load share traffic.

Unicast forwarding

A shown in Figure 11, Device D is dual-homed to the M-LAG system, and the M-LAG system forwards unicast traffic as follows:

·     North-south unicast traffic—When an M-LAG member devices receive traffic transmitted by the access device over the multichassis link aggregation, they follow the local-first forwarding rule to forward traffic jointly. The traffic heading to the network side will be forwarded according to the routing table by the M-LAG member devices.

·     East-west unicast traffic—Local forwarding is prioritized for Layer 2 traffic, and Layer 3 traffic is forwarded by dual-active gateways. Both types of traffic do not pass through the peer link, but are directly forwarded by the local M-LAG member devices.

Figure 11 Unicast forwarding

 

North-south unicast traffic forwarding

As shown in Figure 12, uplink traffic is forwarded as follows:

·     For unicast traffic from an M-LAG interface, Device A and Device B load share the traffic and perform local-first forwarding to avoid heavy load on the peer link.

·     For unicast traffic from a non-M-LAG interface, Device B performs local-first forwarding. The traffic is not transmitted over the peer link.

Figure 12 Uplink traffic forwarding

 

As shown in Figure 13, local-first forwarding is performed on unicast traffic from the network.

Figure 13 Downlink traffic forwarding

 

East-west unicast traffic forwarding

As shown in Figure 14, east-west unicast traffic is forwarded as follows:

·     If a local outgoing interface exists, Device A and Device B perform local-first forwarding.

·     If no local outgoing interface exists, the traffic is sent over the peer link to the M-LAG peer for forwarding.

Figure 14 East-west unicast traffic forwarding

 

Unknown unicast and broadcast traffic forwarding

As shown in Figure 15, Device D is dual-homed to the M-LAG system, and unknown unicast and broadcast traffic is forwarded as follows:

·     Upon receiving unknown unicast or broadcast traffic from a non-M-LAG interface, Device B will forward the traffic to the same broadcast domain on the connected device. When the traffic reaches Device A, the unidirectional isolation mechanism between the peer-link interface and M-LAG interfaces will prevent the traffic from being forwarded to Device D.

·     When the unknown unicast or broadcast traffic transmitted by Device D reaches an M-LAG member device (Device A for example), the traffic is forwarded to the connected devices within the same broadcast domain. When the traffic reaches Device B, the unidirectional isolation mechanism between the peer-link interface and M-LAG interfaces will prevent the traffic from being forwarded to Device D.

Figure 15 Unknown unicast and broadcast traffic forwarding

 

Multicast traffic forwarding

For more information, see "Layer 2 multicast support for M-LAG" and "Layer 3 multicast support for M-LAG."

M-LAG failure handling mechanisms

M-LAG interface failure handling mechanism

When the M-LAG interface of one M-LAG member device fails, the M-LAG system forwards traffic through the other M-LAG member device.

As shown in Figure 16, Device A and Device B form an M-LAG system, to which Device C is attached through a multichassis aggregation. If traffic to Device C arrives at Device B after the M-LAG interface connected Device B to Device C has failed, the M-LAG system forwards the traffic as follows:

1.     Device B sends the traffic to Device A over the peer link.

2.     Device A forwards the downlink traffic received from the peer link to Device C.

After the faulty M-LAG interface comes up, Device B forwards traffic to Device C through the M-LAG interface.

Figure 16 M-LAG interface failure handling mechanism

 

NOTE: An M-LAG member device supports zero packet loss during downlink known unicast traffic failover in the following conditions: ·     An M-LAG interface is manually restarted. ·     The device reboots.

 

Peer link failure handling mechanism

As shown in Figure 17, multi-active collision occurs if the peer link goes down while the keepalive link is up. To avoid network issues, the secondary M-LAG member device sets all network interfaces to M-LAG MAD DOWN state, except for interfaces excluded from the shutdown action by M-LAG MAD.

In this situation, the primary M-LAG member device forwards all traffic for the M-LAG system.

When the peer-link interface comes up, the secondary M-LAG member device does not bring up the network interfaces immediately. Instead, it starts a delay timer and begins to recover data from the primary M-LAG member device. When the delay timer expires, the secondary M-LAG member device brings up all network interfaces.

Figure 17 Peer link failure handling mechanism

 

Device failure handling mechanism

As shown in Figure 18, when the primary M-LAG member device fails, the secondary M-LAG member device takes over the primary role to forward all traffic for the M-LAG system. When the faulty device recovers, it becomes the secondary M-LAG member device.

When the secondary M-LAG member device fails, the primary M-LAG member device forwards all traffic for the M-LAG system.

Figure 18 Device failure handling mechanism

 

Uplink failure handling mechanism

Uplink failure does not interrupt traffic forwarding of the M-LAG system. As shown in Figure 19, when the uplink of Device A fails, Device passes traffic destined for the IP network to Device B for forwarding.

To enable faster traffic switchover in response to an uplink failure and minimize traffic losses, configure Monitor Link to associate the M-LAG interfaces with the uplink interfaces. When the uplink interface of an M-LAG member device fails, that device shuts down its M-LAG interface for the other M-LAG member device to forward all traffic of Device C.

Figure 19 Uplink failure handling mechanism

 

Mechanisms to handle concurrent peer link and keepalive link failures

When both the peer link and the keepalive link are down, the M-LAG member devices handle this situation depending on your configuration.

Default failure handling mechanism

Figure 20 shows the default mechanism to handle peer link and keepalive link failures when the M-LAG standalone mode and M-LAG MAD DOWN state persistency features are not configured.

·     If the peer link goes down while the keepalive link is up, the M-LAG member devices negotiate their roles over the keepalive link. M-LAG MAD shuts down all network interfaces on the secondary M-LAG member device except those excluded from the shutdown action by M-LAG MAD.

·     If the keepalive link goes down while the peer link is down, the secondary M-LAG member device sets its role to primary and brings up the network interfaces in M-LAG MAD DOWN state to forward traffic. In this situation, both of the M-LAG member devices might operate with the primary role to forward traffic. Forwarding errors might occur because the M-LAG member devices cannot synchronize MAC address entries over the peer link.

·     If the keepalive link is down before the peer link goes down, M-LAG MAD will not place network interfaces in M-LAG MAD DOWN state. Both M-LAG member devices can operate with the primary role to forward traffic.

Figure 20 Default failure handling mechanism

 

Failure handling mechanism with M-LAG MAD DOWN state persistence

Figure 21 shows the mechanism to handle peer link and keepalive link failures when the M-LAG MAD DOWN state persistence feature is configured.

·     If the peer link goes down while the keepalive link is up, the M-LAG member devices negotiate their roles over the keepalive link. M-LAG MAD shuts down all network interfaces on the secondary M-LAG member device except those excluded from the shutdown action by M-LAG MAD.

·     If the keepalive link goes down while the peer link is down, the secondary M-LAG member device sets its role to primary, but it does not bring up the network interfaces in M-LAG MAD DOWN state. Only the original primary member device can forward traffic.

·     If the keepalive link is down before the peer link goes down, M-LAG MAD will not place network interfaces in M-LAG MAD DOWN state. Both M-LAG member devices can operate with the primary role to forward traffic.

Figure 21 Failure handling mechanism with M-LAG MAD DOWN state persistence

 

As shown in Figure 22, you can bring up the interfaces in M-LAG MAD DOWN state on the secondary M-LAG member device for it to forward traffic if the following conditions exist:

·     Both the peer link and the keepalive link are down.

·     The primary M-LAG member device fails or its M-LAG interface fails.

Figure 22 Bringing up the interfaces in M-LAG MAD DOWN state

 

Failure handling mechanism with M-LAG standalone mode

Figure 23 shows the mechanism to handle peer link and keepalive link failures when the M-LAG standalone mode feature is configured.

·     If the peer link goes down while the keepalive link is up, the M-LAG member devices negotiate their roles over the keepalive link. M-LAG MAD shuts down all network interfaces on the secondary M-LAG member device except those excluded from the shutdown action by M-LAG MAD.

·     If the keepalive link goes down while the peer link is down, both M-LAG member devices change to M-LAG standalone mode. The secondary M-LAG member device sets its role to primary and brings up its network interfaces in M-LAG MAD DOWN state. In M-LAG standalone mode, only the aggregation member ports on one M-LAG member device can become Selected to forward traffic.

·     If the keepalive link is down before the peer link goes down, both M-LAG member devices change to M-LAG standalone mode.

Figure 23 Failure handling mechanism with M-LAG standalone mode

 

STP in M-LAG network

STP application scenarios

M-LAG itself has a loop avoidance mechanism. In normal conditions, loops will not occur in an M-LAG network. In a multi-level M-LAG network, loops might occur in the network when you set up the network incorrectly, initialize M-LAG configuration, or restart devices with empty configurations. In this case, you must deploy spanning tree protocol (STP) to avoid loops. Typical scenarios that require deploying STP include:

·     The downlink access devices generate a loop.

·     A loop occurs on access devices between multi-level M-LAG system.

·     Initializing M-LAG configuration causes a loop.

·     Restarting the devices with empty configurations generates a loop.

Loop generated by the downlink access devices

As shown in Figure 24, downlink devices Device M and Device N are connected to Device A and Device B through non-M-LAG interfaces, respectively. Device M and Device N are interconnected.

Figure 24 Schematic diagram for a loop between downlink devices accessing an M-LAG system

 

As shown in Figure 25, the traffic between Device M and Device N will pass through the peer link between Device A and Device B to form a loop. To avoid the loop, deploy STP on Device A, Device B, Device M, and Device N to block the link between Device M and Device N. Configure Device A and Device B as designated bridges. Then, traffic between Device M and Device N will be forwarded through Device A and Device B to avoid a loop.

Figure 25 Topology diagram for a loop between downlink devices accessing an M-LAG system

 

Loop generated between multi-level M-LAG systems

As shown in Figure 26, in a multi-level M-LAG network, Device A and Device C are mistakenly connected through non-M-LAG interfaces.

Figure 26 Schematic diagram for a loop between multi-level M-LAG systems

 

As shown in Figure 27, the traffic between Device A and Device C will pass through the peer links to form a loop. To avoid a loop, deploy STP on Device A, Device B, Device C, and Device D to block the incorrect connection between Device A and Device C. Configure Device C and Device D as designated bridges. Then, traffic between Device A and Device C will be forwarded through M-LAG interfaces of the multi-level M-LAG system to avoid a loop.

Figure 27 Topology diagram for a loop between multi-level M-LAG systems

 

Loop caused by initializing M-LAG configuration

As shown in Figure 28, connect the devices according to the M-LAG network requirements. After you configure the M-LAG settings on the M-LAG system, a temporary loop exists in the network before the M-LAG system is set up. At this time, you can deploy STP to block ports to avoid a loop.

As a best practice, first complete setting up the M-LAG system and verify the M-LAG system before integrating it into the existing network to avoid a loop generated due to initializing the M-LAG configuration.

Figure 28 Schematic diagram for a loop generated through initializing M-LAG configuration

 

Loop generated by restarting devices with empty configuration

As shown in Figure 29, two M-LAG member devices form an M-LAG system. If one device is restarted with empty configuration, it will not join the M-LAG system after restart. Instead, it will operate as a standalone physical device that can forward traffic. The other M-LAG member device believes the peer M-LAG member device fails and takes over to forward traffic. As a result, a loop occurs in the network. By deploying STP, you can avoid loops.

Figure 29 Schematic diagram for a loop generated through restarting the devices with empty configurations

 

Operation mechanism of STP in M-LAG

In an M-LAG network, the STP operating mechanism on M-LAG member devices must be adjusted to ensure normal operation in the M-LAG network, because the two M-LAG member devices is virtualized into one device.

·     STP is controlled by the primary device. No matter which M-LAG member device the designated port resides on, the primary device generates the STP BPDUs and sends them out the designated port. The STP status of a port is also determined by the primary device.

·     The secondary device does not generate BPDUs, and it cannot determine the STP status of a port. The secondary device forwards the received BPDUs to the primary device through the peer link.

·     The STP port status of the M-LAG interfaces on the two M-LAG member devices always remains consistent.

·     The peer link does not run the STP protocol.

 

NOTE: The two M-LAG member devices that form the M-LAG system have the same virtual MAC address (MAC address of the M-LAG system). The M-LAG member devices run the STP protocol based on the virtual MAC address. Therefore, both M-LAG member devices can act as the root bridges of STP.

 

Layer 3 M-LAG gateway

VLAN dual-active gateways

As shown in Figure 30, VLAN dual-active gateways refer to two M-LAG member devices in the M-LAG system acting as user-side gateways. They respond to user-side ARP/ND requests and forward user-side packets to improve gateway reliability.

Figure 30 M-LAG VLAN dual-active gateway deployment scheme

 

The deployment scheme for the M-LAG VLAN dual-active gateways is as follows: On two M-LAG member devices in the same M-LAG system, create VLAN interfaces with the same number (for example, VLAN-interface 100) and configure them with the same IPv4 address, IPv6 address, and MAC address. The IPv4 address and IPv6 address of this interface act as the gateway addresses, so that both IPv4 and IPv6 users can access external networks through the gateways.

The operating mechanism of M-LAG VLAN dual-active gateways is:

·     M-LAG member devices use the local-first forwarding principle. A member device directly forwards the received packets rather than transmits the packets over the peer link to the peer M-LAG member device for forwarding. For example, Device E, an M-LAG member device, receives an ARP request sent by a VM. In this case, Device E directly sends an ARP reply to the VM rather than forwards the ARP request to M-LAG member device Device F for processing.

·     When a link fails, traffic can quickly switch to the other link to ensure reliability. For example, if the link between Device E and Device G fails, the traffic is processed as follows:

¡The downlink traffic accessing Device D is quickly switched to Device F for processing rather than forwarded to Device E.

¡When the uplink traffic accessing the VM is forwarded to Device F, Device F directly forwards the traffic to the VM after processing. When the traffic is forwarded to Device E, the traffic will be transmitted over the peer link to Device F for processing and then be forwarded to the VM.

·     The two access links can simultaneously process user traffic to improve bandwidth usage and load-share traffic between both access links.

In the M-LAG VLAN dual-active gateway scenario, M-LAG member devices act as gateways for Layer 3 forwarding. Because the VLAN interfaces acting as the gateways have the same IP address and MAC address, the M-LAG member devices cannot establish routing neighbor relationships with the user-side devices by using that IP address. When the VLAN dual-active gateways need to establish a routing neighbor relationship with Device B, you can configure the M-LAG virtual IP addresses and deploy routing protocols on the VLAN interfaces acting as the gateways. Then, the virtual IP addresses will be used to establish a neighbor relationship with the downlink device Device B. For specific deployment methods, see Figure 31 and Table 3.

Figure 31 Configuring the M-LAG virtual IP addresses on the gateway interfaces to establish a routing neighbor in the M-LAG VLAN dual-active gateway scenario

 

Table 3 Configuring the M-LAG virtual IP addresses on the gateway interfaces to establish a routing neighbor in the M-LAG VLAN dual-active gateway scenario

Application scenario	Deployment scheme	Traffic model
Deploy dynamic routing on downlink device Device B and M-LAG member devices	·     Create VLAN interfaces with the same number (for example, VLAN-interface 100) on two M-LAG member devices in the same M-LAG system, and the VLAN interfaces act as the IPv4 and IPv6 dual-active gateways. Configure the same IP addresses and MAC address for this VLAN interface on both M-LAG member devices, and the IP addresses act as the gateway addresses. Device B is connected to the M-LAG member devices through M-LAG interfaces, and both IPv4 and IPv6 traffic can access the external network through gateway addresses. ·     Configure different M-LAG virtual IP addresses in the same subnet for the VLAN interfaces acting as gateways on the two M-LAG member devices in the same M-LAG system. Use the virtual IP addresses to establish Layer 3 connections to Device B through BGP or OSPF for Layer 3 connectivity. ·     Create a VLAN interface with the same number (for example, VLAN-interface 101) on the two M-LAG member devices in the same M-LAG system, and add the peer link aggregate interface to that VLAN. Configure the VLAN interface on the two M-LAG member devices with different IP addresses in the same subnet to achieve Layer 3 communication between the two M-LAG member devices. If the link from M-LAG 1 or M-LAG 2 to uplink device Device A fails, the packets can be routed to the peer M-LAG member device for processing. ·     Traffic between the M-LAG member devices and the uplink device Device A is load-balanced through ECMP routes deployed on the Layer 3 interfaces.	·     When Device B sends Layer 2 traffic, it looks up the MAC address table, finds the entries with the outgoing interfaces as aggregate interfaces, and load-shares traffic on M-LAG member devices. An M-LAG member device performs Layer 2 forwarding according to the MAC address table based on the local-first principle. ·     When Device B sends Layer 3 traffic, it finds the routes with the outgoing interface as VLAN-interface 100 based on the routing table generated by the configured dynamic routing protocol. Device B forwards the traffic through the aggregate interface assigned to VLAN 100 to load-share traffic on M-LAG member devices. The M-LAG member devices perform Layer 3 forwarding based on the FIB table. ·     External network traffic accessing Device B is load-balanced and forwarded to the M-LAG member devices based on ECMP routes. The M-LAG member devices forward traffic to Device B based on local routing information.
BFD fast detection (if necessary)	The two M-LAG member devices separately use the M-LAG virtual IP address to establish BFD sessions with the secondary IP address of VLAN-interface 100 on the downlink device.	N/A

 

VRRP gateways

Deploy VRRP on M-LAG member devices to provide redundant backup gateways for downlink access devices. For the Layer 3 forwarding deployment scheme of M-LAG+VRRP, see Figure 32 and Table 4.

Figure 32 M-LAG+VRRP Layer 3 forwarding scheme

 

Table 4 M-LAG+VRRP Layer 3 forwarding scheme

Deployment scheme

Traffic model

·     Deploy VRRP on M-LAG member devices, with the VRRP virtual IP address as the gateway address for Device B. Device B is dual-homed to the VRRP gateways through M-LAG interfaces. ·     Create a VLAN interface for the VLAN to which the M-LAG interfaces belong. Configure the VLAN interfaces of the two M-LAG member devices with different IP addresses in the same subnet as the primary IP addresses and with different IP addresses in another subnet as the secondary IP addresses. ·     Two M-LAG member devices establish a routing neighbor by using the Layer 3 interfaces of the peer link to act as a backup Layer 3 link. If the link between M-LAG1 or M-LAG2 and the uplink device Device A fails, the packets can be routed to the peer M-LAG member device for processing. ·     Traffic between the M-LAG member devices and the uplink device Device A is load-balanced through ECMP routes deployed on the Layer 3 interfaces.

·     When Device B sends packets to other subnets, the packets are load-balanced to two M-LAG member devices through M-LAG interfaces. Both M-LAG member devices can act as VRRP virtual routers to forward the packets. ·     When Device B sends Layer 3 traffic, traffic is forwarded based on IP routes established between Device B and the secondary IP addresses of the VLAN interfaces on the M-LAG member devices. ·     External network traffic accessing Device B is load-balanced and forwarded to the M-LAG member devices based on ECMP routes. The M-LAG member devices forward traffic to Device B based on local routing information.

 

A VRRP gateway receives a packet sent from the external network to Device B and with the destination MAC address as the real MAC address of the peer M-LAG member device. According to the normal forwarding process, the local M-LAG member device will send the packet to the destination M-LAG member device through the peer link. However, due to the loop avoidance mechanism of M-LAG, the destination M-LAG member device will not forward the packet received through the M-LAG interface to Device B. As a result, the packet is discarded. For loop avoidance in an M-LAG+VRRP network, M-LAG member devices must synchronize their real MAC addresses. Then, an M-LAG member device can locally forward packets with the destination MAC address as the real MAC address of the peer M-LAG member device at Layer 3. The MAC address synchronization feature is always enabled, and does not require manual configuration.

Loop detection in M-LAG network

Introduction

With loop detection running in an M-LAG network, if a device detects a loop on an M-LAG interface, the two M-LAG member devices of the M-LAG system act as one virtual device to respond to the loop on the M-LAG interface with the same number in the same way. If a device detects a loop on the interface of a single-homed access device, the two M-LAG member devices respond to the loop as standalone devices.

Operating mechanism

Loop detection mechanism of M-LAG member devices in a common network

Introduction

As shown in Figure 33, Device A and Device B form an M-LAG system and act as a virtual device connected to the network to improve availability. Device C and Device D are dual-homed to the M-LAG system, while Device F, Device G, and Device H are single-homed to Device A. This section will describe the operating mechanism of loop detection in the following scenarios.

·     Dual-homed access scenario—For example, a loop is formed by the M-LAG system, Device C, Device F, and Device D.

·     Single-homed access scenario 1—For example, a loop is formed by Device A, Device B, Device C, and Device F.

·     Single-homed access scenario 2—For example, a loop is formed by Device A, Device G, and Device H.

Figure 33 Diagram for a common network running loop detection

 

Operating mechanism of loop detection

M-LAG member devices can detect loops and process the ports where loops occur only after loop detection is enabled. According to different scenarios, the recommended loop detection configurations are as follows:

·     Dual-homed access scenario—You must enable loop detection on both member devices of an M-LAG system. If you do not do that, only one member device will process the port loop, which will not eliminate the loop.

·     Single-homed access scenario 1—If you enable loop detection only on Device A, enable loop detection on interface BAGG4 or globally as a best practice. Then, Device A can detect a loop on Port 1 and effectively eliminate the loop. If you enable loop detection only on Port 1 of Device A and the device detects a loop on interface BAGG4, the device only blocks or shuts down interface BAGG4. This is not enough to eliminate the loop. As a best practice, also enable loop detection on Device B to ensure that the loop can be eliminated.

·     Single-homed scenario 2—You only need to enable loop detection on Device A, which will act as a standalone device to detect loops.

Loop detection frame transmission mechanism

For the dual-homed access scenario, the loop detection frame transmission mechanism is as follows:

·     After loop detection is enabled, both devices of the M-LAG system will send loop detection frames on the M-LAG interfaces. The loop detection frames have the same source MAC address (MAC address of the M-LAG system).

·     When an M-LAG member device receives a loop detection frame from the M-LAG interface, the M-LAG member device will synchronize the frame to the other M-LAG member device through the peer link to avoid a single point of failure, which will cause the other M-LAG member device to fail to receive the loop detection frame.

For single-homed access scenario 1, the loop detection frame transmission mechanism is as follows:

·     After loop detection is enabled, an M-LAG member device sends loop detection frames out the interface with loop detection enabled. The source MAC address of the frames is the MAC address of the M-LAG system.

·     An M-LAG member device synchronizes the loop detection frames received from the M-LAG interface to the other M-LAG member device through the peer link. An M-LAG member device does not synchronize the loop detection frames received from a non-M-LAG interface through the peer link.

For single-homed access scenario 2, the loop detection frame transmission mechanism is as follows:

·     After loop detection is enabled, an M-LAG member device sends loop detection frames out the interface with loop detection enabled. The source MAC address of the frames is the MAC address of the M-LAG system.

·     An M-LAG member device does not synchronize the loop detection frames received from a non-M-LAG interface to the other M-LAG member device through the peer link.

Mechanism for determining the occurrence of a loop

With loop detection enabled, when an M-LAG member device receives loop detection frames sent by the M-LAG system on any port outside the peer link, the device determines that a loop has occurred on the port.

When an M-LAG member device with loop detection enabled receives synchronized loop detection frames through the peer link, the local member device determines that a loop also has occurred on the local M-LAG interface in the same M-LAG group as the peer M-LAG interface that receives the loop detection frames. For example, Device A receives a loop detection frame on interface BAGG4 and synchronizes the frame to Device B through the peer link. Even if Device B does not receive a loop detection frame on interface BAGG4 , Device B still determines that a loop has occurred on interface BAGG4 and takes an appropriate action.

Processing mechanism for detected loops

Suppose an M-LAG member device receives loop detection frames sent by the local device or synchronized through the peer link, and determines a loop has occurred on a port. In this case, the device will take an action such as shutting down, blocking, or prohibiting MAC address learning on the port according to the device configuration.

Loop detection mechanism on M-LAG member devices in a VXLAN network

Introduction

As shown in Figure 34, VXLAN uses M-LAG to connect two physical devices and virtualize them into one device. Use this virtual device as a VTEP to avoid single point of failure and improve reliability of the VXLAN network. This section will describe the operating mechanism of loop detection in the following scenarios.

·     Dual-homed access scenario—Server 2 is dual-homed to the VTEP for mutual communication with Server 1 through the VXLAN network, and also communicates with Server 1 through a directly connected Layer 2 network. In this case, a loop occurs.

·     Single-homed access scenario—Server 4 is single-homed to the VTEP for mutual communication with Server 3 through the VXLAN network, and also communicates with Server 3 through a directly connected Layer 2 network. In this case, a loop occurs.

Figure 34 Network diagram for support of VXLAN for running loop detection on an M-LAG network

 

Operating mechanism of loop detection

An M-LAG member device can detect loops and process the ACs where loops have occurred only when it has loop detection enabled. According to different scenarios, the recommended loop detection configurations are as follows:

·     Dual-homed access scenario—You must enable loop detection on both member devices of an M-LAG system. If you do not do that, only one member device will process the loops, which will not eliminate the loops.

·     Single-homed access scenario—You only need to enable loop detection on the VTEP with attached endpoints.

Loop detection frame transmission mechanism

For dual-homed access scenarios, the loop detection frame transmission mechanism is as follows:

·     After loop detection is enabled, both devices of the M-LAG system will send loop detection frames out ACs on the M-LAG interfaces. The loop detection frames have the same source MAC address (MAC address of the M-LAG system).

·     When an M-LAG member device receives a loop detection frame from the M-LAG interface, the M-LAG member device will synchronize the frame to the other M-LAG member device through the peer link to avoid a single point of failure, which will cause the other M-LAG member device to fail to receive the loop detection frame.

For single-homed access scenarios, the loop detection frame transmission mechanism is as follows:

·     After loop detection is enabled, the M-LAG member devices send loop detection frames out ACs with attached endpoints. The loop detection frames have the same source MAC address (MAC address of the M-LAG system).

·     An M-LAG member device synchronizes the loop detection frames received from the M-LAG interface to the other M-LAG member device through the peer link. An M-LAG member device does not synchronize the loop detection frames received from a non-M-LAG interface through the peer link.

Mechanism for determining the occurrence of a loop

After loop detection is enabled, an M-LAG member device determines that a loop has occurred on an AC when the following conditions exist:

·     It receives a loop detection frames on an AC outside the peer link.

·     The VLAN tag of the loop detection frame is the same as the VLAN tag of the loop detection frame sent by the AC.

When an M-LAG member device with loop detection enabled receives synchronized loop detection frames through the peer link, the local member device determines that a loop also has occurred on the local AC in the same VXLAN as the loop detection frames in the same M-LAG group. For example, VTEP 1 receives a loop detection frame on an AC on the BAGG interface and synchronizes the frame to VTEP 2 through the peer link. Even if VTEP 2 does not receive a loop detection frame on an AC on the BAGG interface, VTEP 2 still determines that a loop has occurred on the local AC in the same VXLAN as the AC that receives the loop detection frame on VTEP 1.

Processing mechanism for detected loops

Suppose an M-LAG member device receives a loop detection frame on an AC or peer link and determines that a loop has occurred on the AC. In this case, the M-LAG member device processes the loop based on the priority of the received loop detection frame:

·     If the received loop detection frame has a higher priority, the M-LAG member device will block the AC where the loop has occurred according to the configuration.

·     If the received loop detection frame has a lower priority, the M-LAG member device will not trigger the loop processing action for the AC where the loop has occurred.

Support of DHCP/DHCPv6 for M-LAG

·     M-LAG is mainly used in the DHCP snooping scenario. The operating mechanism of DHCP relay M-LAG is basically the same as that of DHCP snooping M-LAG. This chapter mainly introduces the operating mechanism of DHCP snooping M-LAG.

·     In this chapter, DHCP snooping refers to both DHCPv4 snooping and DHCPv6 snooping.

Overview

M-LAG can virtualize two DHCP snooping devices into one system through multichassis link aggregation, improving the availability of DHCP snooping devices and facilitating device-level redundancy and load sharing.

Operating mechanism

M-LAG system setup

DHCP snooping devices are members in the M-LAG system. For more information about M-LAG system establishment and operation, see “M-LAG system setup and operating mechanisms.”

User information entry sync mechanism

Real-time backup

When a DHCP user goes online, renews its lease, or goes offline, one DHCP snooping device generates, updates or deletes the related user entry, and then backs up the data in real time to another DHCP snooping device through the peer link.

Batch backup

When the state of a peer-link interface changes from DOWN to UP, batch backup is triggered. The primary M-LAG device sends local user entries to the secondary M-LAG device for backup purposes. The secondary device also sends user entries that the primary device does not have to the primary device. Only entries in the union set of both devices will be saved. During batch backup, real-time backup will be delayed.

Traffic forwarding

DHCP snooping M-LAG has the following application scenarios:

Bidirectional M-LAG on both the uplink and downlink directions

Figure 35 Network diagram (The uplink M-LAG interface is UP on the local device)

 

Figure 36 Network diagram (The uplink M-LAG interface is DOWN on the local device)

 

As shown in Figure 35 and Figure 36, DHCP requests are sent to M-LAG device 1 (local device).

1.     After receiving a DHCP request from the DHCP client, the local device generates a temporary MAC address entry for the DHCP client. The access interface in the entry is M-LAG interface 1.

2.     The local device synchronizes the received DHCP request to the peer device through the peer link. The peer device also generates a temporary MAC address entry for the DHCP client. The access interface in the entry is the peer-link interface.

3.     Forwarding of the DHCP request depends on the uplink interface of the local device as follows:

¡     As shown in Figure 35, if the uplink M-LAG interface of the local device is UP, the local device will forward the DHCP request to the DHCP server. Enabled by the peer link, the peer device perceives that the uplink M-LAG interface of the local device is UP through the peer link, and then directly discards the DHCP request.

¡     As shown in Figure 36, when the uplink M-LAG interface of the local device is DOWN, the peer device can detect this event through the peer link. Then, the peer device will forward the DHCP request to the DHCP server.

4.     After processing the DHCP request, the DHCP server returns a reply.

¡     If the local device receives the reply, it will looks up the MAC address forwarding entry for the DHCP client. After finding that the access interface is M-LAG interface 1, it directly forwards the reply to the DHCP client without synchronizing it to the peer device. Meanwhile, the local device generates a DHCP snooping entry for the DHCP client, and then backs it up in real time to the peer device through the peer link.

¡     If the peer device receives the reply, it will synchronize the reply to the local device through the peer link. The peer device drops a DHCP reply if the following conditions exist:

-     The reply is received on the uplink M-LAG interface.

-     The access interface in the MAC address forwarding entry for the DHCP client is the peer-link interface.

The local device will forward the reply to the DHCP client. Likewise, it will generate a DHCP snooping entry for the DHCP client, and then back up the entry in real time to the peer device.

Unidirectional M-LAG on only the downlink direction

Figure 37 Unidirectional M-LAG on only the downlink direction

 

As shown in Figure 37, this M-LAG application scenario is different from the bidirectional M-LAG scenario as follows:

·     In the bidirectional M-LAG scenario, the uplink interfaces of M-LAG devices are all M-LAG interfaces. In the unidirectional downlink M-LAG scenario, the uplink interfaces of M-LAG devices are all common interfaces.

·     In the bidirectional M-LAG scenario, the uplink interfaces of M-LAG devices can be in UP state simultaneously. However, in the unidirectional downlink M-LAG scenario, the spanning tree protocol blocks the uplink interface of an M-LAG device to avoid loops.

·     After receiving a DHCP reply from the DHCP server, an M-LAG device forwards it to the peer device through the DHCP snooping module rather than the peer link.

The local device forwards traffic as follows:

·     If the uplink interface of the peer device is blocked by the spanning tree protocol, traffic forwarding is the same as Figure 35.

·     If the uplink interface of the local device is blocked by the spanning tree protocol, the traffic forwarding paths are the same as Figure 36. The difference is that the input interface is a common interface. After receiving a DHCP reply destined for a DHCP client attached to the local device, the peer device processes the reply instead of discarding it. However, since the DHCP client-facing interface for the peer-link interface, the peer device cannot forward the reply to the DHCP client. In this situation, the peer device synchronizes the reply to the local device for further forwarding to the DHCP client.

Unidirectional M-LAG on only the uplink direction

Figure 38 Unidirectional M-LAG on only the uplink direction

 

As shown in Figure 38, this M-LAG application scenario is different from the bidirectional M-LAG scenario as follows:

·     In the bidirectional M-LAG scenario, the downlink interfaces of M-LAG devices are all M-LAG interfaces. In the unidirectional uplink M-LAG scenario, the downlink interfaces of M-LAG devices are all common interfaces.

·     In the unidirectional uplink M-LAG scenario, the local device generates a temporary MAC address entry upon receiving a DHCP request. The access interface in the entry is a common interface.

·     In the bidirectional M-LAG scenario, the downlink interfaces are all M-LAG interfaces. The DHCP client sends a DHCP request to a random M-LAG device. In the unidirectional uplink M-LAG scenario, restricted by the spanning tree protocol, the DHCP client sends DHCP requests to only one M-LAG device.

·     After receiving a DHCP request, an M-LAG device forwards it to the peer device through the DHCP snooping module rather than the peer link.

Traffic forwarding in the unidirectional uplink M-LAG scenario is the same as that in the bidirectional M-LAG scenario.

M-LAG support in security mechanisms

Introduction

When a device is operating normally, users can initiate authentication through M-LAG interfaces. For the same user, authentication and accounting actions are executed on only one M-LAG member device, while authorization actions are executed on both M-LAG member devices. Users authorized on one M-LAG interface can access network resources through an M-LAG interface on any M-LAG member device. When one of the M-LAG member devices fails, users who were originally authenticated and accounted on that device will switch to another M-LAG member device to continue interacting with the server for exchanging accounting packets and other information. Thus, the users can maintain their online status and continue accessing network resources. Therefore, in an M-LAG scenario, device-level load sharing and redundancy backup can be achieved for security services such as port security and portal authentication.

The term "port security users” in this document collectively refers to 802.1X users, MAC authentication users, Web authentication users, and static users.

Operating mechanism

Port security support for M-LAG

Typical network

The following diagram is the typical network model for M-LAG support for wired 802.1X users, MAC authentication users, Web authentication users, and static users.

Figure 39 Port security support for M-LAG network diagram

 

 

Configuration consistency check

The M-LAG system will perform a Type 2 configuration consistency check on port security services.

·     If the consistency check fails, the system discards the user packets that trigger authentication.

·     If the port security configuration changes from being consistent to inconsistent, existing online users will remain online, but new users will not be allowed to come online.

·     If the peer-link recovers from a fault and the M-LAG system detects inconsistent port security configurations, it will force the currently online users to go offline.

·     In the case of allowing single-sided access through M-LAG interfaces, consistency checks will not be performed for the port security services on the M-LAG interfaces. When the situation is changed from single-sided M-LAG interface access to M-LAG interface access on both M-LAG member devices, the M-LAG system will detect inconsistent port security configurations. This will force the previously online users on the single-side M-LAG interfaces to go offline.

 

NOTE: Single-sided M-LAG interface access refers to the situation where only one device has configured with M-LAG interfaces, and no M-LAG interface is configured on the peer device.

 

Forwarding and distribution of user packets

When receiving user packets, the access device forwards the packets to the M-LAG interface of a specific M-LAG member device for processing, based on the load sharing mode configured on the aggregation interface of the access device.

Upon receiving a packet with unknown source MAC address or an 802.1X EAPOL protocol packet from a user, the M-LAG member device creates a user entry locally. It processes the packet locally or notifies the peer M-LAG member device for processing, depending on the load sharing mode of user authentication configured on the M-LAG interface by the port security module.

·     In centralized processing mode:

¡     If the primary device receives a user packet, the primary device processes the packet locally.

¡     If the secondary device receives a user packet, it first parses the packet and then notifies the primary device for subsequent processing. The primary device actively interacts with the AAA server and client for authentication-related packet exchange.

In this mode, the configuration is relatively simple. On the RADIUS server, only one access device IP is required. The two M-LAG member devices only need to be configured with the same RADIUS packet source IP address. However, this mode has low packet distribution efficiency, and is suitable for scenarios with a small number of access users.

·     In distributed processing mode:

¡     Local mode: User packets received on a local M-LAG interface are processed by the local M-LAG member device.

¡     Parity mode: When the primary device receives user packets, it resolves the source MAC addresses of the packets. Based on the port security configuration, packets with odd MACs are processed on one M-LAG member device, and packets with even MACs are processed on the other M-LAG member device. If a packet needs to be processed on the peer device, the local device will transparently transmit the packet to the peer device.

In distributed processing mode, the packet distribution efficiency is high. Two access device IPs need to be managed on the RADIUS server, and the RADIUS packet source IP addresses used by the local and peer devices need to be configured on both M-LAG member devices. This mode is suitable for scenarios with a large number of access users. The local mode has higher distribution efficiency than the parity mode.

 

NOTE: When single M-LAG interface access is allowed: ·     In centralized processing mode, the M-LAG interface discards the authentication packets of users. ·     In distributed processing mode, packets are processed in local mode regardless of whether the local mode or parity mode is configured. If the load sharing mode of user authentication is changed on the M-LAG interface of any M-LAG member device, all users on all M-LAG interfaces will be forced offline. When a peer-link interface encounters a fault, all M-LAG interfaces on the secondary M-LAG member device will enter MAD DOWN state, and the user entries on the interfaces will be deleted. At this time, only the primary M-LAG member device will process user packets.

 

User authentication, authorization, and accounting

1.     Assuming that a user packet is distributed to M-LAG 1 for processing, the user entry on M-LAG 1 will be set to an active state. M-LAG 1 will then interact with the RADIUS server for the user's authentication, authorization, and accounting.

2.     After a user passes authentication on M-LAG 1, M-LAG 1 executes local authorization and simultaneously syncs user data containing authorization information to the peer M-LAG member device. After the peer M-LAG member device synchronizes the user data, it also carries out local authorization. This allows the user to access authorized network resources, whether through the M-LAG interface on M-LAG 1 or the M-LAG interface on M-LAG 2.

3.     After the authorization succeeds at both M-LAG member devices, M-LAG 1 sends an authentication success packet to the client and an accounting start packet to the server. The user on M-LAG 2 is a backup user and therefore does not need to interact with the server.

User entry synchronization

The procedure for creating a user entry and the real-time backup process are as follows:

1.     After the local user authentication is successful, the local M-LAG member device will synchronize user information (including source MAC address, VLAN, and authorization information) to the peer M-LAG member device in real time.

2.     The peer M-LAG member device creates a user entry based on synchronization information and issues authorization information.

 

NOTE: For Web authentication, both devices must issue the same redirect URL rule to ensure that the user's HTTP packets can be redirected to the portal Web server when coming from either device.

 

The procedure for deleting a user entry is as follows:

3.     When a local user goes offline, the local M-LAG member device notifies the peer M-LAG member device to synchronize the offline processing for the user. This includes deleting the user entry, revoking user authorization, and counting the total traffic of the user. The total user traffic is exchanged along with the offline notification message.

4.     The M-LAG member device where the user is in active state sends an accounting stop packet to the RADIUS server. The user traffic data carried in the packet is the cumulative total user traffic data from the two M-LAG member devices.

5.     For 802.1X users, the M-LAG member device where the user state is active needs to notify the 802.1X client to go offline.

 

NOTE: When both M-LAG member devices have enabled with the offline detection feature for users of the corresponding authentication type, a user will be triggered to go offline and the user entry will be deleted only when both M-LAG member devices have not detected the user’s traffic.

 

When the peer-link is up, the port security process restarts, or a single M-LAG member device undergoes an active/standby switchover, the two M-LAG member devices will synchronize their user data with each other in bulk. After receiving data from the peer device, both devices will save the user entries based on the data of users in active state.

Traffic statistics and data synchronization

Since the same user entries exist on both M-LAG member devices and a single user's traffic may be distributed across both devices, it is necessary to synchronize the traffic statistics of a single user between the two devices.

·     Both M-LAG member devices start the traffic statistics function after user authorization is successful, and periodically (default 60 seconds) send the user's traffic statistics data to the peer end. Upon receiving data from the peer, the devices calculate the user's total traffic.

·     If the user on the current M-LAG member device is in Active state, this M-LAG member device will send the cumulative total traffic of the user to the RADIUS server through accounting update packets. Also, this M-LAG member device will send an accounting stop packet when the user goes offline.

Port migration

Port security supports user migration between two M-LAG member devices or on a single M-LAG member device under the following circumstances:

·     User migration occurs between M-LAG interfaces.

·     User migration between M-LAG and non-M-LAG interfaces.

·     User migration between non-M-LAG interfaces.

 

NOTE: If user migration is required between non-M-LAG interfaces on two M-LAG member devices, you need to configure both devices to allow synchronized remote MAC entries to override existing local MAC entries.

 

User state switching

System state	Load sharing mode for user authentication	User state
Normal	Centralized processing mode	·     All users on the primary device are in active state. ·     All users on the secondary device are in inactive state.
	Distributed processing mode	·     The users on the local device are in active state. ·     The users synchronized from the peer device are in inactive state.
Peer-link interface failed	Centralized processing mode	·     All users on the primary device are in active state. ·     All users on the secondary device are in inactive state.
	Distributed processing mode	·     All users on the primary device are in active state. ·     User entries on the secondary device are deleted.

 

Portal support for M-LAG

Typical network

In this network scenario, if a portal user passes authentication on the local M-LAG member device, the device will transmit the user data to the peer M-LAG member device for backup. When one M-LAG member device fails, the peer M-LAG member device can take over service processing using the backup user data, ensuring the normal operation of user services.

Figure 40 Portal support for M-LAG network diagram

 

Configuration consistency check

After the portal service starts, the system performs a consistency check on the portal configurations on the VLAN interfaces to which the M-LAG interfaces belong.

·     If the consistency check fails, the system discards the user packets that trigger authentication.

·     If the portal configuration changes from being consistent to inconsistent, existing online users will remain online, but new users are not allowed to come online.

·     If the peer-link recovers from a fault and the M-LAG system detects inconsistent portal configurations, existing online users will remain online, but new users are not allowed to come online.

 

NOTE: Portal users cannot come online from a single-homed device.

 

Forwarding and distribution of user packets

When receiving user packets, the access device forwards the packets to the M-LAG interface of a specific M-LAG member device for processing, based on the load sharing mode configured on the aggregation interface of the access device.

Processing of HTTP redirect packets

An M-LAG member device redirects user's HTTP/HTTPS packets based on the following rules:

1.     If the primary or secondary device receives the first HTTP/HTTPS packet, it records the 5-tuple information of the packet.

2.     When the next HTTP/HTTPS packet arrives at the primary or secondary device, the device checks if it has a record for the first HTTP/HTTPS packet. If yes, the device redirects the packet. If not, it discards the HTTP/HTTPS packet. If a non-fragmented packet or the last HTTP fragmented packet is received, the device will also try to redirect the packet.

 

NOTE: If the same HTTP/HTTPS flow (packets with the same 5-tuple info) from a user is sent to different M-LAG member devices, the HTTP redirect processing for that user might fail.

 

Processing of portal protocol packets

After the M-LAG member device completes the redirect processing of the user's HTTP packet, the portal server sends an authentication request to the M-LAG member device.

·     In centralized processing mode:

¡     If the primary device receives a portal protocol packet, it will process it locally and create a user entry.

¡     If the secondary device receives a portal protocol packet, it first resolves the packet, and then transparently forwards the packet to the primary device for subsequent portal service processing. The primary device then responds to the portal server.

In this mode, the configuration is relatively simple. On the RADIUS server, only one access device IP is required. The two M-LAG member devices only need to be configured with the same RADIUS packet source IP address. However, this mode has low packet distribution efficiency, and is suitable for scenarios with a small number of access users.

·     In distributed processing mode:

¡     M-LAG member devices perform load sharing based on the user IP address information carried in the portal protocol packets.

¡     When an M-LAG member device receives portal protocol packets, it resolves the user source IP in the packets. Based on the portal configuration on the device, packets with odd IPs are processed on one M-LAG member device, and packets with even IPs are processed on the other M-LAG member device. If a received packet is required to be processed on the peer device, the local device transparently transmits the packet to the peer device for portal service processing, and the peer device responds to the portal server. Otherwise, the local device performs portal processing for the packet and creates a user entry.

In distributed processing mode, the packet distribution efficiency is high. Two access device IPs need to be managed on the RADIUS server, and the RADIUS packet source IP addresses used by the local and peer devices need to be configured on both M-LAG member devices. This mode is suitable for scenarios with a large number of access users.

User authentication, authorization, and accounting

1.     Assume that a portal protocol packet is distributed to M-LAG 1 for processing. M-LAG 1 will then interact with the RADIUS server for the user's authentication, authorization, and accounting.

2.     After the user passes authentication on M-LAG 1, the user entry will be set to active state. M-LAG 1 will then send an authentication success packet to the portal server.

3.     M-LAG 1 uses the authorization information issued by the RADIUS server to authorize the user locally, and simultaneously synchronizes the user data containing the authorization information to the peer M-LAG member device. After the peer M-LAG member device synchronizes the user data, it also carries out local authorization. This allows the user to access authorized network resources, whether through the M-LAG interface on M-LAG 1 or the M-LAG interface on M-LAG 2.

4.     After the authorization succeeds on both M-LAG member devices, M-LAG 1 sends an accounting start packet to the RADIUS server. The user on M-LAG 2 is a backup user and therefore does not need to interact with the server.

User entry synchronization

The procedure for creating a user entry and the real-time backup process are as follows:

1.     After the local user authentication is successful, the local M-LAG member device will synchronize user information (including user IP address, source MAC address, VLAN, and authorization information) to the peer M-LAG member device in real time.

2.     The peer M-LAG member device creates a user entry based on synchronization information and issues authorization information.

The procedure for deleting a user entry is as follows:

3.     If the user proactively goes offline, the portal server will notify the M-LAG member device to process the user's offline status. The M-LAG system will distribute the offline request packet from the portal server according to the M-LAG processing mode configured for the portal service. The M-LAG member device where the user state is active processes the user's offline status and notifies the peer M-LAG member device to synchronize the process. This includes deleting the user entry, revoking user authorization, and counting the total traffic of the user.

4.     If you force a user offline on an M-LAG member device, the M-LAG member device where the user state is active sends an offline notification packet to the portal server and notifies the peer M-LAG member device to synchronize the processing, including deleting the user entry, revoking the user's authorization, and calculating the total traffic of the user.

5.     The M-LAG member device where the user state is active sends an accounting stop packet to the RADIUS server. The user traffic data carried in the packet is the cumulative total user traffic data from the two M-LAG member devices.

When the peer-link is up, the port security process restarts, a single M-LAG active/standby switchover occurs, or the portal configuration changes from being inconsistent to consistent, the two M-LAG member devices will synchronize their user data in bulk. After receiving the data from the peer, both devices will save the user entries based on the data of users in active state.

Traffic statistics and data synchronization

Since both M-LAG member devices have issued the same portal rules, and the traffic of the same user might be distributed to both devices, the traffic statistics of the same user on both devices needs to be synchronized (including ITA user traffic).

·     When the size of the traffic received on an M-LAG member device differs from the size of the previously backed up traffic by a set threshold, or when the portal user traffic backup interval has been reached, the device backs up the user traffic to the peer M-LAG member device.

·     If the user on the current M-LAG member device is in Active state, this M-LAG member device will send the cumulative total traffic of the user to the RADIUS server through accounting update packets. Also, this M-LAG member device will send an accounting stop packet when the user goes offline.

User state switching

System state	Load sharing mode for user authentication	User state
Normal	Centralized processing mode	·     All users on the primary device are in active state. ·     All users on the secondary device are in inactive state.
	Distributed processing mode	·     The users on the local device are in active state. ·     The users synchronized from the peer device are in inactive state.
Peer-link interface failed	Centralized processing mode	·     All users on the primary device are in active state. ·     All users on the secondary device are in inactive state.
	Distributed processing mode	·     All users on the primary device are in active state. ·     User entries on the secondary device remain unchanged.

 

Processing of RADIUS protocol packets

The authentication server needs to identify a user’s identity based on the NAS-IP. To ensure normal exchanges for processing of backup user services, use the IP address of the peer M-LAG member device as the source IP address of packets. After an M-LAG virtual IP (VIP) address is configured on two M-LAG member device, a member device exchanges local user information with the RADIUS server by using its own M-LAG VIP address when it is operating normally. If one member device fails, the other member device will use the peer's M-LAG VIP address to exchange the peer’s backup user information with the RADIUS server.

Authentication, authorization, and accounting packets (with the device acting as a RADIUS client)

General processing principle: After a user comes online, the RADIUS packet source IP address used for interaction with the RADIUS server remains unchanged.

·     In centralized processing mode, the same local source IP address needs to be configured on both M-LAG member devices. All authentication, authorization, and accounting packets are processed on the primary device. The source IP address must be the same M-LAG VIP address.

¡     When the primary device is operating normally, it exchanges the RADIUS protocol packets with the server by using the local source IP address.

¡     When the primary device fails, the secondary device exchanges RADIUS protocol packets with the server by using the local source IP address.

·     In distributed processing mode, you need to configure a local source IP address and a peer source IP address on both M-LAG member devices. That is, two different M-LAG VIP addresses are configured on each of the two devices, and these two pairs of addresses are opposite to each other on the two devices.

Assume that on M-LAG 1, the local source IP address is A and the peer source IP address is B, and on M-LAG 2, the local source IP address is B and the peer source IP address is A.

¡     When both M-LAG member devices are operating normally, M-LAG 1 uses address A to exchange RADIUS protocol packets with the server, and M-LAG 2 uses address B to do the same.

¡     When M-LAG 1 fails, M-LAG 2 will use its peer source IP address A to exchange RADIUS packets for the users originally on M-LAG 1.

 

NOTE: In an M-LAG network environment, the source IP address specified on an M-LAG member device for transmitting RADIUS packets must be the M-LAG VIP address. The Loopback interfaces of the two M-LAG member devices are configured with different virtual IP (VIP) addresses, and both are set to the active state.

 

COA packets (with the device acting as a RADIUS server)

For port security services, the processing mechanism for COA packets is as follows:

·     If the primary device receives a COA protocol packet, the primary device processes the packet locally.

·     If the secondary device receives a COA protocol packet, the secondary device process the packet locally.

For portal services, the processing mechanism for COA packets is as follows:

·     If the user state on the M-LAG member device that receives the COA protocol packet is active, this member device processes the packet locally.

·     If the user state on the M-LAG member device that receives the COA protocol packet is inactive, this member device will transparently transmits the packet to the peer member device for processing.

 

Layer 2 multicast support for M-LAG

Introduction

Layer 2 multicast uses multichassis link aggregation (M-LAG) to virtualize two physical devices into one system, which connects to a remote aggregation system and then to a multicast source or multicast receivers. This avoids multicast interruption caused by a single point of failure and improves the reliability of the multicast network.

Operating mechanisms

NOTE: The peer-link interface acts as a static router port to transport all multicast traffic.

 

Multicast source dual-homed to M-LAG

Establishment of Layer 2 multicast forwarding entries

As shown in Figure 41, the peer link of the M-LAG system is established between M-LAG member devices (Device A and Device B), which are connected to the remote aggregation system (Device C). Device C is connected to the multicast source. The M-LAG system is connected to a Layer 3 network, and STP is configured  to avoid Layer 2 loops by blocking ports. This ensures that the multicast traffic transmitted from the multicast source to the M-LAG system can only be forwarded by either Device A or Device B to the multicast receiver. In this example, the link connecting Device B and the Layer 2 network is blocked.

Figure 41 Establishment of Layer 2 multicast forwarding entries

 

The following describes the establishment of Layer 2 multicast forwarding entries:

1.     The IGMP/MLD query messages transmitted by Device C is load-shared via the aggregation interface of Device C, reaching the M-LAG interfaces of Device A and Device B, respectively.

2.     Device A and Device B each add their respective M-LAG interfaces to the router port list and transmit the information to each other through the peer link. Device A and Device B will not forward the query messages received from the peer-link interface to their respective M-LAG interfaces.

3.     After Device A receives an IGMP/MLD report messages from a downstream receiver, it resolves the multicast group address G that the host wants to join from the report message, generating a layer 2 multicast forwarding entry (*, G), and adds the receiving port as a member port to the output port list. At the same time, it sends the report message to Device B through the peer link.

4.     After Device B receives report message, it also generates a layer 2 multicast forwarding  entry (*, G), and adds the peer-link interface to the router port list. However, Device B will not forward this message from its router port (M-LAG interface).

The above mechanism enables Device A and Device B to generate the same Layer 2 multicast forwarding entry, forming a device-level backup. When a member device experiences a fault (device fault, uplink/downlink link fault, and so on), the multicast traffic can be forwarded by another member device, avoiding disruption in multicast traffic forwarding.

Multicast traffic forwarding process

The multicast traffic forwarding process is as follows:

1.     The multicast traffic transmitted by the multicast source reaches Device A and Device B through load sharing.

2.     Device A and Device B transmit the multicast data traffic they receive to each other through the peer link. This ensures that both Device A and Device B can receive the complete multicast data flow.

3.     Device A will send the complete multicast data flow to the downstream receiver.

Figure 42 Multicast traffic forwarding process

 

Multicast receiver dual-homed to M-LAG

As shown in Figure 43, the peer link of the M-LAG system is established between M-LAG member devices (Device A and Device B), which are connected to the remote aggregation system (Device C). Device C is connected to the multicast receiver. The M-LAG system is connected to a Layer 3 network, and STP is configured  to avoid Layer 2 loops by blocking ports. This ensures that the multicast traffic transmitted from the multicast source to the M-LAG system can only be forwarded by either Device A or Device B to the multicast receiver. In this example, the link connecting Device B and the Layer 2 network is blocked.

Figure 43 Establishment of Layer 2 multicast forwarding entries

 

The following describes the establishment of Layer 2 multicast forwarding entries:

1.     After Device A receives an IGMP/MLD query message from the upstream Layer 2 network, it adds the port that received the message to the dynamic router port list, and then transmits the message to Device B via the peer link.

2.     After Device B receives the query message, it adds the peer-link interface to the router port list.

3.     After Device A or Device B receives an IGMP/MLD report message from the downstream receiver (Device C), it resolves the multicast group addresses G1 and G2 that the host wants to join from the report message, and generates a layer 2 multicast forwarding entry (*, G1) and (*, G2), with the output ports as their respective M-LAG interfaces.

 

NOTE: Assume that Device A receives a report message to join multicast group G1, and Device B receives a report message to join multicast group G2.

 

4.     Device A and Device B transmit the their report messages to each other through the peer link. Take Device A as an example. After receiving the report message, Device A generates a Layer 2 multicast forwarding entry (*, G2), and adds its M-LAG interface to the router port list. Similarly, Device B generates a Layer 2 multicast forwarding entry (*, G1).

The above mechanism enables Device A and Device B to generate the same Layer 2 multicast forwarding entry, forming a device-level backup. When a member device experiences a fault (device fault, uplink/downlink link fault, and so on), the multicast traffic can be forwarded by another member device, avoiding disruption in multicast traffic forwarding.

Multicast traffic forwarding process

The multicast traffic forwarding process is as follows:

1.     The multicast traffic transmitted by the multicast source reaches Device A through the Layer 2 network.

2.     Device A transmits the multicast data traffic it receives to Device B through the peer link. Only Device A will forward the multicast traffic to the downstream device Device C. Although Device B generates a Layer 2 multicast forwarding entry, it will not forward the multicast traffic to the downstream device.

3.     After Device C receives multicast traffic, it will forward the traffic to the receiver.

Figure 44 Multicast traffic forwarding process

 

 

Layer 3 multicast support for M-LAG

Introduction

Layer 3 multicast uses M-LAG to virtualize two physical devices into one system, which connects to a remote aggregation system and then to a multicast source or multicast receivers. This avoids multicast interruption caused by a single point of failure and improves the reliability of the multicast network.

Operating mechanisms

NOTE: An independent Layer 3 link needs to be configured between M-LAG member devices to establish PIM neighbors, and it is used as the keepalive link for the M-LAG system..

 

Multicast source dual-homed to M-LAG

Establishment of Layer 3 multicast forwarding entries

As shown in Figure 45, the peer link of the M-LAG system is established between M-LAG member devices (Device A and Device B), which are connected to the remote aggregation system (Device C). Device C is connected to the multicast source. The peer link and the M-LAG interface belong to the same VLAN. The receiver is on the Layer 3 network side, and will select an optimal path to the multicast source on the Layer 3 network. In this example, upstream device A is selected.

Figure 45 Establishment of Layer 3 multicast forwarding entries

 

The following describes the establishment of Layer 3 multicast forwarding entries:

1.     Upon receiving a PIM/IPv6 PIM join message, Device A will not synchronize the message to Device B via the peer link. Instead, after obtaining the receiver's information of the multicast group, it generates an (*,G) entry.

2.     The multicast data packets transmitted from the multicast source to the receiver will reach the M-LAG interfaces of Device A and Device B through load sharing. Device A and Device B transmit the multicast data traffic they receive to each other through the peer link, enabling both devices to receive the complete multicast traffic. Device A establishes an (S, G) entry based on the received multicast data traffic.

Multicast traffic forwarding process

The multicast traffic forwarding process is as follows:

1.     The multicast traffic transmitted by the multicast source reaches Device A and Device B through load sharing.

2.     Device A and Device B transmit the multicast data traffic they receive to other through the peer link. This ensures that both Device A and Device B can receive the complete multicast data flow.

3.     Device A will send the complete multicast data flow to the downstream receiver.

Figure 46 Multicast traffic forwarding process

 

Multicast receiver dual-homed to M-LAG

As shown in Figure 47, the peer link of the M-LAG system is established between M-LAG member devices (Device A and Device B), which are connected to the remote aggregation system (Device C). Device C is connected to the multicast receiver. Configure the PIM/IPv6 PIM passive mode on the M-LAG interfaces of Device A and Device B that are connected to Device C to ensure that both Device A and Device B can receive all multicast traffic sent by the multicast source.

Figure 47 Establishment of Layer 3 multicast forwarding entries

 

1.     Device A or Device B receives an IGMP/MLD report message from the downstream receiver (Device C). Device A or Device B resolves the multicast group address G that the host wants to join from the report message, generating a Layer 3 multicast forwarding entries (*, G1) and  (*, G2), and adds the M-LAG interfaces to the output port list.

2.     Device A and Device B transmit the their report messages to each other through the peer link. Take Device A as an example. After receiving the report message, Device A generates a Layer 3 multicast forwarding entry (*, G2), and adds its M-LAG interface to the router port list. Similarly, Device B generates a multicast forwarding entry (*, G1).

3.     The multicast group information on Device A and Device B will remain synchronized, and the same multicast forwarding  entries  (*, G1) and (*, G2) are generated on them.

4.     Based on the PIM mode configured, Device A and Device B generate identical PIM routing entries. In this example, the PIM-SM mode is configured.

The above mechanism enables Device A and Device B to generate the same PIM routing entries, forming a device-level backup. When a member device experiences a fault (device fault, uplink/downlink link fault, and so on), the multicast traffic can be forwarded by another member device, avoiding disruption in multicast traffic forwarding.

Multicast traffic forwarding process

The multicast traffic forwarding process is as follows:

1.     The multicast traffic transmitted by the multicast source reaches Device A and Device B through the Layer 3 network.

2.     Device A and Device B forward multicast traffic to downstream receivers according to the odd/even rule. The M-LAG member with an odd M-LAG system number forwards multicast data destined for odd multicast group addresses. The M-LAG member with an even M-LAG system number forwards multicast data destined for even multicast group addresses

Figure 48 Multicast traffic forwarding process

 

 

EVPN VXLAN support for M-LAG

Overview

Ethernet Virtual Private Network (EVPN) VXLAN adopts the M-LAG technology to connect two physical devices, and virtualize them into one device. This virtual device is used as a VTEP (either for Layer 2 forwarding or as an EVPN gateway). This avoids the impact of a single point failure of VTEP in the network, enhancing the reliability of the EVPN network.

 

IMPORTANT: In the current software version, the site and underlay networks for this feature can only be IPv4 networks or IPv6 networks.

 

Typical networking

Figure 49 shows the typical networking of EVPN VXLAN that supports M-LAG. In this network:

·     Two VTEPs form an M-LAG system. They both have the same virtual VTEP address, and act as a single virtual VTEP device externally.

·     The peer-link connection between M-LAG devices can be either an Ethernet aggregate link or a VXLAN tunnel. When the Ethernet link is used as the peer-link connection, the network is known as a network with peer link. When the VXLAN tunnel is used as the peer-link connection, the network is known as a network without peer link. The VXLAN tunnel used as the peer-link automatically associates with all VXLANs on the device.

·     Server 2 and Server 3 connect to the VTEP by using M-LAG.

·     Server 1 and Server 4 are each attached to the VTEP through a single link.

Figure 49 Network diagram

 

Synchronizing the MAC address and ARP/ND information

The two VTEPs in the M-LAG system are connected through a peer link to synchronize MAC addresses, ARP/ND entries, and ARP/ND flooding suppression entries, ensuring consistency of the MAC address and ARP/ND information on both VTEPs. If one VTEP fails, the other VTEP can immediately take over to forward traffic.

The entries of the M-LAG interface are synced to the peer M-LAG device through the peer link, and then the peer M-LAG device adds these entries to its corresponding local M-LAG interface. The entries of the single-attached interface are also synced to the peer M-LAG device through the peer link, and the peer M-LAG device adds these entries to its peer-link interface.

Establishing a VXLAN tunnel

In the EVPN VXLAN network, the VTEPs can establish a VXLAN tunnel based on BGP EVPN (Inclusive Multicast Ethernet Tag) IMET routes, MAC/IP advertisement routes, and IP prefix routes.

User link backup

On the user side, the two VTEP devices are both connected to the same VM through Ethernet links. The two links between the devices form a Layer 2 aggregate link. The aggregate interface is configured as an AC (by creating an Ethernet service instance on the aggregate interface, setting up packet match rules, and associating the Ethernet service instance with VSI) to prevent a single Ethernet link fault from blocking network access of the VM.

User link backup mechanism with an Ethernet aggregate link as the peer link

When the peer link is an Ethernet aggregate link, the VTEPs perform user link backup by automatically creating ACs or automatically setting up a VXLAN tunnel on the peer link.

·     Automatically create an AC on the peer link

On the peer link, the M-LAG device will automatically create an AC based on the user-side AC or the VXLAN ID to which the user belongs.

The process for user link backup through automatic AC creation is as follows: When an AC on a VTEP fails, the packet sent to this AC through the VXLAN tunnel is then forwarded through the peer link to another VTEP. The VTEP obtains the VSI of the packet based on the AC configured for the peer link, and then forwards the packet, thus ensuring service continuity.

·     Automatically create a VXLAN tunnel

The two VTEPs acting as M-LAG devices automatically establish a VXLAN tunnel between them. This VXLAN tunnel is automatically associated with all VXLANs.

The process for user link backup through automatic VXLAN tunnel creation is as follows: If an AC on a VTEP fails, the VTEP adds VXLAN encapsulation to the packet received from the remote VTEP (non-M-LAG device) sent to the faulty AC through the VXLAN tunnel. The encapsulated VXLAN ID corresponds to the VXLAN ID of the VSI to which the faulty AC belongs, and the packet is then forwarded to another VTEP (M-LAG device) through the automatically created VXLAN tunnel. This VTEP obtains the VSI to which the packet belongs based on the VXLAN ID, and forwards the packet.

User link backup mechanism with a VXLAN tunnel as the peer link

When the peer link is a VXLAN tunnel, the user link backup mechanism is as follows: If an AC fault occurs on a VTEP, this VTEP adds VXLAN encapsulation to the packet received from the VXLAN tunnel destined for the faulty AC. The encapsulated VXLAN ID corresponds to the VXLAN ID of the VSI to which the faulty AC belongs. The packet is then forwarded to another VTEP through the VXLAN tunnel acting as the peer link. This VTEP obtains the VSI to which the packet belongs based on the VXLAN ID and forwards the packet accordingly.

Traffic forwarding

Unicast traffic forwarding through M-LAG interfaces

Uplink unicast traffic of M-LAG interfaces

As shown in Figure 50, the access server (Server 2) is connected to the VTEPs in an M-LAG system through aggregate interfaces. The uplink unicast traffic destined for the external network is load shared between multiple M-LAG devices (VTEP 1 and VTEP 2). Upon receiving the unicast traffic, the M-LAG devices forward the unicast traffic to the remote VTEP through the local VXLAN tunnel according to the local-first forwarding principle.

Figure 50 Uplink unicast traffic forwarding through M-LAG interfaces

 

Downlink unicast traffic of M-LAG interfaces

As shown in Figure 51, the access server (Server 2) is connected to the VTEPs in an M-LAG system through aggregation interfaces. When the external network sends unicast traffic to Server 2, the traffic is forwarded through the VXLAN tunnel to M-LAG devices (VTEP 1 and VTEP 2). In the underlay network, all M-LAG devices advertise routes to the virtual VTEP addresses, thus forming equal-cost routes on VTEP 3. This allows the unicast traffic sent to Server 2 from the external network to be load-shared across multiple M-LAG devices.

After an M-LAG device receives the downlink unicast traffic, it forwards the unicast traffic to the access server (Server 2) through the local AC by using the local-first forwarding principle.

Figure 51 Downlink unicast traffic forwarding through M-LAG interfaces

 

Unicast traffic sent from the M-LAG interfaces to the single-attached interfaces

As shown in Figure 52, the access server (Server 2) is connected to the VTEPs in an M-LAG system through aggregate interfaces. Traffic from Server 2 to the single-attached interfaces are load-shared among multiple M-LAG devices (VTEP 1 and VTEP 2). The M-LAG device (VTEP 1) to which the single-attached interface belongs forwards the unicast traffic to the single-attached interface by searching local entries. Another M-LAG device (VTEP 2), upon receiving the unicast traffic, forwards the traffic to VTEP 1 through the peer link. VTEP 1 then forwards it to the single-attached interface.

Figure 52 Unicast traffic forwarding from M-LAG interfaces to single-attached interfaces

 

Unicast traffic forwarding of single-attached interfaces

Uplink unicast traffic of single-attached interfaces

As shown in Figure 53, upon receiving the unicast traffic sent by the single-attached access server (Server 1), VTEP 1 forwards the traffic to the remote VTEP through its local VXLAN tunnel.

Figure 53 Uplink unicast traffic forwarding through single-attached interfaces

 

Downlink unicast traffic of single-attached interfaces

As shown in Figure 54, unicast traffic sent from the external network to a single-attached server (Server 1) is forwarded to VTEP 1 through the VXLAN tunnel. Upon receiving the traffic, VTEP 1 directly forwards it to the single-attached interface. This traffic will not be sent to VTEP 2, avoiding the issue of forwarding traffic to the peer link.

Figure 54 Downlink unicast traffic forwarding of single-attached interfaces

 

Unicast traffic sent from single-attached interfaces to M-LAG interfaces

As shown in Figure 55, upon receiving unicast traffic sent from the access server (Server 1) to the M-LAG access server (Server 2), VTEP 1 forwards the traffic to the access server (Server 2) through the local AC by using the local-first forwarding principle.

Figure 55 Unicast traffic forwarding from single-attached interfaces to M-LAG interfaces

 

Intercommunication unicast traffic between single-attached interfaces

When the intercommunicating single-attached interfaces are connected to the same M-LAG device, the traffic forwarding process is the same as the EVPN VXLAN traffic forwarding process.

As shown in Figure 56, when the single-attached interfaces are connected to different M-LAG devices, the intercommunication is implemented through the peer link.

·     Intercommunication mechanism when the peer link is an Ethernet aggregate link: After a single-attached AC is created on a single-attached interface, the M-LAG device automatically creates a corresponding AC on the peer link and associates it with the VSI. When one M-LAG device receives a packet from a single-attached AC, it forwards the packet through the peer link to another M-LAG device. This M-LAG device obtains the packet's VSI based on the AC automatically created on the peer link, and then forwards the packet.

·     Intercommunication mechanism when the peer link is a VXLAN tunnel: When an M-LAG device receives a packet from a single-attached AC, it adds VXLAN encapsulation to the packet. The encapsulated VXLAN ID is the VXLAN ID corresponding to the VSI associated with the single-attached AC. The packet is then forwarded through the VXLAN tunnel acting as the peer link to another M-LAG device. This M-LAG device obtains the packet's VSI based on the VXLAN ID, and then forwards the packet.

Figure 56 Intercommunication unicast traffic forwarding between single-attached interfaces

 

BUM traffic forwarding

Uplink BUM traffic of M-LAG interfaces

As shown in Figure 57, the access server (Server 2) is connected to the VTEPs in a M-LAG system through aggregate interfaces. The uplink broadcast/unknown unicast/unknown multicast (BUM) traffic from the access server to the external network is load-shared among multiple M-LAG devices (VTEP 1 and VTEP 2). Upon receiving the BUM traffic, the M-LAG device identifies the VSI to which the BUM traffic belongs. It then forwards the traffic through all local ACs within that VSI (including single-attached ACs and except the receiving AC), VXLAN tunnel, and peer link. When the M-LAG device (VTEP) receives BUM traffic from the peer link, it forwards the traffic only to the local single-attached AC.

Figure 57 Uplink BUM traffic forwarding of M-LAG interfaces

 

Uplink BUM traffic of single-attached interfaces

As shown in Figure 58, upon receiving the BUM traffic from the single-attached access server (Server 1), VTEP 1 obtains the VSI of the traffic. It then forwards the traffic through all local ACs within that VSI (except the receiving AC), VXLAN tunnel, and peer link. Upon receiving the traffic from the peer link, VTEP 2 forwards it only to the local single-attached AC.

Figure 58 Uplink BUM traffic forwarding of single-attached interfaces

 

Downlink BUM traffic on the network side

As shown in Figure 59, the access server (Server 2) is connected to the VTEPs in an M-LAG system through aggregation interfaces. Server 1 and Server 3 are connected to VTEP 1 and VTEP 2, respectively. When the external network sends BUM traffic to the internal network where the server is located, VTEP 3 forwards the traffic to VTEP 1 and VTEP 2 through VXLAN tunnels.

Upon receiving BUM traffic, the M-LAG device obtains the VSI to which the traffic belongs and forwards the traffic within that VSI to all ACs (including the ACs corresponding to the M-LAG interface and single-attached ACs) and the peer link. Upon receiving BUM traffic from the peer link, the M-LAG device forwards the traffic only to the local single-attached AC.

Figure 59 Downlink BUM traffic forwarding of M-LAG interfaces

 

Fault processing mechanism

In an EVPN VXLAN network that supports M-LAG, the fault processing mechanism for peer link is the same as that for the M-LAG group network. For more information, see "M-LAG failure handling mechanisms." This section introduces only the fault processing mechanism for downlink fault, uplink fault, and fault of both the peer link and keepalive link.

Downlink fault processing mechanism

Ethernet aggregate link as peer link

As shown in Figure 60, in the EVPN VXLAN network with an Ethernet aggregate link as the peer link, when a downlink fault occurs on a certain M-LAG device (VTEP 2), the processing methods for uplink and downlink traffic are as follows:

·     The uplink traffic is transmitted through a functional link to another M-LAG device (VTEP 1). All uplink traffic is then forwarded through VTEP 1.

·     The process of downlink traffic forwarding is as follows:

Because the network side cannot detect the downlink fault, traffic will continue to be transmitted to all M-LAG devices.

After VTEP 2 receives the network traffic accessing Server 2, it forwards the traffic to VTEP 1 through the automatically created AC on the peer link. Based on the automatically created AC on the peer link, VTEP 1 obtains the VSI to which the packet belongs, and forwards the packet to Server 2.

After the fault is cleared, the AC on VTEP 2 comes up to forward traffic correctly.

Figure 60 Downlink fault processing mechanism (Ethernet aggregate link as peer link)

 

VXLAN tunnel as peer link

As shown in Figure 61, in the EVPN VXLAN network with a VXLAN tunnel as the peer link, when a downlink fault occurs on an M-LAG device (VTEP 2), the processing for the uplink and downlink traffic is as follows:

·     The uplink traffic is transmitted through a functional link to another M-LAG device (VTEP 1). All uplink traffic is then forwarded through VTEP 1.

·     The process of downlink traffic forwarding is as follows:

Because the network side cannot detect the downlink fault, traffic will continue to be transmitted to all M-LAG devices.

After VTEP 2 receives the network-side traffic accessing Server 2, it adds VXLAN encapsulation to the packet (the encapsulated VXLAN ID is the VXLAN ID corresponding to the VSI associated with the faulty AC), and then forwards the packet to VTEP 1 through the VXLAN tunnel acting as the peer link. VTEP 1 obtains the packet's VSI based on the VXLAN ID field carried in the received packet, and forwards the packet to Server 2.

After the fault is cleared, the AC on VTEP 2 comes up to forward traffic correctly.

Figure 61 Downlink fault processing mechanism (VXLAN tunnel as peer link)

 

Uplink fault processing mechanism

Ethernet aggregate link as peer link

In the EVPN VXLAN network with an Ethernet aggregate link as the peer link, as a best practice, deploy the peer link as a failover link. With a failover link deployed, the peer link can forward Layer 3 traffic and run routing protocols. In this way, the M-LAG device can communicate with the remote VTEP at Layer 3 through the peer link. On the underlay network, the path where the peer link is located serves as the backup path for the VXLAN tunnel between the M-LAG device and the remote VTEP. When the uplink of the M-LAG device fails, and the underlay main path of the VXLAN tunnel also fails, the VXLAN tunnel remains up and forwards traffic through the underlay backup path where the peer link is located.

As shown in Figure 62, to deploy a failover link, allow a certain VLAN to pass through the peer link, create the associated VLAN interface on the M-LAG device, and run a routing protocol on the VLAN interface to enable Layer 3 intercommunication with the remote VTEP. After the failover link is deployed, the VXLAN tunnel between VTEP 1 and VTEP 3 on the underlay network has two paths, a primary path and a backup path. The path passing through the peer link is the backup path. As a best practice, use VLAN 4094 to deploy the failover link.

Figure 62 Failover link deployment diagram

 

As shown in Figure 63, when a downlink fault occurs on an M-LAG device (VTEP 2), the processing for the uplink and downlink traffic on the M-LAG interface is as follows:

·     Uplink traffic: The access server (Server 2) is connected to the VTEPs in an M-LAG system through aggregate interfaces. The uplink unicast traffic destined for the external network is load shared between multiple M-LAG devices (VTEP 1 and VTEP 2).

¡     Upon receiving the uplink traffic, VTEP 1 forwards the traffic normally.

¡     Because a failover link is deployed, the VXLAN tunnel between the M-LAG device (VTEP 2) with an uplink fault and the remote VTEP remains up. The corresponding underlay path for this VXLAN tunnel is the one passing through the peer link. Therefore, upon receiving the uplink traffic, VTEP 2 performs VXLAN encapsulation, and sends the traffic through the peer link to another M-LAG device (VTEP 1), which then sends the traffic to the remote VTEP.

·     Downlink traffic: The remote VTEP sends downlink traffic to the M-LAG device through the VXLAN tunnel. Due to the uplink fault of VTEP 2, the downlink traffic is only transmitted to VTEP 1. Then, VTEP 1 forwards the traffic to Server 2.

Figure 63 Uplink fault processing mechanism (Ethernet aggregate link as peer link)

 

When an uplink fault occurs on a specific M-LAG device, the uplink and downlink traffic of the single-attached interface on this M-LAG device is transmitted to another M-LAG device through the peer link. This M-LAG device will then forward the traffic.

VXLAN tunnel as peer link

In an EVPN VXLAN network with a VXLAN tunnel as the peer link, when an uplink fault occurs for an M-LAG device, the VXLAN tunnel serving as the peer link also goes down. The fault processing mechanism is as follows:

·     If no keepalive link exists between the M-LAG devices, the M-LAG system splits. Both M-LAG devices will use their actual IP addresses to establish VXLAN tunnels with the remote VTEP, and both of them can forward traffic.

·     If a keepalive link exists between M-LAG devices, the interface on the backup device will be multi-active detection (MAD) down. Only the master device uses the virtual VTEP address to establish a VXLAN tunnel with the remote VTEP, and only the master device can forward traffic.

Both peer link and keepalive link are faulty

Ethernet aggregate link as peer link

In the EVPN VXLAN network with an Ethernet aggregate link as the peer link, when both the peer link and keepalive link fail, the M-LAG system splits. Both M-LAG devices use their actual IP addresses to establish a VXLAN tunnel with the remote VTEP, and both M-LAG devices can forward traffic.

As shown in Figure 64, when both the peer link and keepalive link are faulty, the uplink and downlink traffic are forwarded by the M-LAG interface as follows: Server 2 choose to transmit the uplink traffic to one M-LAG device (such as VTEP 1) based on the LACP priority. The downlink traffic is also transmitted to VTEP 1 through a VXLAN tunnel with the destination IP corresponding to the actual IP address. That is, both the uplink and downlink traffic of the M-LAG interface is forwarded through VTEP 1.

Figure 64 Both the peer link (Ethernet aggregate link) and keepalive link fail

 

When both the peer link and keepalive link fail, the traffic forwarding of single-attached interfaces is not affected.

VXLAN tunnel as peer link

In the EVPN VXLAN network with a VXLAN tunnel as the peer link, when both the peer link and keepalive link fail, the fault processing mechanism is the same as that in the EVPN VXLAN network where the peer link is an Ethernet aggregate link.

VXLAN support for M-LAG

Virtual eXtensible LAN (VXLAN) adopts the M-LAG technology to connect two physical devices, and virtualize them into one device. This virtual device is used as a VTEP (either for Layer 2 forwarding or as an VXLAN IP gateway). This avoids the impact of a single point failure of VTEP in the network, enhancing the reliability of the VXLAN network.

 

IMPORTANT: ·     In the current software version, the site and underlay networks for this feature can only be IPv4 networks or IPv6 networks. ·     M-LAG is not supported by the centralized VXLAN IP gateway protection group.

 

The working mechanism of VXLAN support for M-LAG is similar to that of the EVPN VXLAN support for M-LAG, and will not be discussed further in this section. This document introduces only the differences between the two working mechanisms.

Synchronizing the MAC address and ARP/ND information

VXLAN supports entry synchronization for M-LAG interfaces and single-attached interfaces in the M-LAG network. The working mechanism is similar to that in the EVPN VXLAN network deployed with M-LAG.

The MAC addresses, ARP/ND entries, and ARP/ND flooding suppression entries dynamically or statically learned on the VXLAN tunnel need to be synchronized to the peer M-LAG device through the peer link. Based on the VXLAN ID, tunnel source address, and tunnel destination address in a synchronized entry, the peer M-LAG device searches for a VXLAN tunnel with the same source and destination addresses of the same VXLAN ID. If a matching VXLAN tunnel is found, the entry is added to the VXLAN tunnel interface. If no match is found, the entry is not added.

Establishing a VXLAN tunnel

In the VXLAN network, VXLAN tunnels are manually created between VTEPs. That is, you need to manually specify the tunnel source and destination addresses as the IP addresses of the interfaces on the local and remote VTEPs, respectively. On the two VTEPs forming the M-LAG system, you need to configure the same IP address as the virtual VTEP address, and use this virtual VTEP address as the local source address to establish a tunnel with the remote VTEP.

In the VXLAN network with a VXLAN tunnel as the peer link, you need to manually create a VXLAN tunnel between the two VTEPs that form the M-LAG system.

Downlink BUM traffic on the network side

As shown in Figure 59, the access server (Server 2) connects to the VTEPs in an M-LAG system through aggregate interfaces. Server 1 and Server 3 is each attached to VTEP 1 and VTEP 2 through a single link, respectively. Upon receiving BUM traffic from the external network to the internal network where the servers are located, VTEP 3 forwards the traffic to M-LAG devices (VTEP 1 and VTEP 2) through VXLAN tunnels with a virtual VTEP IP address as the destination address. Because VTEP 1 and VTEP 2 share the virtual VTEP IP address, the traffic forwarded through this VXLAN tunnel is load-shared between VTEP 1 and VTEP 2. VTEP 1 and VTEP 2 obtain the VSI of the traffic and forward the traffic within that VSI to all ACs (including ACs associated with M-LAG interfaces and single-attached ACs) and the peer link. Upon receiving BUM traffic from the peer link, an M-LAG device only forwards the traffic to the local single-attached AC.

Figure 65 Downlink BUM traffic forwarding of M-LAG interfaces

 

EVPN data center interconnect support for M-LAG

NOTE: In the current software version, the site and underlay networks for this feature can only be IPv4 networks or IPv6 networks.

 

As shown in Figure 66, in the EVPN data center interconnect network, M-LAG is used to connect two physical devices and virtualize them into one device. This virtual device acts as the ED to avoid single point failure, enhancing the reliability of the EVPN network.

Figure 66 Network diagram

 

The working mechanism of EVPN data center interconnect support for M-LAG is the same as that of EVPN VXLAN support for M-LAG. For more information, see "EVPN VXLAN support for M-LAG."

 

MVXLAN support for M-LAG

NOTE: Only MDT-based MVXLAN in an EVPN VXLAN supports M-LAG. Ingress replication MVXLAN in a VXLAN does not support M-LAG.

 

Operating mechanism

Typical networking

MVXLAN uses M-LAG to virtualize two physical devices into an M-LAG system to prevent single points of failure from interrupting traffic. In a MVXLAN, both the VTEPs and border devices support M-LAG and can have both multicast sources and receivers attached.

Figure 67 shows EVPN VXLAN support for M-LAG. VTEP 1 and VTEP 2 form an M-LAG system, VTEP 3 and VTEP 4 form an M-LAG system, and Border 1 and Border 2 form an M-LAG system. The two VTEPs or border devices forming an M-LAG system share the same virtual address and are represented as a single virtual device externally. The other devices use the virtual address to establish unicast VXLAN tunnels with the virtual device. The MVXLAN tunnels also use the virtual address as the multicast source. Due to the multicast RPF check mechanism in the underlay network, a device will join the multicast VXLAN tunnel of only one device in an M-LAG system. For example, when VTEP 3 and VTEP 4 join the multicast VXLAN tunnel, they will only join one of the multicast VXLAN tunnels of VTEP 1 or VTEP 2 and will not join both simultaneously.

Figure 67 EVPN VXLAN support for M-LAG

 

User-site backup mechanism

In an M-LAG system, the M-LAG member devices synchronize multicast traffic and multicast join requests (IGMP membership reports or PIM join messages) over the peer link to maintain consistency in multicast source and receiver information. When one M-LAG member device fails or its uplink or downlink fails, the other M-LAG member device forwards all multicast traffic to avoid traffic interruption.

As shown in Figure 67, the user-site backup mechanism is as follows:

·     Multicast source-side backup: After Source 1 is connected to the M-LAG system, the multicast traffic of Source 1 will be transmitted to one of the member devices (VTEP 1 or VTEP 2). The VTEP that receives the multicast traffic synchronizes the multicast traffic to the other VTEP through the peer link, thereby realizing multicast traffic on both VTEP 1 and VTEP 2.

·     Multicast receiver-side backup: After the receiver is connected to the M-LAG system, the join request of the receiver will be transmitted to one of the member devices (VTEP 3 or VTEP 4). The VTEP that receives the join request synchronizes the request to the other VTEP through the peer link, thereby establishing multicast forwarding entries with the M-LAG interface as the output interface on both VTEP 3 and VTEP 4.

Multicast load sharing

The receiver-side M-LAG member devices load share he multicast traffic according to the odd/even rule. The M-LAG member with an odd M-LAG system number forwards multicast data destined for odd multicast group addresses. The M-LAG member with an even M-LAG system number forwards multicast data destined for even multicast group addresses When one M-LAG member device fails, the other M-LAG member device forwards all multicast traffic.

 

NOTE: The odd/even rule takes effect only on Layer 3 multicast forwarding, not on Layer 2 multicast forwarding.

 

Layer 2 multicast support for M-LAG

Layer 2 multicast refers to the scenario where the multicast source and the multicast receiver are located within the same VXLAN network. The multicast traffic within the same VXLAN network is forwarded according to Layer 2 multicast forwarding entries, such as IGMP snooping entries and PIM snooping entries.

As shown in Figure 68, only multicast receivers can be to connected to an M-LAG system. Multicast sources cannot be to connected to an M-LAG system. Additionally, only head-end replication is supported for traffic forwarding .

Figure 68 Layer 2 multicast support for M-LAG

 

Layer 3 multicast support for M-LAG

Layer 3 multicast refers to the scenario where the multicast source and the multicast receiver are located within different VXLAN networks but in the same VPN. The multicast traffic within different VXLAN networks but in the same VPN is forwarded according to Layer 3 multicast forwarding entries, such as IGMP entries and PIM entries. As shown in Figure 69, the EVPN VXLAN Layer 3 multicast scenario supports improving network reliability through M-LAG.

Figure 69 Layer 3 multicast support for M-LAG

 

Layer 3 multicast DCI support for M-LAG

To avoid single point of failure for Layer 3 multicast in DCI scenarios, you can deploy two EDs at the edge of a data center. The two EDs form an M-LAG system for ED redundancy.

As shown in Figure 70, MVXLAN tunnels are established within the DC, and unicast VXLAN-DCI tunnels are established between EDs. The EDs in the M-LAG system use the same virtual ED address. The M-LAG system uses the virtual ED address to establish  tunnels with VTEPs and remote EDs for ED redundancy and load balancing.

 

NOTE: M-LAG interfaces do not exist on ED devices.

 

Figure 70 Layer 3 multicast DCI support for M-LAG

 

 

Application scenarios

Single-tier M-LAG

As shown in Figure 71, to ensure reliability, Device C requires link redundancy when it accesses the network. Although deploying a loop prevention protocol (such as MSTP) is supported, this method results in low link utilization, wasting bandwidth resources. To achieve link redundancy and high link resource utilization, configure Device A and Device B as an M-LAG system to provide dual-homed access services for Device C. Device A and Device B can then form a load sharing relationship. When one device fails, another device can quickly take over to continue the traffic forwarding, ensuring correct running of services.

Figure 71 Single-tier M-LAG

 

Multi-tier M-LAG interconnect

As shown in Figure 72, multi-tier M-LAG interconnect extends the dual-homed access network with high reliability and link resource utilization, providing a stable network environment against an extended Layer 2 DC network deployed with multiple devices. Meanwhile, devices at the convergence layer act as dual-active gateways. M-LAG provides device-level reliability by forming aggregated links that connect devices at the core layer and those at the convergence layer.

Figure 72 Multi-tier M-LAG interconnect

 

Collaboration between M-LAG and STP

As shown in Figure 73, you can deploy M-LAG in conjunction with STP as follows:

·     Device A and Device B form an M-LAG system (for example, M-LAG system 1) at the access layer. Device C and Device D form an M-LAG system (for example, M-LAG system 2) at the convergence layer. The M-LAG systems prevent traffic interruption caused by single point of failure and improve network reliability.

·     Device F and the VM are dual-homed to M-LAG system 1 and M-LAG system 2, respectively, ensuring reliable forwarding of uplink and downlink traffic. The VM is dual-homed to M-LAG system 2 through Device E.

·     The network is deployed with multi-tier M-LAG. Device C and Device D at the convergence layer act as Layer 3 gateways to provide gateway and routing services for Device F. M-LAG supports two gateway deployment schemes including VLAN dual-active gateway and VRRP gateway.

·     STP is deployed on Devices A to D, and Device C and Device D are specified as the root bridges to eliminate loops between M-LAG systems.

Figure 73 M-LAG+STP at two tiers

 

Collaboration between M-LAG and QinQ

As shown in Figure 74, M-LAG and QinQ are deployed on devices at the convergence layer to provide high reliability and load sharing for QinQ services. The spanning tree protocol is deployed between the convergence layer and the core layer to avoid loops.

Figure 74 Collaboration between M-LAG and QinQ

 

Collaboration between M-LAG and super VLAN

As shown in Figure 75, the devices that provide access services for user endpoints are dual-homed to the gateway M-LAG system (consisting of Device A and Device B) for Layer 3 communication with external networks. The M-LAG devices run super VLAN services to conserve IP address resources and achieve device-level reliability and load sharing through the M-LAG system. The uplinks of the two M-LAG devices can achieve load sharing with the help of routing protocols.

Figure 75 Collaboration between M-LAG and super VLAN

 

Collaboration between M-LAG and private VLAN

As shown in Figure 76, the devices that provide access services for user endpoints are dual-homed to the gateway M-LAG system (consisting of Device A and Device B) for Layer 2 and Layer 3 communication with external networks. The M-LAG devices run private VLAN services to conserve IP address resources and achieve device-level reliability and load sharing through the M-LAG system. The uplinks of the two M-LAG devices can achieve load sharing with the help of routing protocols. If the two M-LAG devices require Layer 2 communication with the external network, you need to deploy the spanning tree protocol on the uplinks to avoid loops.

Figure 76 Collaboration between M-LAG and private VLAN

 

EVPN VXLAN with M-LAG

In an EVPN VXLAN network deployed with M-LAG, two VTEPs are virtualized as one VTEP. The two VTEPs  synchronize MAC address and ARP information over the peer link. As shown in Figure 77 and Figure 78, the peer link can be an Ethernet aggregation link or a VXLAN tunnel.

In the downlink direction, the links that connect a server to the two VTEP form an aggregation link, achieving backup of user-side links. A VM can access the network access upon failure of a single Ethernet link.

Figure 77 EVPN VXLAN with M-LAG (Ethernet aggregation link acts as the peer link)

 

Figure 78 EVPN VXLAN with M-LAG (VXLAN tunnel acts as the peer link)

 

Related documentation

·     IEEE P802.1AX-REV™/D4.4c: Draft Standard for Local and Metropolitan Area Networks

·     RFC 7432: BGP MPLS-Based Ethernet VPN