03-Layer 2—LAN Switching Configuration Guide

HomeSupportSwitchesS6300 SeriesConfigure & DeployConfiguration GuidesH3C S6300 Switch Series Configuration Guides-Release 243x-6W10003-Layer 2—LAN Switching Configuration Guide
05-Ethernet link aggregation configuration
Title Size Download
05-Ethernet link aggregation configuration 326.25 KB

Contents

Configuring Ethernet link aggregation· 1

Overview· 1

Ethernet link aggregation application scenario· 1

Aggregation group, member port, and aggregate interface· 1

Aggregation states of member ports in an aggregation group· 1

Operational key· 2

Configuration types· 2

Link aggregation modes· 3

How static link aggregation works· 3

LACP· 4

How dynamic link aggregation works· 5

Edge aggregate interface· 8

Load sharing modes for link aggregation groups· 8

Ethernet link aggregation configuration task list 8

Configuring an aggregation group· 9

Configuration restrictions and guidelines· 9

Configuring a static aggregation group· 9

Configuring a dynamic aggregation group· 10

Configuring an aggregate interface· 11

Configuring the description of an aggregate interface· 11

Specifying ignored VLANs for a Layer 2 aggregate interface· 11

Setting the minimum and maximum numbers of Selected ports for an aggregation group· 12

Setting the expected bandwidth for an aggregate interface· 12

Configuring an edge aggregate interface· 13

Enabling BFD for an aggregation group· 13

Shutting down an aggregate interface· 14

Restoring the default settings for an aggregate interface· 14

Specifying link aggregation management VLANs and management port 14

Configuring load sharing for link aggregation groups· 15

Setting load sharing modes for link aggregation groups· 15

Configuring the ignored fields for the default load sharing mode· 16

Enabling local-first load sharing for link aggregation· 16

Configuring per-flow load sharing algorithm settings for Ethernet link aggregation· 17

Enabling link-aggregation traffic redirection· 17

Configuration restrictions and guidelines· 18

Configuration procedure· 18

Specifying a link aggregation management subnet 18

Displaying and maintaining Ethernet link aggregation· 19

Ethernet link aggregation configuration examples· 20

Layer 2 static aggregation configuration example· 20

Layer 2 dynamic aggregation configuration example· 21

Layer 2 aggregation load sharing configuration example· 23

Layer 2 edge aggregate interface configuration example· 26


Configuring Ethernet link aggregation

Overview

Ethernet link aggregation bundles multiple physical Ethernet links into one logical link, called an aggregate link. Link aggregation has the following benefits:

·           Increased bandwidth beyond the limits of any single link. In an aggregate link, traffic is distributed across the member ports.

·           Improved link reliability. The member ports dynamically back up one another. When a member port fails, its traffic is automatically switched to other member ports.

Ethernet link aggregation application scenario

As shown in 0, Device A and Device B are connected by three physical Ethernet links. These physical Ethernet links are combined into an aggregate link called link aggregation 1. The bandwidth of this aggregate link can reach up to the total bandwidth of the three physical Ethernet links. At the same time, the three Ethernet links back up one another. When a physical Ethernet link fails, the traffic previously carried on the failed link is switched to the other two links.

Ethernet link aggregation diagram

 

Aggregation group, member port, and aggregate interface

Link bundling is implemented through interface bundling. An aggregation group is a group of Ethernet interfaces bundled together, which are called member ports of the aggregation group. For each aggregation group, a logical interface (called an aggregate interface), is created. When an upper layer entity uses the link aggregation service, the following events occur:

·           A link aggregation group appears the same as a single logical link.

·           An aggregate interface transmits data traffic.

When you create an aggregate interface, the device automatically creates an aggregation group of the same type and number as the aggregate interface. For example, when you create aggregate interface 1, aggregation group 1 is created.

The port rate of an aggregate interface equals the total rate of its Selected member ports. Its duplex mode is the same as that of the Selected member ports. For more information about the states of member ports in an aggregation group, see "Aggregation states of member ports in an aggregation group."

Aggregation states of member ports in an aggregation group

A member port in an aggregation group can be in any of the following aggregation states:

·           SelectedA Selected port can forward traffic.

·           UnselectedAn Unselected port cannot forward traffic.

·           Individual—An Individual port can forward traffic as a normal physical port. A port is placed in the Individual state when the following conditions are met:

¡  The corresponding aggregate interface is configured as an edge aggregate interface.

¡  The port does not receive LACPDUs from its peer port.

Operational key

When aggregating ports, the system automatically assigns each port an operational key based on port information, such as port rate and duplex mode. Any change to this information triggers a recalculation of the operational key.

In an aggregation group, all Selected ports are assigned the same operational key.

Configuration types

Every configuration setting on a port might affect its aggregation state. Port configurations include the following types:

·           Attribute configurations—To become a Selected port, a member port must have the same attribute configurations as the aggregate interface. Figure 1 describes the attribute configurations.

Attribute configurations made on an aggregate interface are automatically synchronized to all member ports. These configurations are retained on the member ports even after the aggregate interface is removed.

Any attribute configuration change might affect the aggregation state of link aggregation member ports and running services. The system displays a warning message every time you try to change an attribute configuration setting on a member port.

Figure 1 Attribute configurations

Feature

Considerations

Port isolation

Indicates whether the port has joined an isolation group and which isolation group the port belongs to.

QinQ

QinQ enable state (enabled/disabled), TPID for VLAN tags, and VLAN transparent transmission. For information about QinQ, see "Configuring QinQ."

VLAN mapping

Different types of VLAN mapping configured on the port. For more information about VLAN mapping, see "Configuring VLAN mapping."

VLAN

·          Permitted VLAN IDs.

·          PVID.

·          Link type (trunk, hybrid, or access).

·          Operating mode (promiscuous, trunk promiscuous, host).

·          VLAN tagging mode.

For information about VLAN, see "Configuring VLANs."

 

·           Protocol configurations—Protocol configurations do not affect the aggregation state of the member ports. MAC address learning and spanning tree settings are examples of protocol configurations.

 

 

NOTE:

The protocol configuration for a member port is effective only when the member port leaves the aggregation group.

 

Link aggregation modes

An aggregation group operates in one of the following modes:

·           StaticStatic aggregation is stable. An aggregation group in static mode is called a static aggregation group. The aggregation states of the member ports in a static aggregation group are not affected by the peer ports.

·           DynamicAn aggregation group in dynamic mode is called a dynamic aggregation group. The local system and the peer system automatically maintain the aggregation states of the member ports. Dynamic link aggregation reduces the administrators' workload.

How static link aggregation works

Choosing a reference port

When setting the aggregation state of the ports in an aggregation group, the system automatically chooses a member port as the reference port. A Selected port must have the same operational key and attribute configurations as the reference port.

The system chooses a reference port from the member ports in up state.

The candidate reference ports are organized into different priority levels following these rules:

1.      In descending order of port priority.

2.      Full duplex.

3.      In descending order of speed.

4.      Half duplex.

5.      In descending order of speed.

From the candidate ports with the same attribute configurations as the aggregate interface, the one with the highest priority level is chosen as the reference port.

·           If multiple ports have the same priority level, the port that has been Selected (if any) is chosen. If multiple ports with the same priority level have been Selected, the one with the smallest port number is chosen.

·           If multiple ports have the same priority level and none of them has been Selected, the port with the smallest port number is chosen.

Setting the aggregation state of each member port

After a static aggregation group reaches the limit on Selected ports, ports attempting to join the group are put in Unselected state. This prevents traffic interruption on the existing Selected ports.

Setting the aggregation state of a member port in a static aggregation group

 

For more information about configuring the maximum number of Selected ports in a static aggregation group, see "Setting the minimum and maximum numbers of Selected ports for an aggregation group."

Any operational key or attribute configuration change might affect the aggregation state of link aggregation member ports.

LACP

Dynamic aggregation mode is implemented through IEEE 802.3ad Link Aggregation Control Protocol (LACP).

LACP uses LACPDUs to exchange aggregation information between LACP-enabled devices.

Each member port in an LACP-enabled aggregation group exchanges information with its peer. When a member port receives an LACPDU, it compares the received information with information received on the other member ports. In this way, the two systems reach an agreement on which ports are placed in the Selected state.

LACP functions

LACP offers basic LACP functions and extended LACP functions, as described in Figure 2.

Figure 2 Basic and extended LACP functions

Category

Description

Basic LACP functions

Implemented through the basic LACPDU fields, including the system LACP priority, system MAC address, port priority, port number, and operational key.

Extended LACP functions

Implemented by extending the LACPDU with new TLV fields. This is how the LACP MAD mechanism of the IRF feature is implemented.

The switch series can participate in LACP MAD as either an IRF member device or an intermediate device.  For more information about IRF and the LACP MAD mechanism, see IRF Configuration Guide.

 

LACP operating modes

LACP can operate in active mode or passive mode.

When LACP is operating in passive mode on a local member port and its peer port, both ports cannot send LACPDUs. When LACP is operating in active mode on the port on either end of a link, both ports can send LACPDUs.

LACP priorities

LACP priorities include system LACP priority and port priority, as described in Figure 3. The smaller the priority value, the higher the priority.

Figure 3 LACP priorities

Type

Description

System LACP priority

Used by two peer devices (or systems) to determine which one is superior in link aggregation.

In dynamic link aggregation, the system that has higher system LACP priority sets the Selected state of member ports on its side. The system that has lower priority sets port state accordingly.

Port priority

Determines the likelihood of a member port to be selected on a system. A port with a higher port priority is more likely to become Selected.

 

LACP timeout interval

The LACP timeout interval specifies how long a member port waits to receive LACPDUs from the peer port. If a local member port does not receive LACPDUs from the peer within the LACP timeout interval, the member port considers the peer as failed.

The LACP timeout interval also determines the LACPDU sending rate of the peer. LACP timeout intervals include the following types:

·           Short timeout interval—3 seconds. If you use the short timeout interval, the peer sends one LACPDU per second.

·           Long timeout interval—90 seconds. If you use the long timeout interval, the peer sends one LACPDU every 30 seconds.

How dynamic link aggregation works

Choosing a reference port

The system chooses a reference port from the member ports that are in up state and have the same attribute configurations as the aggregate interface. A Selected port must have the same operational key and attribute configurations as the reference port.

The local system (the actor) and the remote system (the partner) negotiate a reference port by using the following workflow:

1.      The two systems compare their system IDs to determine the system with the smaller system ID.

A system ID contains the system LACP priority and the system MAC address.

a.    The two systems compare their LACP priority values.

The lower the LACP priority, the smaller the system ID. If LACP priority values are the same, the two systems proceed to step b.

b.    The two systems compare their MAC addresses.

The lower the MAC address, the smaller the system ID.

2.      The system with the smaller system ID chooses the port with the smallest port ID as the reference port.

A port ID contains a port priority and a port number. The lower the port priority, the smaller the port ID.

a.    The system chooses the port with the lowest priority value as the reference port.

If ports have the same priority, the system proceeds to step b.

b.    The system compares their port numbers.

The smaller the port number, the smaller the port ID.

The port with the smallest port number and the same attribute configurations as the aggregate interface is chosen as the reference port.

Setting the aggregation state of each member port

After the reference port is chosen, the system with the smaller system ID sets the state of each member port on its side.

Setting the state of a member port in a dynamic aggregation group

 

Meanwhile, the system with the higher system ID is aware of the aggregation state changes on the remote system. The system sets the aggregation state of local member ports the same as their peer ports.

When you aggregate interfaces in dynamic mode, follow these guidelines:

·           A dynamic link aggregation group preferably sets full-duplex ports as the Selected ports. The group will set only one half-duplex port as a Selected port when either of the following conditions exist:

¡  None of the full-duplex ports can be chosen as Selected ports.

¡  Only half-duplex ports exist in the group.

·           To ensure stable aggregation and service continuity, do not change the operational key or attribute configurations on any member port.

·           When the aggregation state of a local port changes in a dynamic aggregation group, the aggregation state of the peer port also changes.

·           After the Selected port limit is reached, a newly joining port becomes a Selected port if it is more eligible than a current Selected port.

For more information about configuring the maximum number of Selected ports in a dynamic aggregation group, see "Setting the minimum and maximum numbers of Selected ports for an aggregation group."

Edge aggregate interface

Dynamic link aggregation fails on a server-facing aggregate interface if dynamic link aggregation is configured only on the device. The device forwards traffic by using only one of the physical ports that are connected to the server.

To improve link reliability, configure the aggregate interface as an edge aggregate interface. This feature enables all member ports of the aggregation group to forward traffic. When a member port fails, its traffic is automatically switched to other member ports.

After dynamic link aggregation is configured on the server, the device can receive LACPDUs from the server. Then, link aggregation between the device and the server operates correctly.

An edge aggregate interface takes effect only when it is configured on an aggregate interface corresponding to a dynamic aggregation group.

Load sharing modes for link aggregation groups

In a link aggregation group, traffic can be load shared across the Selected ports based on any of the following modes:

·           Per-flow load sharing—Load shares traffic on a per-flow basis. The load sharing mode classifies packets into flows and forwards packets of the same flow on the same link. This mode can be one or any combination of the following criteria that classify traffic:

¡  Source or destination MAC address.

¡  Source or destination port number.

¡  Ingress port.

¡  Source or destination IP address.

·           Packet type-based load sharing—Load shares traffic automatically based on packet types (Layer 2, IPv4, or IPv6 for example).

Ethernet link aggregation configuration task list

Tasks at a glance

(Required.) Configuring an aggregation group:

·          Configuring a static aggregation group

·          Configuring a dynamic aggregation group

(Optional.) Configuring an aggregate interface:

·          Configuring the description of an aggregate interface

·          Specifying ignored VLANs for a Layer 2 aggregate interface

·          Setting the minimum and maximum numbers of Selected ports for an aggregation group

·          Setting the expected bandwidth for an aggregate interface

·          Configuring an edge aggregate interface

·          Enabling BFD for an aggregation group

·          Shutting down an aggregate interface

·          Restoring the default settings for an aggregate interface

·          Specifying link aggregation management VLANs and management port

(Optional.) Configuring load balancing for link aggregation group:

·          Setting load sharing modes for link aggregation groups

·          Configuring the ignored fields for the default load sharing mode

·          Enabling local-first load sharing for link aggregation

·          Configuring per-flow load sharing algorithm settings for Ethernet link aggregation_Ref440288950

(Optional.) Enabling link-aggregation traffic redirection

(Optional.) Specifying a link aggregation management subnet

 

Configuring an aggregation group

This section explains how to configure an aggregation group.

Configuration restrictions and guidelines

When you configure an aggregation group, follow these restrictions and guidelines:

·           You cannot assign a port to a Layer 2 aggregation group if any of the following features are configured on the port:

¡  MAC authentication (see Security Configuration Guide).

¡  Port security (see Security Configuration Guide).

¡  802.1X (see Security Configuration Guide).

·           If a port is used as a reflector port for port mirroring, do not assign it to an aggregation group. For more information about reflector ports, see Network Management and Monitoring Configuration Guide.

·           Removing an aggregate interface also removes its aggregation group and causes all member ports to leave the aggregation group.

·           You must configure the same aggregation mode on the two ends of an aggregate link.

·           This switch series supports a maximum of 128 aggregation groups.

Configuring a static aggregation group

To guarantee a successful static aggregation, make sure that the ports at both ends of each link are in the same aggregation state.

Avoid assigning ports to a static aggregation group that has reached the limit on Selected ports. These ports will be placed in the Unselected state to avoid traffic interruption on the current Selected ports. However, a device reboot can cause the aggregation state of member ports to change.

To configure a Layer 2 static aggregation group:

Step

Command

Remarks

1.      Enter system view.

system-view

N/A

2.      Create a Layer 2 aggregate interface and enter Layer 2 aggregate interface view.

interface bridge-aggregation interface-number

When you create a Layer 2 aggregate interface, the system automatically creates a Layer 2 static aggregation group numbered the same.

3.      Exit to system view.

quit

N/A

4.      Assign an interface to the specified Layer 2 aggregation group.

a       Enter Layer 2 Ethernet interface view:
interface interface-type interface-number

b       Assign the interface to the specified Layer 2 aggregation group:
port link-aggregation group number

Repeat these two sub-steps to assign more Layer 2 Ethernet interfaces to the aggregation group.

 

Configuring a dynamic aggregation group

To guarantee a successful dynamic aggregation, make sure that the peer ports of the ports aggregated at one end are also aggregated. The two ends can automatically negotiate the aggregation state of each member port.

To configure a Layer 2 dynamic aggregation group:

Step

Command

Remarks

1.      Enter system view.

system-view

N/A

2.      Set the system LACP priority.

lacp system-priority system-priority

By default, the system LACP priority is 32768.

Changing the system LACP priority might affect the aggregation state of the ports in a dynamic aggregation group.

3.      Create a Layer 2 aggregate interface and enter Layer 2 aggregate interface view.

interface bridge-aggregation interface-number

When you create a Layer 2 aggregate interface, the system automatically creates a Layer 2 static aggregation group numbered the same.

4.      Configure the aggregation group to operate in dynamic aggregation mode.

link-aggregation mode dynamic

By default, an aggregation group operates in static aggregation mode.

5.      Exit to system view.

quit

N/A

6.      Assign an interface to the specified Layer 2 aggregation group.

a       Enter Layer 2 Ethernet interface view:
interface interface-type interface-number

b       Assign the interface to the specified Layer 2 aggregation group:
port link-aggregation group number

Repeat these two sub-steps to assign more Layer 2 Ethernet interfaces to the aggregation group.

7.      Set the LACP operating mode on the interface.

·          Set LACP to operate in passive mode:
lacp mode
passive

·          Set LACP to operate in active mode:
undo lacp mode

By default, LACP is operating in active mode.

8.      Configure the port priority for the interface.

link-aggregation port-priority port-priority

The default setting is 32768.

9.      Configure the short LACP timeout interval (3 seconds) on the interface.

lacp period short

By default, the long LACP timeout interval (90 seconds) is adopted by the interface. The peer sends LACPDUs slowly.

Do not configure the short LACP timeout interval before performing an ISSU. Otherwise, traffic interruption will occur during the ISSU. For more information about ISSU, see Fundamentals Configuration Guide.

 

Configuring an aggregate interface

Most of the configurations that can be performed on Layer 2 Ethernet interfaces can also be performed on Layer 2 aggregate interfaces.

Configuring the description of an aggregate interface

You can configure the description of an aggregate interface for administration purposes, for example, describing the purpose of the interface.

To configure the description of an aggregate interface:

 

Step

Command

Remarks

1.      Enter system view.

system-view

N/A

2.      Enter aggregate interface view.

interface bridge-aggregation interface-number

N/A

3.      Configure the description of the aggregate interface.

description text

By default, the description of an interface is interface-name Interface.

 

Specifying ignored VLANs for a Layer 2 aggregate interface

By default, to become Selected ports, the member ports must have the same VLAN permit state and VLAN tagging mode as the corresponding Layer 2 aggregate interface.

The system ignores the permit state and tagging mode of an ignored VLAN when choosing Selected ports.

To specify ignored VLANs for a Layer 2 aggregate interface:

 

Step

Command

Remarks

1.      Enter system view.

system-view

N/A

2.      Enter Layer 2 aggregate interface view.

interface bridge-aggregation interface-number

N/A

3.      Specify ignored VLANs.

link-aggregation ignore vlan vlan-id-list

By default, a Layer 2 aggregate interface does not ignore any VLANs.

 

Setting the minimum and maximum numbers of Selected ports for an aggregation group

IMPORTANT

IMPORTANT:

The minimum and maximum number of Selected ports must be the same for the local and peer aggregation groups.

 

The bandwidth of an aggregate link increases as the number of Selected member ports increases. To avoid congestion, you can set the minimum number of Selected ports required for bringing up an aggregate interface.

This minimum threshold setting affects the aggregation state of both aggregation member ports and the aggregate interface.

·           When the number of member ports eligible to be Selected ports is smaller than the minimum threshold:

¡  All member ports are placed in the Unselected state.

¡  The link of the aggregate interface goes down.

·           When the minimum threshold is reached, the eligible member ports are placed in the Selected state, and the link of the aggregate interface goes up.

The maximum number of Selected ports allowed in an aggregation group is limited by either the configured maximum number or hardware capability, whichever value is smaller.

You can configure backup between two ports by performing the following tasks:

·           Assign two ports to an aggregation group.

·           Configure 1 as the maximum number of Selected ports allowed in the aggregation group.

Then, only one Selected port is allowed in the aggregation group at any point in time, and the Unselected port acts as a backup port.

To set the minimum and maximum numbers of Selected ports for an aggregation group:

 

Step

Command

Remarks

1.      Enter system view.

system-view

N/A

2.      Enter aggregate interface view.

interface bridge-aggregation interface-number

N/A

3.      Set the minimum number of Selected ports for the aggregation group.

link-aggregation selected-port minimum number

By default, the minimum number of Selected ports for the aggregation group is not specified.

4.      Set the maximum number of Selected ports for the aggregation group.

link-aggregation selected-port maximum number

By default, the maximum number of Selected ports for an aggregation group is 32.

 

Setting the expected bandwidth for an aggregate interface

Step

Command

Remarks

1.      Enter system view.

system-view

N/A

2.      Enter aggregate interface view.

interface bridge-aggregation interface-number

N/A

3.      Set the expected bandwidth for the interface.

bandwidth bandwidth-value

By default, the expected bandwidth (in kbps) is the interface baud rate divided by 1000.

 

Configuring an edge aggregate interface

When you configure an edge aggregate interface, follow these restrictions and guidelines:

·           This configuration takes effect only on the aggregate interface corresponding to a dynamic aggregation group.

·           Link-aggregation traffic redirection does not operate correctly on an edge aggregate interface. For more information about link-aggregation traffic redirection, see "Enabling link-aggregation traffic redirection."

To configure an edge aggregate interface:

 

Step

Command

Remarks

1.      Enter system view.

system-view

N/A

2.      Enter aggregate interface view.

interface bridge-aggregation interface-number

N/A

3.      Configure the aggregate interface as an edge aggregate interface.

lacp edge-port

By default, an aggregate interface does not operate as an edge aggregate interface.

 

Enabling BFD for an aggregation group

BFD for Ethernet link aggregation can monitor member link status in an aggregation group. After you enable BFD on an aggregate interface, each Selected port in the aggregation group establishes a BFD session with its peer port. All the BFD sessions use UDP port 6784 and destination MAC address 01-00-5E-90-00-01. BFD operates differently depending on the aggregation mode.

·           BFD for static aggregation—When BFD detects a link failure, BFD notifies the Ethernet link aggregation module that the peer port is unreachable. The local port is placed in the Unselected state. The BFD session between the local and peer ports remains, and the local port keeps sending BFD packets. When the link is recovered, the local port receives BFD packets from the peer port, and BFD notifies the Ethernet link aggregation module that the peer port is reachable. The local port is placed in the Selected state again. This mechanism ensures that the local and peer ports of a static aggregate link have the same aggregation state.

·           BFD for dynamic aggregation—When BFD detects a link failure, BFD notifies the Ethernet link aggregation module that the peer port is unreachable. BFD clears the session and stops sending BFD packets. When the link is recovered and the local port is placed in the Selected state again, the local port establishes a new session with the peer port. BFD notifies the Ethernet link aggregation module that the peer port is reachable. Because BFD provides fast failure detection, the local and peer systems of a dynamic aggregate link can negotiate the aggregation state of their member ports faster.

To enable BFD for an aggregation group:

 

Step

Command

Remarks

1.      Enter system view.

system-view

N/A

2.      Enter aggregate interface view.

interface bridge-aggregation interface-number

N/A

3.      Enable BFD for the aggregation group.

link-aggregation bfd ipv4 source ip-address destination ip-address

By default, BFD is disabled for an aggregation group.

 

Shutting down an aggregate interface

Make sure no member port in an aggregation group is configured with the loopback command when you shut down the aggregate interface. Similarly, a port configured with the loopback command cannot be assigned to an aggregate interface already shut down. For more information about the loopback command, see Layer 2—LAN Switching Command Reference.

Shutting down or bringing up an aggregate interface affects the aggregation state and link state of ports in the corresponding aggregation group in the following ways:

·           When an aggregate interface is shut down, all Selected ports in the corresponding aggregation group become unselected and their link state becomes down.

·           When an aggregate interface is brought up, the aggregation state of ports in the corresponding aggregation group is recalculated.

To shut down an aggregate interface:

 

Step

Command

1.      Enter system view.

system-view

2.      Enter aggregate interface view.

interface bridge-aggregation interface-number

3.      Shut down the aggregate interface.

shutdown

 

Restoring the default settings for an aggregate interface

You can return all configurations on an aggregate interface to default settings.

To restore the default settings for an aggregate interface:

 

Step

Command

1.      Enter system view.

system-view

2.      Enter aggregate interface view.

interface bridge-aggregation interface-number

3.      Restore the default settings for the aggregate interface.

default

 

Specifying link aggregation management VLANs and management port

For an aggregation group to forward traffic of some VLANs through a specific port, specify the VLANs as management VLANs and the port as a management port.

Do not use the display link-aggregation load-sharing path command to identify the outgoing port for the traffic in a management VLAN. This command calculates the outgoing port for a traffic flow based on the algorithm for the specified load sharing mode. The outgoing port output from this command for management VLAN traffic might not be the management port (the actual outgoing port).

To specify link aggregation management VLANs and management port for an aggregation group:

 

Step

Command

Remarks

1.      Enter system view.

system-view

N/A

2.      Specify link aggregation management VLANs.

link-aggregation management-vlan vlan-id1 [ vlan-id2 ]

By default, no link aggregation management VLANs are specified.

If you execute this command multiple times, the most recent configuration takes effect.

3.      Enter Layer 2 Ethernet interface view.

interface interface-type interface-number

N/A

4.      Configure the port as a management port for its aggregation group.

link-aggregation management-port

By default, a port does not act as a management port in its aggregation group.

The management port must be a Selected port.

 

Configuring load sharing for link aggregation groups

This section explains how to configure load sharing modes for link aggregation groups and how to enable local-first load sharing for link aggregation.

Setting load sharing modes for link aggregation groups

You can configure the global or group-specific load sharing mode. A link aggregation group preferentially uses the group-specific load sharing mode. If no group-specific load sharing mode is available, the group uses the global load sharing mode.

Setting the global link-aggregation load sharing mode

Step

Command

Remarks

1.      Enter system view.

system-view

N/A

2.      Configure the global link-aggregation load sharing mode.

link-aggregation global load-sharing mode { destination-ip | destination-mac | destination-port | ingress-port | source-ip | source-mac | source-port } *

By default, the system automatically chooses the global link-aggregation load sharing mode according to the packet type.

 

Setting the group-specific load sharing mode

Step

Command

Remarks

1.      Enter system view.

system-view

N/A

2.      Enter aggregate interface view.

interface bridge-aggregation interface-number

N/A

3.      Configure the load sharing mode for the aggregation group.

link-aggregation load-sharing mode  { destination-ip | destination-mac | source-ip | source-mac } *

The default settings are the same as the global load sharing mode.

 

Configuring the ignored fields for the default load sharing mode

When the default load sharing mode is used and traffic is not evenly load shared, you can configure this feature. Then, some fields in packets are ignored to prevent these fields from affecting load sharing.

To configure the ignored fields for the default load sharing mode:

 

Step

Command

Remarks

1.      Enter system view.

system-view

N/A

2.      Configure the ignored fields for the default load sharing mode.

link-aggregation load-sharing ignore ethernet-type

By default, no ignored fields are configured for the default load sharing mode.

 

Enabling local-first load sharing for link aggregation

Use local-first load sharing in a multidevice link aggregation scenario to distribute traffic preferentially across member ports on the ingress card or device.

When you aggregate ports on different member devices in an IRF fabric, you can use local-first load sharing to reduce traffic on IRF links, as shown in 0. For more information about IRF, see IRF Configuration Guide.

Load sharing for multiswitch link aggregation in an IRF fabric

 

To enable local-first load sharing for link aggregation:

 

Step

Command

Remarks

1.      Enter system view.

system-view

N/A

2.      Enable local-first load sharing for link aggregation.

link-aggregation load-sharing mode local-first

By default, local-first load sharing for link aggregation is enabled.

 

Configuring per-flow load sharing algorithm settings for Ethernet link aggregation

Configure the per-flow load sharing algorithm and algorithm seed to optimize traffic distribution on aggregate links based on existing per-flow load sharing settings. You can set only the algorithm or the algorithm seed, or both. You can combine an algorithm with different algorithm seeds to obtain different effects.

This feature does not take effect on per-flow load sharing that uses the following traffic classification criteria:

·           Source IP address.

·           Destination IP address.

·           Source MAC address.

·           Destination MAC address.

·           Source and destination IP addresses.

·           Source and destination MAC addresses.

To configure per-flow load sharing algorithm settings for Ethernet link aggregation:

 

Step

Command

Remarks

1.      Enter system view.

system-view

N/A

2.      Configure the load sharing algorithm.

link-aggregation global load-sharing algorithm algorithm-number

By default, algorithm 0 is used.

If the device fails to load share traffic flows across all Selected ports, you can specify algorithm 1 to 8 in sequence until the problem is solved.

3.      Configure the load sharing algorithm seed.

link-aggregation global load-sharing seed seed-number

By default, algorithm seed 0 is used.

 

Enabling link-aggregation traffic redirection

Link-aggregation traffic redirection prevents traffic interruption.

When you shut down a Selected port in an aggregation group, this feature redirects traffic to other Selected ports.

When you restart an IRF member device that contains Selected ports, this feature redirects traffic to other IRF member devices.

You can enable link-aggregation traffic redirection globally or for an aggregation group. Global link-aggregation traffic redirection settings take effect on all aggregation groups. A link aggregation group preferentially uses the group-specific link-aggregation traffic redirection settings. If group-specific link-aggregation traffic redirection is not configured, the group uses the global link-aggregation traffic redirection settings.

Configuration restrictions and guidelines

When you enable ink-aggregation traffic redirection, follow these restrictions and guidelines:

·           Link-aggregation traffic redirection applies only to dynamic link aggregation groups and takes effect on only known unicast packets.

·           To prevent traffic interruption, enable link-aggregation traffic redirection on devices at both ends of the aggregate link.

·           To prevent packet loss that might occur at a reboot, do not enable spanning tree together with link-aggregation traffic redirection.

·           Link-aggregation traffic redirection does not operate correctly on an edge aggregate interface.

·           As a best practice, enable link-aggregation traffic redirection on aggregate interfaces. If you enable this feature globally, communication with a third-party peer device might be affected if the peer is not compatible with this feature.

Configuration procedure

To enable link-aggregation traffic redirection globally:

 

Step

Command

Remarks

1.      Enter system view.

system-view

N/A

2.      Enable link-aggregation traffic redirection globally.

link-aggregation lacp traffic-redirect-notification enable

By default, link-aggregation traffic redirection is disabled globally.

 

To enable link-aggregation traffic redirection for an aggregation group:

 

Step

Command

Remarks

1.      Enter system view.

system-view

N/A

2.      Enter aggregate interface view.

interface bridge-aggregation interface-number

N/A

3.      Enable link-aggregation traffic redirection for the aggregation group.

link-aggregation lacp traffic-redirect-notification enable

By default, link-aggregation traffic redirection is disabled for an aggregation group.

 

Specifying a link aggregation management subnet

By default, when an aggregate interface receives an ARP packet, the aggregate interface will create the corresponding ARP entry. When the aggregate interface receives packets with the matching destination address, the packets are load shared among all Selected ports of the aggregate interface.

To manage traffic with the specified destination addresses, you can specify a link aggregation management subnet.

When a subnet is specified and an aggregate interface receives an ARP packet of the subnet, the device uses the IP address of the packet to match ARP entries.

·           If no match is found, the aggregate interface creates an ARP entry only on the member port receiving the packet. Then, packets that access the subnet through the aggregate interface are forwarded through the specified member port and are not load shared.

·           If a MAC address moves, the output interface of the matching ARP entry changes. The device does not update the ARP entry. Instead, the device sends the ARP request throughout the network, and the ARP entry for the MAC address is re-learned.

When an aggregate interface sends ARP requests of the specified subnet, the requests are sent out of all member ports of the aggregate interface.

To specify a link aggregation management subnet:

 

Step

Command

Remarks

1.      Enter system view.

system-view

N/A

2.      Specify a link aggregation management subnet.

link-aggregation management-subnet ip-address { mask | mask-length }

By default, no link aggregation management subnet is specified.

 

Displaying and maintaining Ethernet link aggregation

Execute display commands in any view and reset commands in user view.

 

Task

Command

Display information for an aggregate interface or multiple aggregate interfaces.

display interface [ bridge-aggregation ] [ brief [ down ] ]

display interface [ bridge-aggregation [ interface-number ] ] [ brief [ description ] ]

Display the local system ID.

display lacp system-id

Display the global or group-specific link-aggregation load sharing modes.

display link-aggregation load-sharing mode [ interface [ bridge-aggregation interface-number ] ]

Display forwarding information for the specified traffic flow.

display link-aggregation load-sharing path interface bridge-aggregation interface-number ingress-port interface-type interface-number [ route ] { { destination-ip ip-address | destination-ipv6 ipv6-address } | { source-ip ip-address | source-ipv6 ipv6-address } | destination-mac mac-address | destination-port port-id | ethernet-type type-number | ip-protocol protocol-id | source-mac mac-address | source-port port-id | vlan vlan-id }*

Display detailed link aggregation information for link aggregation member ports.

display link-aggregation member-port [ interface-list ]

Display summary information about all aggregation groups.

display link-aggregation summary

Display detailed information about the specified aggregation groups.

display link-aggregation verbose [ bridge-aggregation [ interface-number ] ]

Clear LACP statistics for the specified link aggregation member ports.

reset lacp statistics [ interface interface-list ]

Clear statistics for the specified aggregate interfaces.

reset counters interface [ bridge-aggregation [ interface-number ] ]

 

Ethernet link aggregation configuration examples

Layer 2 static aggregation configuration example

Network requirements

On the network shown in 0, perform the following tasks:

·           Configure a Layer 2 static aggregation group on both Device A and Device B.

·           Enable VLAN 10 at one end of the aggregate link to communicate with VLAN 10 at the other end.

·           Enable VLAN 20 at one end of the aggregate link to communicate with VLAN 20 at the other end.

Network diagram

 

Configuration procedure

1.      Configure Device A:

# Create VLAN 10, and assign port Ten-GigabitEthernet 1/0/4 to VLAN 10.

<DeviceA> system-view

[DeviceA] vlan 10

[DeviceA-vlan10] port ten-gigabitethernet 1/0/4

[DeviceA-vlan10] quit

# Create VLAN 20, and assign port Ten-GigabitEthernet 1/0/5 to VLAN 20.

[DeviceA] vlan 20

[DeviceA-vlan20] port ten-gigabitethernet 1/0/5

[DeviceA-vlan20] quit

# Create Layer 2 aggregate interface Bridge-Aggregation 1.

[DeviceA] interface bridge-aggregation 1

[DeviceA-Bridge-Aggregation1] quit

# Assign ports Ten-GigabitEthernet 1/0/1 through Ten-GigabitEthernet 1/0/3 to link aggregation group 1.

[DeviceA] interface ten-gigabitethernet 1/0/1

[DeviceA-Ten-GigabitEthernet1/0/1] port link-aggregation group 1

[DeviceA-Ten-GigabitEthernet1/0/1] quit

[DeviceA] interface ten-gigabitethernet 1/0/2

[DeviceA-Ten-GigabitEthernet1/0/2] port link-aggregation group 1

[DeviceA-Ten-GigabitEthernet1/0/2] quit

[DeviceA] interface ten-gigabitethernet 1/0/3

[DeviceA-Ten-GigabitEthernet1/0/3] port link-aggregation group 1

[DeviceA-Ten-GigabitEthernet1/0/3] quit

# Configure Layer 2 aggregate interface Bridge-Aggregation 1 as a trunk port and assign it to VLANs 10 and 20.

[DeviceA] interface bridge-aggregation 1

[DeviceA-Bridge-Aggregation1] port link-type trunk

[DeviceA-Bridge-Aggregation1] port trunk permit vlan 10 20

[DeviceA-Bridge-Aggregation1] quit

2.      Configure Device B in the same way Device A is configured. (Details not shown.)

Verifying the configuration

# Display detailed information about all aggregation groups on Device A.

[DeviceA] display link-aggregation verbose

Loadsharing Type: Shar -- Loadsharing, NonS -- Non-Loadsharing

Port Status: S -- Selected, U -- Unselected

             I -- Individual, * -- Management port

Flags:  A -- LACP_Activity, B -- LACP_Timeout, C -- Aggregation,

        D -- Synchronization, E -- Collecting, F -- Distributing,

        G -- Defaulted, H -- Expired

 

Aggregate Interface: Bridge-Aggregation1

Aggregation Mode: Static

Loadsharing Type: Shar

Management VLAN : None

  Port               Status  Priority Oper-Key

--------------------------------------------------------------------------------

  XGE1/0/1           S       32768    1

  XGE1/0/2           S       32768    1

  XGE1/0/3           S       32768    1

The output shows that:

·           Link aggregation group 1 is a Layer 2 static aggregation group.

·           The aggregation group contains three Selected ports.

Layer 2 dynamic aggregation configuration example

Network requirements

On the network shown in 0, perform the following tasks:

·           Configure a Layer 2 dynamic aggregation group on both Device A and Device B.

·           Enable VLAN 10 at one end of the aggregate link to communicate with VLAN 10 at the other end.

·           Enable VLAN 20 at one end of the aggregate link to communicate with VLAN 20 at the other end.

Network diagram

 

Configuration procedure

1.      Configure Device A:

# Create VLAN 10, and assign the port Ten-GigabitEthernet 1/0/4 to VLAN 10.

<DeviceA> system-view

[DeviceA] vlan 10

[DeviceA-vlan10] port ten-gigabitethernet 1/0/4

[DeviceA-vlan10] quit

# Create VLAN 20, and assign the port Ten-GigabitEthernet 1/0/5 to VLAN 20.

[DeviceA] vlan 20

[DeviceA-vlan20] port ten-gigabitethernet 1/0/5

[DeviceA-vlan20] quit

# Create Layer 2 aggregate interface Bridge-Aggregation 1, and set the link aggregation mode to dynamic.

[DeviceA] interface bridge-aggregation 1

[DeviceA-Bridge-Aggregation1] link-aggregation mode dynamic

[DeviceA-Bridge-Aggregation1] quit

# Assign ports Ten-GigabitEthernet 1/0/1 through Ten-GigabitEthernet 1/0/3 to link aggregation group 1.

[DeviceA] interface ten-gigabitethernet 1/0/1

[DeviceA-Ten-GigabitEthernet1/0/1] port link-aggregation group 1

[DeviceA-Ten-GigabitEthernet1/0/1] quit

[DeviceA] interface ten-gigabitethernet 1/0/2

[DeviceA-Ten-GigabitEthernet1/0/2] port link-aggregation group 1

[DeviceA-Ten-GigabitEthernet1/0/2] quit

[DeviceA] interface ten-gigabitethernet 1/0/3

[DeviceA-Ten-GigabitEthernet1/0/3] port link-aggregation group 1

[DeviceA-Ten-GigabitEthernet1/0/3] quit

# Configure Layer 2 aggregate interface Bridge-Aggregation 1 as a trunk port and assign it to VLANs 10 and 20.

[DeviceA] interface bridge-aggregation 1

[DeviceA-Bridge-Aggregation1] port link-type trunk

[DeviceA-Bridge-Aggregation1] port trunk permit vlan 10 20

[DeviceA-Bridge-Aggregation1] quit

2.      Configure Device B in the same way Device A is configured. (Details not shown.)

Verifying the configuration

# Display detailed information about all aggregation groups on Device A.

[DeviceA] display link-aggregation verbose

Loadsharing Type: Shar -- Loadsharing, NonS -- Non-Loadsharing

Port Status: S -- Selected, U -- Unselected

             I -- Individual, * -- Management port

Flags:  A -- LACP_Activity, B -- LACP_Timeout, C -- Aggregation,

        D -- Synchronization, E -- Collecting, F -- Distributing,

        G -- Defaulted, H -- Expired

 

Aggregate Interface: Bridge-Aggregation1

Aggregation Mode: Dynamic

Loadsharing Type: Shar

Management VLAN : None

System ID: 0x8000, 000f-e267-6c6a

Local:

  Port               Status  Priority Oper-Key  Flag

--------------------------------------------------------------------------------

  XGE1/0/1           S       32768    1         {ACDEF}

  XGE1/0/2           S       32768    1         {ACDEF}

  XGE1/0/3           S       32768    1         {ACDEF}

Remote:

  Actor            Partner Priority Oper-Key  SystemID               Flag

--------------------------------------------------------------------------------

  XGE1/0/1           1       32768    1         0x8000, 000f-e267-57ad {ACDEF}

  XGE1/0/2           2       32768    1         0x8000, 000f-e267-57ad {ACDEF}

  XGE1/0/3           3       32768    1         0x8000, 000f-e267-57ad {ACDEF}

The output shows that:

·           Link aggregation group 1 is a Layer 2 dynamic aggregation group.

·           The aggregation group contains three Selected ports.

Layer 2 aggregation load sharing configuration example

Network requirements

On the network shown in 0, perform the following tasks:

·           Configure two Layer 2 static aggregation groups 1 and 2 on Device A and Device B, respectively.

·           Enable VLAN 10 at one end of the aggregate link to communicate with VLAN 10 at the other end.

·           Enable VLAN 20 at one end of the aggregate link to communicate with VLAN 20 at the other end.

·           Configure link aggregation groups 1 and 2 to load share traffic across aggregation group member ports.

¡  Configure link aggregation group 1 to load share packets based on source MAC addresses.

¡  Configure link aggregation group 2 to load share packets based on destination MAC addresses.

Network diagram

 

Configuration procedure

1.      Configure Device A:

# Create VLAN 10, and assign the port Ten-GigabitEthernet 1/0/5 to VLAN 10.

<DeviceA> system-view

[DeviceA] vlan 10

[DeviceA-vlan10] port ten-gigabitethernet 1/0/5

[DeviceA-vlan10] quit

# Create VLAN 20, and assign the port Ten-GigabitEthernet 1/0/6 to VLAN 20.

[DeviceA] vlan 20

[DeviceA-vlan20] port ten-gigabitethernet 1/0/6

[DeviceA-vlan20] quit

# Create Layer 2 aggregate interface Bridge-Aggregation 1.

[DeviceA] interface bridge-aggregation 1

# Configure Layer 2 aggregation group 1 to load share packets based on source MAC addresses.

[DeviceA-Bridge-Aggregation1] link-aggregation load-sharing mode source-mac

[DeviceA-Bridge-Aggregation1] quit

# Assign ports Ten-GigabitEthernet 1/0/1 and Ten-GigabitEthernet 1/0/2 to link aggregation group 1.

[DeviceA] interface ten-gigabitethernet 1/0/1

[DeviceA-Ten-GigabitEthernet1/0/1] port link-aggregation group 1

[DeviceA-Ten-GigabitEthernet1/0/1] quit

[DeviceA] interface ten-gigabitethernet 1/0/2

[DeviceA-Ten-GigabitEthernet1/0/2] port link-aggregation group 1

[DeviceA-Ten-GigabitEthernet1/0/2] quit

# Configure Layer 2 aggregate interface Bridge-Aggregation 1 as a trunk port and assign it to VLAN 10.

[DeviceA] interface bridge-aggregation 1

[DeviceA-Bridge-Aggregation1] port link-type trunk

[DeviceA-Bridge-Aggregation1] port trunk permit vlan 10

[DeviceA-Bridge-Aggregation1] quit

# Create Layer 2 aggregate interface Bridge-Aggregation 2.

[DeviceA] interface bridge-aggregation 2

# Configure Layer 2 aggregation group 2 to load share packets based on destination MAC addresses.

[DeviceA-Bridge-Aggregation2] link-aggregation load-sharing mode destination-mac

[DeviceA-Bridge-Aggregation2] quit

# Assign ports Ten-GigabitEthernet 1/0/3 and Ten-GigabitEthernet 1/0/4 to link aggregation group 2.

[DeviceA] interface ten-gigabitethernet 1/0/3

[DeviceA-Ten-GigabitEthernet1/0/3] port link-aggregation group 2

[DeviceA-Ten-GigabitEthernet1/0/3] quit

[DeviceA] interface ten-gigabitethernet 1/0/4

[DeviceA-Ten-GigabitEthernet1/0/4] port link-aggregation group 2

[DeviceA-Ten-GigabitEthernet1/0/4] quit

# Configure Layer 2 aggregate interface Bridge-Aggregation 2 as a trunk port and assign it to VLAN 20.

[DeviceA] interface bridge-aggregation 2

[DeviceA-Bridge-Aggregation2] port link-type trunk

[DeviceA-Bridge-Aggregation2] port trunk permit vlan 20

[DeviceA-Bridge-Aggregation2] quit

2.      Configure Device B in the same way Device A is configured. (Details not shown.)

Verifying the configuration

# Display detailed information about all aggregation groups on Device A.

[DeviceA] display link-aggregation verbose

Loadsharing Type: Shar -- Loadsharing, NonS -- Non-Loadsharing

Port Status: S -- Selected, U -- Unselected

             I -- Individual, * -- Management port

Flags:  A -- LACP_Activity, B -- LACP_Timeout, C -- Aggregation,

        D -- Synchronization, E -- Collecting, F -- Distributing,

        G -- Defaulted, H -- Expired

 

Aggregate Interface: Bridge-Aggregation1

Aggregation Mode: Static

Loadsharing Type: Shar

Management VLAN : None

  Port               Status  Priority Oper-Key

--------------------------------------------------------------------------------

  XGE1/0/1           S       32768    1

  XGE1/0/2           S       32768    1

 

Aggregate Interface: Bridge-Aggregation2

Aggregation Mode: Static

Loadsharing Type: Shar

    Port             Status  Priority Oper-Key

--------------------------------------------------------------------------------

  XGE1/0/3           S       32768    2

  XGE1/0/4           S       32768    2

The output shows that:

·           Link aggregation groups 1 and 2 are both load-shared Layer 2 static aggregation groups.

·           Each aggregation group contains two Selected ports.

# Display all the group-specific load sharing modes on Device A.

[DeviceA] display link-aggregation load-sharing mode interface

 

Bridge-Aggregation1 Load-Sharing Mode:

source-mac address

 

Bridge-Aggregation2 Load-Sharing Mode:

destination-mac address

The output shows that:

·           Link aggregation group 1 load shares packets based on source MAC addresses.

·           Link aggregation group 2 load shares packets based on destination MAC addresses.

Layer 2 edge aggregate interface configuration example

Network requirements

As shown in 0, a Layer 2 dynamic aggregation group is configured on the device. The server is not configured with dynamic link aggregation.

Configure an edge aggregate interface so that both Ten-GigabitEthernet 1/0/1 and Ten-GigabitEthernet 1/0/2 can forward traffic to improve link reliability.

Network diagram

 

Configuration procedure

# Create Layer 2 aggregate interface Bridge-Aggregation 1, and set the link aggregation mode to dynamic.

<Device> system-view

[Device] interface bridge-aggregation 1

[Device-Bridge-Aggregation1] link-aggregation mode dynamic

# Configure Layer 2 aggregate interface Bridge-Aggregation 1 as an edge aggregate interface.

[Device-Bridge-Aggregation1] lacp edge-port

[Device-Bridge-Aggregation1] quit

# Assign ports Ten-GigabitEthernet 1/0/1 and Ten-GigabitEthernet 1/0/2 to link aggregation group 1.

[Device] interface ten-gigabitethernet 1/0/1

[Device-Ten-GigabitEthernet1/0/1] port link-aggregation group 1

[Device-Ten-GigabitEthernet1/0/1] quit

[Device] interface ten-gigabitethernet 1/0/2

[Device-Ten-GigabitEthernet1/0/2] port link-aggregation group 1

[Device-Ten-GigabitEthernet1/0/2] quit

Verifying the configuration

# Display detailed information about all aggregation groups on the device when the server is not configured with dynamic link aggregation.

[Device] display link-aggregation verbose

Loadsharing Type: Shar -- Loadsharing, NonS -- Non-Loadsharing

Port Status: S -- Selected, U -- Unselected

             I -- Individual, * -- Management port

Flags:  A -- LACP_Activity, B -- LACP_Timeout, C -- Aggregation,

        D -- Synchronization, E -- Collecting, F -- Distributing,

        G -- Defaulted, H -- Expired

 

Aggregate Interface: Bridge-Aggregation1

Aggregation Mode: Dynamic

Loadsharing Type: NonS

Management VLAN : None

System ID: 0x8000, 000f-e267-6c6a

Local:

  Port             Status  Priority Oper-Key  Flag

--------------------------------------------------------------------------------

  XGE1/0/1          I       32768    1         {AG}

  XGE1/0/2          I       32768    1         {AG}

Remote:

  Actor            Partner Priority Oper-Key  SystemID               Flag

--------------------------------------------------------------------------------

  XGE1/0/1          0       32768    0         0x8000, 0000-0000-0000 {DEF}

  XGE1/0/2          0       32768    0         0x8000, 0000-0000-0000 {DEF}

The output shows that Ten-GigabitEthernet 1/0/1 and Ten-GigabitEthernet 1/0/2 are in Individual state when they do not receive LACPDUs from the server. Both Ten-GigabitEthernet 1/0/1 and Ten-GigabitEthernet 1/0/2 can forward traffic. When one port fails, its traffic is automatically switched to the other port.

 

  • Cloud & AI
  • InterConnect
  • Intelligent Computing
  • Security
  • SMB Products
  • Intelligent Terminal Products
  • Product Support Services
  • Technical Service Solutions
All Services
  • Resource Center
  • Policy
  • Online Help
All Support
  • Become a Partner
  • Partner Resources
  • Partner Business Management
All Partners
  • Profile
  • News & Events
  • Online Exhibition Center
  • Contact Us
All About Us
新华三官网