RIR Technology White Paper

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Contents

Overview·· 1

RIR implementation· 1

Flow template· 1

Link types· 1

Preference-based link selection· 2

Link preference· 2

Link selection rules· 2

Redundant link selection· 2

Primary and backup links· 2

Quality-based link selection· 3

RIR server and RIR client 3

NQA link probes· 5

RIR quality evaluation mechanism·· 6

RIR quality policy· 6

Bandwidth-based link selection· 7

Link bandwidth· 7

Bandwidth-based link selection policy· 8

Load balancing· 8

RIR working mechanisms· 8

Overview of the link selection mechanism·· 8

Preference-based primary link selection· 10

Preference-based backup link selection· 12

Quality tolerant link selection· 13

Bandwidth tolerant link selection· 13

Link selection delay and link selection suppression period· 13

Typical network applications· 13

RIR-based multilink optimization· 14

RIR-based load sharing· 14

 

Overview

In the networking of enterprise data centers, like the typical Hub-Spoke network, traditional enterprises normally deploy some public applications at the center. Through leasing the network from the carrier, they connect the branch offices to the data center.

Leasing carrier networks can have varying prices, for example, MPLS dedicated lines are expensive, while regular Internet networks are relatively cheaper. To reduce costs, companies prefer to select more affordable networks while meeting their service requirements. At the same time, companies typically deploy multiple links between data centers and branch offices. They need to consider how to balance the use of link bandwidth, avoid idle links, or prevent some links from being overloaded.

Additionally, with the development of mobile Internet, cloud computing, and artificial intelligence (AI), traffic models have evolved. The traditional wide area network (WAN) oriented for connectivity can no longer meet the network requirements of the new era. At this point, the network needs to proactively "adapt" to service traffic, and change as per service requirements.

Resilient intelligent routing (RIR) is a technical solution to the above-mentioned problems.

Compared to traditional link selection, RIR can dynamically selects the most suitable links for traffic forwarding based on service requirements (for example, link quality and link bandwidth). For example, voice services require high link quality, so a high-quality link is preferred. Most IP Internet services, which do not require high link quality, preferentially select a relatively average quality but more affordable link. If the link selected by the service traffic no longer meets its requirements due to a change in link state, RIR can automatically switch it to another link that meets the requirements.

RIR implementation

RIR is applied in the Hub-Spoke network environment based on VXLAN (Virtual eXtensible LAN, the expansion of virtual local area network). RIR can select different VXLAN tunnels for the transmission of diverse service traffic between Hub sites and Spoke sites, according to link preference, link primary and backup roles, link quality, and link bandwidth.

Flow template

A flow template defines link selection policies for a type of service flow. A flow ID uniquely identifies a flow template.

The device applies the link selection policies under a flow template to the service flow marked with the flow ID of the flow template.

The device supports using QoS policies to mark flow IDs for service flows. After QoS identifies the service of a packet based on the quintuple and DSCP of the packet, it assigns a flow ID to the packet. Then, RIR will perform link selection for the packet based on the flow template that uses the flow ID.

The flow ID is marked only in the RIR process, and it will not be added to any outgoing packets.

Link types

RIR defines link types for links, differentiating links under different network types, and also assigns link identifiers to differentiate different links under the same network type. Based on commonly used network types, RIR has defined four link types: 4G, Internet, MPLS, and MSTP. Link types are merely for link identification and do not affect the actual encapsulation form of the message.

During the RIR deployment based on VXLAN networking, there can only be one VXLAN tunnel under each VSI virtual interface (in each VXLAN) between each Hub site and each Spoke site. As shown in Figure 1, for the Hub site, a unique VXLAN tunnel between the Hub site and any Spoke site can be determined via the VSI virtual interface.

Therefore, by configuring the link type and number under the VSI virtual interface, a VXLAN tunnel can be uniquely identified between the Hub site and the Spoke site.

Figure 1 Links in VXLAN network

 

Preference-based link selection

Link preference

For a certain type of service traffic, users can define the preference for its optional links based on link signatures, service requirements, and other factors such as link price. When the device selects a route for service traffic, it prioritizes links with a higher preference.

When deploying RIR based on VXLAN network, you can configure the link preference for VSI virtual interfaces under flow templates, based on link types and numbers. Since there can only be one VXLAN tunnel under each VSI virtual interface (in each VXLAN) between each Hub site and each Spoke site, configuring the link preference for VSI virtual interfaces under flow templates determines the preference of the VXLAN tunnel between a Hub site and a Spoke site.

Link selection rules

You can assign the same preference value to different links in the same flow template.

RIR selects a link for a type of service flows from the links in the flow template in descending order of link preference. If the links with the highest preference cannot meet the service requirements, RIR tries the links with the second highest preference, and so forth to the links with the lowest preference.

If the flow template has two or more links with the same preference, RIR performs link selection based on RIR link load sharing criteria.

Redundant link selection

Primary and backup links

To ensure traffic reliability, in actual network configuration, aside from setting up a primary Hub site, a backup Hub site is generally also established. The Spoke site is connected simultaneously to both the primary and backup Hub sites. In case the link to the primary Hub site is unavailable, the Spoke site can transmit traffic to the backup Hub site, ensuring that traffic is not disrupted.

The link between the Spoke site and the primary Hub site is called the primary link, and the link between the Spoke site and the backup Hub is called the backup link. As shown in Figure 2, when the primary links 1 and 2 between the Spoke site and the primary Hub site do not meet the traffic requirements, the backup link 3 can be used to transmit traffic to the backup Hub site.

Figure 2 Primary and backup links

 

In the flow template, based on link type and number, after configuring the preference for the VSI virtual interface, the associated VXLAN tunnel is selected as the primary link by default. If a VXLAN tunnel associated with a VSI virtual interface is configured as an RIR backup tunnel, the tunnel is selected as an alternate link. If needed in the actual network setup, users can configure the links between the Spoke site and the primary Hub site as alternate links.

Quality-based link selection

RIR server and RIR client

About RIR server and RIR client

In a hub-spoke network, a hub is typically connected to multiple spokes. To avoid the hub from consuming too many resources on link quality detection by Network Quality Analyzer (NQA) probes, RIR-VXLAN provides the following roles:

·     RIR server—An RIR server does not perform NQA link probes. It performs link selection based on the link quality probe results synchronized from RIR clients.

·     RIR client—An RIR client performs NQA link probes to detect the link quality and synchronizes the link quality probe results to RIR servers.

Configure a hub as an RIR server and spokes as RIR clients, so the hub can perform link selection based on the link quality probe results synchronized from the RIR clients.

RIR server function

You can enable the RIR server globally or on an interface, as shown in Figure 3.

·     If you enable the RIR server globally, the RIR server is also enabled on all interfaces on the device. The interfaces can receive RIR link quality probe results from RIR clients.

·     If you enable the RIR server on an interface, only that interface can receive RIR link quality probe results from RIR clients.

Figure 3 Enabling the RIR server

 

In a VXLAN network, only tunnel interfaces support enabling the RIR server. The RIR server uses the tunnel interfaces to receive RIR link quality probe results from RIR clients.

RIR client function

The RIR client synchronizes link quality probe results to RIR servers. Enabling the RIR client is the same as enabling the RIR server.

RIR server and client enabling policy

Enable the RIR server or RIR client, or use them in combination, depending on the role of the device in the network.

·     If the device acts only as a hub, you can enable the RIR server globally.

·     If the device acts only as a spoke, you can enable the RIR client globally.

·     If the device acts as both a hub and a spoke, you can enable the RIR server and RIR client on the corresponding interfaces.

The RIR server and RIR client cannot be both enabled on the same interface. If the enabled role (RIR server or client) on an interface is different from the globally enabled role, the interface-specific role takes effect on that interface.

As shown in Figure 4, enable the global and interface-specific RIR server and RIR client in combination as follows:

·     Device A—Enable the RIR server globally.

·     Device B—Enable the RIR client globally and enable the RIR server on an interface.

·     Device C—Enable the RIR client on an interface and enable the RIR server on an interface.

·     Device D—Enable the RIR client globally.

·     Device E—Enable the RIR client on an interface.

Figure 4 Enabling the RIR server and client

 

NQA link probes

RIR uses NQA to detect the status of candidate links and selects the most suitable link based on the NQA link quality probe results.

A hub performs link selection based on the link quality probe results synchronized from spokes. RIR uses spokes as NQA clients and hubs as NQA servers. The following types of NQA link probe operations are defined:

·     NQA link connectivity probe operation—Performs ICMP echo probes to check the connectivity of each link. If a link is disconnected, RIR does not perform NQA link quality probes on that link. You can configure only one NQA link connectivity probe operation.

·     NQA link quality probe operations—Also referred to as NQA link quality operations, perform UDP jitter probes to detect the link delay, jitter, and packet loss ratio for links that pass NQA link connectivity check. You can configure multiple NQA link quality operations. The operations might offer different link quality probe results for the same link.

The device performs NQA probes only for links that are assigned link types and link indexes.

RIR quality evaluation mechanism

To meet the differentiated requirements of services on link quality, configure a Service Level Agreement (SLA) for each service. An SLA contains a set of link quality evaluation thresholds, including the link delay threshold, packet loss threshold, and jitter threshold.

In RIR, the quality policy of a flow template contains an SLA and an NQA link quality operation. By comparing the NQA link quality probe results with the thresholds in the SLA, the device determines whether a link meets the quality requirements of the service. If all parameter values in the probe results of a link are lower than or equal to the thresholds in the SLA, the link is qualified for the service

RIR quality policy

Link quality probe results

As shown in Figure 5, if a flow template is configured with a quality policy, the spoke determines whether a link is qualified for that type of service flow based on the NQA link quality probe results. In addition, the spoke synchronizes the link quality probe results to the hub. The hub performs link selection based on the link quality probe results synchronized from the spoke. You only need to configure the quality policy on the spoke.

RIR uses the following rules to determine whether a link is qualified for a type of service flow:

·     If the link fails NQA link connectivity check, RIR determines that the link is unqualified.

·     If the link passes NQA link connectivity check, RIR checks the NQA link quality probe results for the link.

¡     If all link quality parameter values in the probe results are lower than or equal to the link quality thresholds in the SLA, RIR determines that the link is qualified.

¡     If any link quality parameter value in the probe results is higher than the corresponding link quality threshold in the SLA, RIR determines that the link is unqualified.

Figure 5 Link quality probe result processing

 

Quality-based link selection on a hub and a spoke

When a spoke performs quality-based link selection, it considers only the quality policy configured on the spoke. If a quality policy has been configured in a flow template on the spoke, the spoke calculates link quality probe results based on the NQA link quality operation and SLA in the quality policy. Then, the spoke performs link selection for the flow that matches the flow template based on the link quality probe results. If no quality policy is configured in a flow template on the spoke, the spoke does not consider the quality factor when it performs link selection. All links in the flow template meet the service requirements in quality.

When a hub performs quality-based link selection for traffic sent to a spoke, it considers the quality policies both on the hub and spoke. Table 1 shows the link selection rules for a flow that matches a flow template on a hub.

Table 1 Quality-based link selection rules on a hub

Whether a quality policy is configured on the hub	Whether a quality policy is configured on the spoke	Quality-based link selection rules on the hub
Yes	Yes	The hub performs link selection based on the link quality probe results synchronized from the spoke.
Yes	No	The spoke does not synchronize link quality probe results to the hub. The hub determines that no link in the flow template meets the quality requirements.
No	Yes	The spoke synchronizes link quality probe results to the hub. The hub determines that all links in the flow template are qualified for packets that match the flow template.
No	No	The spoke does not synchronize link quality probe results to the hub. The hub determines that all links in the flow template are qualified for packets that match the flow template.

 

As a best practice, configure a quality policy both on the hub and spoke for a type of service flow or do not configure any quality policy on the hub or spoke for the type of service flow.

Bandwidth-based link selection

Bandwidth-based link selection not only can select links that meet the service bandwidth requirements, but also can load share service traffic among multiple links. This manner can avoid a link from being overwhelmed or congested.

Link bandwidth

The device can select a suitable link for service traffic based on the following bandwidths:

·     The used bandwidth of the link.

·     The total bandwidth of the link.

·     The per-session expected bandwidth.

The device can automatically obtain the bandwidth already used by the link, while the total link bandwidth and the per-session expected bandwidth need to be configured by the user or calculated based on the user's configuration.

The device uses sessions as the minimum granularity and performs bandwidth-based link selection to achieve refined link bandwidth management. A session is uniquely defined by a quintuple including the source IP address, destination IP address, source port, destination port, and transport layer protocol.

When the device selects links for traffic of a session, it first performs bandwidth detection based on the per-session expected bandwidth in the flow template to which the session belongs. A link is qualified in the bandwidth detection if it meets the following requirements:

In the deployment of RIR in the VXLAN network, a physical interface might correspond to multiple VSI interfaces, and a VSI interface might also correspond to multiple VXLAN tunnels. After allocating a certain bandwidth from the total bandwidth of the physical interface to the VSI interface, all VXLAN tunnels under this VSI interface will equally divide the bandwidth allocated to the VSI interface. The bandwidth allocated to a single VXLAN tunnel is the total bandwidth of the link in the link selection. For example, if the total bandwidth of a physical interface is 100 Mbps, you can allocate 50 Mbps of bandwidth to one of the VSI interfaces using this physical interface. If there are two VXLAN tunnels under this VSI interface, then each VXLAN tunnel is allocated 25 Mbps of bandwidth.

Bandwidth-based link selection policy

When the device selects links for traffic of a session, it first performs bandwidth detection based on the per-session expected bandwidth in the flow template to which the session belongs. A link is qualified in the bandwidth detection if it meets the following requirements: The used bandwidth plus the per-session expected bandwidth is less than 80% of the total bandwidth.

When different sessions of the same service class use the same link selection policy, the link selection results might vary.

Load balancing

If multiple links are available for sessions that match a flow template, the device distributes the traffic of the sessions to these links for load balancing based on the link bandwidths. RIR supports the following load balancing modes:

·     For preference-based primary link selection, preference-based backup link selection, and quality tolerant link selection, if multiple links meet the requirements under the same preference, the UCMP (Unequal Cost Multiple Path) load sharing algorithm is used based on the remaining bandwidth of the optional links to select a link for a session. The higher the remaining bandwidth of a link, the higher the probability it will be selected.

·     For bandwidth-tolerant link selection, if multiple links meet the requirements, a link for the session is selected using the UCMP load sharing algorithm based on the total bandwidth of the optional links, with a higher probability of selecting the link with larger bandwidth.

For detailed information about preference-based primary link selection, preference-based backup link selection, quality tolerant link selection, and bandwidth tolerant link selection, see "RIR working mechanisms."

RIR working mechanisms

Overview of the link selection mechanism

Upon receiving the message, the device firstly classifies the service traffic based on the 5-tuple and DSCP, and re-marks the fow ID through QoS. Then, the device searches the routing table for an available route. If so, it enters the RIR process, otherwise, it discards the packet.

After the device enters the RIR process, the link selection is as follows:

1.     Determine if the flow ID carried by the packet is valid and whether there are two or more equal cost paths that can transmit this packet. If the flow ID carried by the packet is invalid or there are not two or more equal cost paths, the device performs routing table-based forwarding for the packet. If the flow ID carried by the packet is valid and there are two or more equal cost paths, the device continue to execute the following operations.

When two or more equal cost routes exist in a device's routing table, it is considered that there are two or more equal cost paths.

2.     Select the flow template based on the flow ID carried by the packet.

3.     Selects the most suitable link from the links in the flow template by using the following criteria in order:

a.     Preference-based primary link selection

b.     Preference-based backup link selection

c.     Quality tolerant link selection

d.     Bandwidth tolerant link selection

4.     If a link is found suitable, RIR returns the link selection result and stops searching other links. If no link is found suitable for a criterion, RIR uses the next criterion to select links. If RIR fails to find a suitable link by using all criteria, it determines that no link is suitable and returns the link selection result.

5.     If no suitable link is found, the device performs forwarding based on the routing table. If a suitable link is found, RIR forwards the packet based on the link selection result.

After finishing link selection, the device forwards the subsequent packets of the same session based on the optimal link information.

Figure 6 RIR link selection workflow

 

Preference-based primary link selection

RIR preferentially selects primary links that meet both the quality and bandwidth requirements for a service flow that matches a flow template. As shown in Figure 7, the device selects a primary link from the primary links in the flow template by examining the links in descending order of link preference. The device uses the following process to examine links with the same preference:

1.     The device examines all links with the preference and identifies whether a link forms ECMP routes with other links. If a link forms ECMP routes with other links, the device further identifies whether the link is a primary link that meets both the quality and bandwidth requirements of the service.

¡     If yes, the device adds the link to the available suitable link list of that preference.

¡     If no, the device further identifies whether the link is a primary link that meets the bandwidth requirements of the service.

-     If yes, the device adds the link to the quality tolerant link list for quality tolerant link selection. Then, the device continues to examine other links with the same preference.

-     If no, the device continues to examine other links with the same preference.

If a link does not form ECMP routes with other links, the device continues to examine other links with the same preference.

2.     When the device finishes examining all links with the preference, it identifies how many suitable links are available for the service flow.

¡     If only one suitable link is available, the device selects that link as the optimal link.

¡     If multiple suitable links are available, a link is selected as the optimal link using the UCMP load sharing algorithm based on the remaining bandwidth of the link. The link with the greater remaining bandwidth has a higher probability of being selected.

¡     If no suitable link is available, the device examines the links that have a preference value lower than the links with the current preference.

If no primary links in the flow template are suitable, the device determines that no optimal primary link is found for the service flow.

For more information about identifying whether a link meets the quality requirements, see "RIR quality policy." For more information about identifying whether a link meets the bandwidth requirements, see "Bandwidth-based link selection policy."

Figure 7 Preference-based primary link selection workflow

 

Preference-based backup link selection

If no primary link is suitable for a service flow, the device tries to find a backup link. The backup link must meet both the quality and bandwidth requirements for the flow in the flow template. The link selection process is the same as that for selecting a primary link.

Quality tolerant link selection

If preference-based primary link selection and preference-based backup link selection fail to select a suitable link, the device performs quality tolerant link selection.

The links that meet the quality tolerant link selection criterion are those added to the quality tolerant link list during preference-based primary and backup link selection. These links do not meet the quality requirements of the service, but they meet the bandwidth requirements of the service. Quality tolerant link selection selects a link from the links that meet only the bandwidth requirements of the service.

During quality tolerant link selection for service traffic, if there are multiple available links, the device will select an optimal link from the available links using the UCMP load sharing algorithm based on the remaining bandwidth of the links. The higher the remaining bandwidth, the higher the probability of a link being selected.

Bandwidth tolerant link selection

If quality tolerant link selection still cannot find a suitable link for a service flow, the device performs bandwidth tolerant link selection. Bandwidth tolerant link selection selects one link from ECMP routes in the flow template as the optimal link.

When allocating bandwidth for service traffic, if there are multiple optional links available, the device will select an optimal link from these options based on the total bandwidth using the UCMP load sharing algorithm. The greater the total bandwidth, the higher the probability of a link being chosen.

Link selection delay and link selection suppression period

To improve packet forwarding efficiency, the device does not repeatedly perform link selection for traffic of the same session. After the device performs link selection for traffic of a session, it forwards the subsequent traffic of that session according to the previous link selection result. Link reselection is triggered when any link in the session's flow template has one of the following changes:

·     The quality of a link becomes qualified from unqualified or the quality of a link becomes unqualified from qualified.

·     The bandwidth usage of a link has reached its maximum bandwidth.

To avoid frequent link selection caused by link flapping, RIR defines a link selection delay and link selection suppression period.

After the device performs link selection, it starts the link selection suppression period if the period has been configured. Within the link selection suppression period, the device does not perform link reselection, but it maintains the link state data. When the link selection suppression period ends, the link selection delay timer starts. If the link state still meets the conditions that can trigger link reselection when the delay timer expires, the device performs link reselection. If the link state changes to not meet the conditions that can trigger link reselection within the delay time, the device does not perform link reselection.

Typical network applications

The typical applications of RIR typically include the enterprise's multiple link optimization and multiple link load sharing. Enterprises can adjust according to actual service requirements and link conditions.

RIR-based multilink optimization

A company might lease both the carrier's MPLS dedicated line and the ordinary Internet network. The MPLS is more costly, while the Internet network is cheaper. As shown in Figure 8, by configuring different link selection policies for ordinary and critical tasks, when both links are functioning properly, critical services can be transmitted via the MPLS dedicated line, and ordinary tasks can be transmitted through the Internet network. If one link degrades or fails, services can select the other link for transmission. This deployment allows the company to ensure both the user experience for critical services and the normal transmission of ordinary services at a relatively affordable price.

Figure 8 RIR-based multilink optimization

 

RIR-based load sharing

An enterprise might lease multiple Internet networks from the same or different carriers, with little difference in lease prices, all capable of meeting regular service requirements. As shown in Figure 9, during RIR deployment, the same link preference can be defined for these links based on regular service, allowing traffic to be distributed and forwarded across multiple links, preventing some links from being idle or overloaded.

Figure 9 RIR-based load sharing