Contents

Configuring graceful insertion and removal 1

About graceful insertion and removal 1

GIR modes· 1

How GIR works· 1

GIR snapshots· 2

GIR tasks at a glance· 3

Configuring a custom profile· 3

Switching to maintenance mode· 5

Switching back to normal mode· 6

Manually switching back to normal mode· 6

Configuring GIR to automatically switch back to normal mode· 7

Managing GIR snapshots· 7

Manually creating a snapshot 7

Manually deleting snapshots· 8

Including additional data in snapshots· 8

Configuring advanced GIR features· 8

Configuring the delay for the device to enter maintenance mode· 8

Specifying a delay for after_maintenance snapshot creation· 9

Display and maintenance commands for GIR· 9

GIR configuration examples· 9

Example: Configuring GIR in an ECMP-based node redundancy environment 9

 

Configuring graceful insertion and removal

About graceful insertion and removal

Graceful insertion and removal (GIR) enables you to gracefully isolate the device from the network for device maintenance or upgrade. GIR minimizes service interruption by instructing the affected protocols (for example, routing protocols) to isolate the device and switch over to the redundant path. You do not need to configure graceful switchover protocol by protocol.

GIR modes

The device can operate in either of the following GIR modes:

·     Normal mode—The device can forward traffic.

·     Maintenance mode—The device is gracefully placed out of service for maintenance or upgrade purposes. In this mode, the device cannot forward traffic.

How GIR works

Overview

When you place the device in maintenance mode, GIR instructs each affected protocol (for example, BGP) to remove the device from the network. Then, each protocol takes action as instructed to isolate the device and switch over to the redundant path. When you change the device back to normal mode, GIR instructs the affected protocols to add the device back to the network. These protocols are called GIR-capable protocols.

To record the configuration before and after a GIR operation, GIR takes a snapshot of protocol data when the device changes to maintenance mode and when the device changes back to normal mode. You can compare the snapshots to verify configuration data consistency for each protocol.

GIR supports the following protocols: LACP, BGP, IS-IS, OSPF, and OSPFv3. For more information about how each protocol isolates the device, see their respective configuration guides.

Protocol isolation methods

GIR-capable protocols typically use the isolate and shutdown methods to isolate the device from the network. The following information uses OSPF for example to describe the isolation procedures.

·     Isolate switching:

a.     When you place the device in maintenance mode, GIR executes the isolate enable command for OSPF to increase the link cost in the LSAs advertised by the device.

b.     Each OSPF neighbor of the device reselects an optimal route based on the LSAs and stops forwarding traffic to the device. After the device is fully isolated from the network, you can upgrade the device.

c.     When you revert the device back to normal mode, GIR executes the undo isolate enable command for OSPF to restore its original link cost and gracefully add the device back to the network.

·     Shutdown method:

a.     After a process is shut down by using the shutdown process command, it sends 1-way hello packets to its neighbors

b.     On receipt of the packets, the neighbors disconnect from the OSPF process and use the backup path to forward traffic.

c.     The process stops receiving and sending OSPF packets.

d.     The process clears its neighbor, LSDB, and OSPF route information.

Compared with the shutdown method, the isolate method uses more resources, but it can faster restore a protocol to the state before the isolation. For more information about the isolate enable and shutdown process commands for OSPF, see OSPF commands in Layer 3—IP Routing Command Reference.

GIR mode switching

GIR supports the following methods for you to change the device from normal mode to maintenance mode:

·     Isolate switching—By default, GIR performs isolate switching to change the device from normal mode to maintenance mode if you do not specify shutdown or profile-based switching. With isolate switching, GIR uses the isolation commands for each GIR-capable protocol to isolate the device.

·     Shutdown switching—GIR uses the shutdown method for each GIR-aware protocol to isolate the device when it changes the device from normal mode to maintenance mode. With this method, GIR also shuts down all interfaces except the management port.

·     Profile-based switching—GIR executes the custom profile when it changes the system from normal mode to maintenance mode.

 

NOTE: A custom profile contains a series of user-specified commands to be executed in sequence upon a mode switching. With GIR, you can configure one switch-to-maintenance profile and one switch-to-normal profile for changing to maintenance mode and normal mode, respectively.

 

When GIR performs an isolate or shutdown switching, it automatically writes the executed series of commands to the switch-to-maintenance custom profile if the profile is empty.

GIR supports the following methods for you to change the device from maintenance mode back to normal mode:

·     Profile-based switching—Configure GIR to execute a custom profile for mode switching.

·     Reverse switching—Configure GIR to perform operations in reverse to the operations performed when it changed from normal mode to maintenance mode.

When GIR performs a reverse switching, it automatically writes the executed series of commands to the switch-to-normal custom profile if the profile is empty.

 

NOTE: Automatic maintenance-to-normal mode switching is always reverse switching. Manual maintenance-to-normal mode switching can be profile-based or reverse switching.

 

GIR snapshots

A GIR snapshot records the running data of the device in NETCONF format at a point in time for comparison of data to identify changes over time.

GIR snapshots include automatically generated snapshots and manually created snapshots.

The device automatically generates in the following situations:

·     When the device enters maintenance mode, it generates a GIR snapshot named before_maintenance before each protocol switches traffic over to the redundant path.

·     When the device returns to normal mode, it generates a GIR snapshot named after_maintenance after all protocols insert the device back into the traffic path and the system enters the stable state. You can use the display system stable state command to verify that the system is stable.

The device stores only the automatically generated snapshots for the most recent GIR operation.

With GIR, you can compare snapshots to identify data changes over a period of time. For example, you can identify the configuration differences of the device before and after a GIR operation by comparing the two automatically snapshots.

GIR tasks at a glance

To configure GIR, perform the following tasks:

1.     Configuring a custom profile

To perform profile-based switching, configure custom profiles.

2.     Perform a mode switching operation

a.     Switching to maintenance mode

b.     Switching back to normal mode

3.     Managing GIR snapshots

¡     Manually creating a snapshot

¡     Manually deleting snapshot

¡     Including additional data in snapshots

4.     Configuring advanced GIR features

¡     Configuring the delay for the device to enter maintenance mode

¡     Specifying a delay for after_maintenance snapshot creation

Configuring a custom profile

About this task

By default, GIR performs isolate switching to change the device from normal mode to maintenance mode. To customize the commands to execute and the interfaces to shut down for a profile-based mode switching, perform this task.

A custom profile contains a series of user-specified commands to be executed in sequence upon a mode switching.

With GIR, you can configure one switch-to-maintenance profile and one switch-to-normal profile for changing to maintenance mode and normal mode, respectively.

For more information about mode switching with GIR, see "How GIR works."

Restrictions and guidelines

CAUTION: When you configure a custom profile, make sure you understand the impact of each command on the device and enter the commands in their correct form. Misconfiguration might cause traffic switchover failure.

 

When you edit a custom profile, use the following guidelines:

·     You can only append commands to the end of a custom profile. You cannot remove or modify commands in a custom profile or insert a command between commands in the profile, except that you can edit the sleep instance settings.

As a best practice, use the following steps to configure a custom profile:

a.     Add the series of commands to be executed to a text file.

b.     Verify that the commands are correct in form and order.

c.     Copy the commands to the custom profile in custom profile view.

To modify a custom profile:

a.     Edit the command file.

b.     Delete the custom profile.

c.     Copy the commands to the custom profile in custom profile view.

·     The system executes the commands in a custom profile in their configuration order. If one command depends on another, enter the prerequisite command first.

·     Do not add commands in user view to custom profiles. The system can execute only commands in system view or subviews of system view.

·     By default, the system executes the commands in a custom profile in system view. To execute a command in a subview of system view, you must add the command used to access that subview before you add the command to execute in that subview.

·     If it takes some time for a command to take effect after it is executed, you can add the sleep instance command next to it to introduce a delay. The system will wait for the delay to expire before it executes subsequent commands. If you are not sure whether a command requires a delay, add the sleep instance 0 command next to it. Then, you can change the value after you determine that a delay is required.

·     If you have inadvertently entered an incorrect command, you can add its undo form to remove it as long as that command is reversible by simply adding the undo keyword before it. Then, the system will automatically delete that command and its undo form from the profile. You cannot use this method to remove a command if it does not have an undo form or if the command and its undo form is not symmetric.

·     The system provides the following mechanisms for profile optimization:

¡     Deduplication mechanism—If you enter the same command line multiple times, the system automatically removes the duplicate commands. You do not need to do that manually.

¡     Command grouping—The system groups commands in the same view together and then executes them in configuration order.

·     Verify the settings in the custom profiles by executing the display gir custom-profile command. If you introduce irreversible misconfiguration items in a custom profile, delete and then reconfigure that profile.

·     If you need to slightly optimize the series of commands used when the system changes to maintenance or normal mode, use the following method:

a.     Make sure the switch-to-maintenance or switch-to-normal custom profile is empty.

b.     Execute an isolate or shutdown mode switching.

The system automatically populates the custom profile with the series of commands executed on mode switching.

c.     Edit the custom profile.

·     Key-related commands are not supported in a custom profile.

·     The storage space for the configuration file of commands in a custom profile is limited. When a new command is executed and the storage space is insufficient, the command fails to be executed and no other commands can be executed. As a best practice, do not execute irrelevant commands in a custom profile.

Configuring the switch-to-maintenance profile

1.     Enter system view.

system-view

2.     Enter the view of the switch-to-maintenance profile.

gir custom-profile maintenance-mode

3.     Enter isolation commands.

For isolation commands, see the command reference for each service module.

4.     End the configuration and quit the view.

config-end

Configuring the switch-to-normal profile

1.     Enter system view.

system-view

2.     Enter the view of the switch-to-normal profile.

gir custom-profile normal-mode

3.     Enter the commands to remove the isolation.

For information about the commands, see the command reference for each service module.

4.     End the configuration and quit the view.

config-end

Switching to maintenance mode

About this task

Perform this task before you maintain or upgrade the device.

The device will return to normal mode if a service module fails in switchover. The device will display the commands that failed to be executed or generate logs, but it will not roll back the commands that have been executed.

Restrictions and guidelines

CAUTION: To maintain configuration consistency for a feature across mode switching, do not modify its configuration after you place the device in maintenance mode. To check for configuration changes, execute the display gir snapshot compare command after you revert the device back to normal mode.

 

Prerequisites

If the device is performing a snapshot operation, you must wait for the operation to end before you can change the GIR mode. Conversely, you cannot perform a snapshot operation while the device is changing the GIR mode.

Procedure

1.     Verify that the system is stable.

display system stable state

The system is stable if the System State field displays Stable. For a successful GIR operation, you must make sure the system is stable.

2.     Enter system view.

system-view

3.     Switch to maintenance mode.

gir system-mode maintenance [ shutdown | custom-profile ] [ non-interactive ]

 

CAUTION: ·     Packet loss might occur during mode switching. ·     Do not restart the device or a process while GIR is switching mode. The system cannot roll back settings issued during mode switching. If you restart the device or a disruptive process, mode switching will be interrupted, with only some of the commands issued.

 

4.     If a service module fails in switchover, use the following methods to identify and remove the issue:

¡     Remove the issue based on the error message.

¡     Compare the snapshots to identify configuration inconsistencies that might have caused the issue.

Identify snapshot information:

display gir snapshot

Compare the snapshots:

display gir snapshot compare snapshot1-name snapshot2-name [ verbose ]

5.     Verify that no traffic traverses the device before you maintain or upgrade the device.

display counters

This command is available in any view. This step prevents the packet loss that might occur because a protocol is slow or fails in switchover.

Switching back to normal mode

Manually switching back to normal mode

About this task

Perform this task after you maintain or upgrade the device.

The device will return to maintenance mode if a service module fails in switchover. The device will display the commands that failed to be executed or generate logs, but it will not roll back the commands that have been executed.

Prerequisites

If the device is performing a snapshot operation, you must wait for the operation to end before you can change the GIR mode. Conversely, you cannot perform a snapshot operation while the device is changing the GIR mode.

Procedure

1.     Verify that the system is stable.

display system stable state

The system is stable if the System State field displays Stable.

2.     Enter system view.

system-view

3.     Switch back to normal mode.

undo gir system-mode maintenance [ custom-profile ] [ non-interactive ]

 

CAUTION: Do not restart the device or a process while GIR is switching mode. The system cannot roll back settings issued during mode switching. If you restart the device or a disruptive process, mode switching will be interrupted, with only some of the commands issued.

 

4.     If a service module fails in switchover, use the following methods to identify and remove the issue:

¡     Remove the issue based on the error message.

¡     Compare the snapshots to identify configuration inconsistencies that might cause the issue.

Identify the snapshot names:

display gir snapshot

Compare the snapshots:

display gir snapshot compare snapshot1-name snapshot2-name [ verbose ]

Configuring GIR to automatically switch back to normal mode

About this task

Automatic reverse switching enables the system to automatically revert back to normal mode from maintenance mode when the specified revert-delay expires.

This feature is helpful when the window of maintenance is deterministic.

Procedure

1.     Enter system view.

system-view

2.     Specify the delay for the system to automatically revert to normal mode.

gir system-mode maintenance revert-delay value

By default, the system does not automatically revert to normal mode.

Managing GIR snapshots

Manually creating a snapshot

About this task

A GIR snapshot records the running data of the device in NETCONF format at a point in time for system restoration or comparison of data  to identify changes over time.

Restrictions and guidelines

If the device is performing a mode switching or snapshot operation, you must wait for the operation to end before you can create a snapshot.

Procedure

1.     Enter system view.

system-view

2.     Create a snapshot.

gir snapshot create snapshot-name description

The snapshot name can contain only digits, letters, and underscores (_).

Manually deleting snapshots

About this task

Perform this task to delete earliest snapshots to free up memory space.

Restrictions and guidelines

If the device is performing a mode switching or snapshot operation, you must wait for the operation to end before you can delete a snapshot.

Procedure

1.     Enter system view.

system-view

2.     Delete snapshots.

gir snapshot delete { all | snapshot-name }

Including additional data in snapshots

About this task

By default, the system does not include routing information in created snapshots, because the routing table is large in size. Perform this task to include IPv4 or IPv6 routes in created snapshots.

Procedure

1.     Enter system view.

system-view

2.     Include IPv4 or IPv6 routes in snapshots.

gir snapshot add { ipv4routes | ipv6routes }*

By default, the system does not include routing information in created snapshots.

Configuring advanced GIR features

Configuring the delay for the device to enter maintenance mode

About this task

Execution of some commands might take some time. To ensure that all GIR applicable service modules complete traffic path switchover before you maintain the device, GIR introduces a maintenance delay for the device to enter maintenance mode.

Increase the maintenance delay if its default value is not sufficient.

Procedure

1.     Enter system view.

system-view

2.     Set the maintenance delay.

gir system-mode maintenance maintenance-delay value

By default, the maintenance delay is 120 seconds.

Specifying a delay for after_maintenance snapshot creation

About this task

If the default snapshot delay value is not sufficient for a service module to restore its services, increase the delay so that GIR can take an after_maintenance snapshot of complete and accurate data.

Procedure

1.     Enter system view.

system-view

2.     Specify a delay for after_maintenance snapshot creation.

gir system-mode maintenance snapshot-delay value

By default, the delay for creation of after_maintenance snapshots is 120 seconds.

Display and maintenance commands for GIR

Execute display commands in any view.

 

Task	Command
Display all GIR snapshots on the device.	display gir snapshot
Compare GIR snapshots to view their differences.	display gir snapshot compare snapshot1-name snapshot2-name [ verbose ]
Display the current GIR mode.	display gir system-mode
Display the GIR custom profiles.	display gir custom-profile [ maintenance-mode | normal-mode ]

‌

GIR configuration examples

Example: Configuring GIR in an ECMP-based node redundancy environment

Network configuration

As shown in Figure 1, Device A and Device B have ECMP routes for traffic between Internet and Device C. Before upgrading Device A, use GIR to place it in maintenance mode to prevent packet losses during upgrade. Then, change Device A back to normal mode after its software is upgraded.

Figure 1 Network diagram

‌

Procedure

1.     Configure IP addresses and a unicast routing protocol (for example, OSPF). Make sure ECMP routes are available for traffic between the Internet and Device C to traverse Device A or Device B to reach the destination. (Details not shown.)

2.     Isolate Device A from the network before you upgrade it.

# Change Device A from the normal mode to the maintenance mode.

<Device A> display system stable state summary

System state      : Stable

Redundancy state  : No redundance

NSR     state     : No standby

<Device A> system-view

[Device A] gir system-mode maintenance

Collecting commands... Please wait.

Configuration to be applied:

   ospf 1

     isolate enable

Do you want to continue? [Y/N]: y

Generated a snapshot: before_maintenance.

Applying: ospf 1

Applying: isolate enable

Waiting 120 seconds to release the CLI.

Changed to maintenance mode successfully.

# Verify that the mode switching is successful on Device A.

[Device A] display gir system-mode

System Mode: Maintenance

Revert-delay time: 0min

Maintenance-delay time: 120s

Snapshot-delay time: 120s

# Verify that traffic switching has completed on Device A.

[Device A] display counters rate inbound interface

Usage: Bandwidth utilization in percentage

Interface            Usage (%)   Total (pps)   Broadcast (pps)   Multicast (pps)

XGE3/1/1                     0             0                --                --

XGE3/1/2                     0             0                --                --

                                                                               

 Overflow: More than 14 digits.

       --: Not supported.

3.     Upgrade Device A. (Details not shown.)

4.     Restore forwarding services on Device A:

# Change Device A from the maintenance mode back to the normal mode after it is upgraded and rebooted.

<Device A> system-view

[Device A] undo gir system-mode maintenance

Collecting commands... Please wait.

Configuration to be applied:

   ospf 1

     undo isolate enable

Do you want to continue? [Y/N]: y

Applying: ospf 1

Applying: undo isolate enable

Waiting 120 seconds to generate a snapshot.

Generated a snapshot: after_maintenance.

Changed to normal mode successfully.

# Verify that the mode switching is successful on Device A.

[Device A] display gir system-mode

System Mode: Normal

Revert-delay time: 0min

Maintenance-delay time: 120s

Snapshot-delay time: 120s

# Verify that traffic switching has completed on Device A.

[Device A] display counters rate inbound interface

Usage: Bandwidth utilization in percentage

Interface            Usage (%)   Total (pps)   Broadcast (pps)   Multicast (pps)

XGE3/1/1                   100           100                --                --

XGE3/1/2                   100           100                --                --

                                                                               

 Overflow: More than 14 digits.

       --: Not supported.