11-NMM Configuration Guide

HomeSupportResource CenterH3C S6800[S6860][S6861] (R27xx) & S6820 (R630x) Switch Series Configuration Guide-6W10111-NMM Configuration Guide
13-EAA configuration
Title Size Download
13-EAA configuration 146.78 KB

Configuring EAA

About EAA

Embedded Automation Architecture (EAA) is a monitoring framework that enables you to self-define monitored events and actions to take in response to an event. It allows you to create monitor policies by using the CLI or Tcl scripts.

EAA framework

EAA framework includes a set of event sources, a set of event monitors, a real-time event manager (RTM), and a set of user-defined monitor policies, as shown in Figure 1.

Figure 1 EAA framework

Event sources

Event sources are software or hardware modules that trigger events (see Figure 1).

For example, the CLI module triggers an event when you enter a command. The Syslog module (the information center) triggers an event when it receives a log message.

Event monitors

EAA creates one event monitor to monitor the system for the event specified in each monitor policy. An event monitor notifies the RTM to run the monitor policy when the monitored event occurs.

RTM

RTM manages the creation, state machine, and execution of monitor policies.

EAA monitor policies

A monitor policy specifies the event to monitor and actions to take when the event occurs.

You can configure EAA monitor policies by using the CLI or Tcl.

A monitor policy contains the following elements:

·     One event.

·     A minimum of one action.

·     A minimum of one user role.

·     One running time setting.

For more information about these elements, see "Elements in a monitor policy."

Elements in a monitor policy

Elements in an EAA monitor policy include event, action, user role, and runtime.

Event

Table 1 shows types of events that EAA can monitor.

Table 1 Monitored events

Event type

Description

CLI

CLI event occurs in response to monitored operations performed at the CLI. For example, a command is entered, a question mark (?) is entered, or the Tab key is pressed to complete a command.

Syslog

Syslog event occurs when the information center receives the monitored log within a specific period.

NOTE:

The log that is generated by the EAA RTM does not trigger the monitor policy to run.

Process

Process event occurs in response to a state change of the monitored process (such as an exception, shutdown, start, or restart). Both manual and automatic state changes can cause the event to occur.

Hotplug

Hot-swapping event occurs when the monitored member device joins or leaves the IRF fabric or a card is inserted in or removed from the monitored slot.

Interface

Each interface event is associated with two user-defined thresholds: start and restart.

An interface event occurs when the monitored interface traffic statistic crosses the start threshold in the following situations:

·     The statistic crosses the start threshold for the first time.

·     The statistic crosses the start threshold each time after it crosses the restart threshold.

SNMP

Each SNMP event is associated with two user-defined thresholds: start and restart.

SNMP event occurs when the monitored MIB variable's value crosses the start threshold in the following situations:

·     The monitored variable's value crosses the start threshold for the first time.

·     The monitored variable's value crosses the start threshold each time after it crosses the restart threshold.

SNMP-Notification

SNMP-Notification event occurs when the monitored MIB variable's value in an SNMP notification matches the specified condition. For example, the broadcast traffic rate on an Ethernet interface reaches or exceeds 30%.

Track

Track event occurs when the state of the track entry changes from Positive to Negative or from Negative to Positive. If you specify multiple track entries for a policy, EAA triggers the policy only when the state of all the track entries changes from Positive (Negative) to Negative (Positive).

If you set a suppress time for a policy, the timer starts when the policy is triggered. The system does not process the messages that report the track entry state change from  Positive (Negative) to Negative (Positive) until the timer times out.

Action

You can create a series of order-dependent actions to take in response to the event specified in the monitor policy.

The following are available actions:

·     Executing a command.

·     Sending a log.

·     Enabling an active/standby switchover.

·     Executing a reboot without saving the running configuration.

User role

For EAA to execute an action in a monitor policy, you must assign the policy the user role that has access to the action-specific commands and resources. If EAA lacks access to an action-specific command or resource, EAA does not perform the action and all the subsequent actions.

For example, a monitor policy has four actions numbered from 1 to 4. The policy has user roles that are required for performing actions 1, 3, and 4. However, it does not have the user role required for performing action 2. When the policy is triggered, EAA executes only action 1.

For more information about user roles, see RBAC in Fundamentals Configuration Guide.

Runtime

The runtime limits the amount of time that the monitor policy runs its actions from the time it is triggered. This setting prevents a policy from running its actions permanently to occupy resources.

EAA environment variables

EAA environment variables decouple the configuration of action arguments from the monitor policy so you can modify a policy easily.

An EAA environment variable is defined as a <variable_name variable_value> pair and can be used in different policies. When you define an action, you can enter a variable name with a leading dollar sign ($variable_name). EAA will replace the variable name with the variable value when it performs the action.

To change the value for an action argument, modify the value specified in the variable pair instead of editing each affected monitor policy.

EAA environment variables include system-defined variables and user-defined variables.

System-defined variables

System-defined variables are provided by default, and they cannot be created, deleted, or modified by users. System-defined variable names start with an underscore (_) sign. The variable values are set automatically depending on the event setting in the policy that uses the variables.

System-defined variables include the following types:

·     Public variable—Available for any events.

·     Event-specific variable—Available only for a type of event. The hotplug event-specific variables are _slot and _subslot. When a member device in slot 1 joins or leaves the IRF fabric, the value of _slot is 1. When a member device in slot 2 joins or leaves the IRF fabric, the value of _slot is 2.

Table 2 shows all system-defined variables.

Table 2 System-defined EAA environment variables by event type

Event

Variable name and description

Any event

_event_id: Event ID

_event_type: Event type

_event_type_string: Event type description

_event_time: Time when the event occurs

_event_severity: Severity level of an event

CLI

_cmd: Commands that are matched

Syslog

_syslog_pattern: Log message content

Hotplug

_slot: ID of the member device that joins or leaves the IRF fabric

Hotplug

_subslot: ID of the subslot where subcard hot-swapping occurs.

Interface

_ifname: Interface name

SNMP

_oid: OID of the MIB variable where an SNMP operation is performed

_oid_value: Value of the MIB variable

SNMP-Notification

_oid: OID that is included in the SNMP notification.

Process

_process_name: Process name

 

User-defined variables

You can use user-defined variables for all types of events.

User-defined variable names can contain digits, characters, and the underscore sign (_), except that the underscore sign cannot be the leading character.

Configuring a user-defined EAA environment variable

About configuring a user-defined EAA environment variable

Configure user-defined EAA environment variables so that you can use them when creating EAA monitor policies.

Procedure

1.     Enter system view.

system-view

2.     Configure a user-defined EAA environment variable.

rtm environment var-name var-value

For the system-defined variables, see Table 2.

Configuring a monitor policy

Restrictions and guidelines

Make sure the actions in different policies do not conflict. Policy execution result will be unpredictable if policies that conflict in actions are running concurrently.

You can assign the same policy name to a CLI-defined policy and a Tcl-defined policy. However, you cannot assign the same name to policies that are the same type.

A monitor policy supports only one event and runtime. If you configure multiple events for a policy, the most recent one takes effect.

A monitor policy supports a maximum of 64 valid user roles. User roles added after this limit is reached do not take effect.

Configuring a monitor policy from the CLI

Restrictions and guidelines

You can configure a series of actions to be executed in response to the event specified in a monitor policy. EAA executes the actions in ascending order of action IDs. When you add actions to a policy, you must make sure the execution order is correct. If two actions have the same ID, the most recent one takes effect.

Procedure

1.     Enter system view.

system-view

2.     (Optional.) Set the size for the EAA-monitored log buffer.

rtm event syslog buffer-size buffer-size

By default, the EAA-monitored log buffer stores a maximum of 50000 logs

3.     Create a CLI-defined policy and enter its view.

rtm cli-policy policy-name

4.     Configure an event for the policy.

¡     Configure a CLI event.

event cli { async [ skip ] | sync } mode { execute | help | tab } pattern regular-exp

¡     Configure a hotplug event.

event hotplug [ insert | remove ] slot slot-number [ subslot subslot-number ]

¡     Configure an interface event.

event interface interface-list monitor-obj monitor-obj start-op start-op start-val start-val restart-op restart-op restart-val restart-val [ interval interval ]

¡     Configure a process event.

event process { exception | restart | shutdown | start } [ name process-name [ instance instance-id ] ] [ slot slot-number ]

¡     Configure an SNMP event.

event snmp oid oid monitor-obj { get | next } start-op start-op start-val start-val restart-op restart-op restart-val restart-val [ interval interval ]

¡     Configure an SNMP-Notification event.

event snmp-notification oid oid oid-val oid-val op op [ drop ]

¡     Configure a Syslog event.

event syslog priority priority msg msg occurs times period period

¡     Configure a track event.

event track track-list state { negative | positive } [ suppress-time suppress-time ]

By default, a monitor policy does not contain an event.

If you configure multiple events for a policy, the most recent one takes effect.

5.     Configure the actions to take when the event occurs.

Choose the following tasks as needed:

¡     Configure a CLI action.

action number cli command-line

¡     Configure a reboot action.

action number reboot [ slot slot-number ]

¡     Configure an active/standby switchover action.

action number switchover

¡     Configure a logging action.

action number syslog priority priority facility local-number msg msg-body

By default, a monitor policy does not contain any actions.

6.     (Optional.) Assign a user role to the policy.

user-role role-name

By default, a monitor policy contains user roles that its creator had at the time of policy creation.

An EAA policy cannot have both the security-audit user role and any other user roles. Any previously assigned user roles are automatically removed when you assign the security-audit user role to the policy. The previously assigned security-audit user role is automatically removed when you assign any other user roles to the policy.

7.     (Optional.) Configure the policy action runtime.

running-time time

The default policy action runtime is 20 seconds.

If you configure multiple action runtimes for a policy, the most recent one takes effect.

8.     Enable the policy.

commit

By default, CLI-defined policies are not enabled.

A CLI-defined policy can take effect only after you perform this step.

Configuring a monitor policy by using Tcl

About Tcl scripts

A Tcl script contains two parts: Line 1 and the other lines.

·     Line 1

Line 1 defines the event, user roles, and policy action runtime. After you create and enable a Tcl monitor policy, the device immediately parses, delivers, and executes Line 1.

Line 1 must use the following format:

::platformtools::rtm::event_register event-type arg1 arg2 arg3user-role role-name1 | [ user-role role-name2 | [ ] ] [ running-time running-time ]

¡     The arg1 arg2 arg3 … arguments represent event matching rules. If an argument value contains spaces, use double quotation marks ("") to enclose the value. For example, "a b c."

¡     The configuration requirements for the event-type, user-role, and running-time arguments are the same as those for a CLI-defined monitor policy.

·     The other lines

From the second line, the Tcl script defines the actions to be executed when the monitor policy is triggered. You can use multiple lines to define multiple actions. The system executes these actions in sequence. The following actions are available:

¡     Standard Tcl commands.

¡     EAA-specific Tcl actions:

-     switchover ( ::platformtools::rtm::action switchover )

-     syslog (::platformtools::rtm::action syslog priority priority facility local-number msg msg-body). For more information about these arguments, see EAA commands in Network Management and Monitoring Command Reference.

¡     Commands supported by the device.

Restrictions and guidelines

To revise the Tcl script of a policy, you must suspend all monitor policies first, and then resume the policies after you finish revising the script. The system cannot execute a Tcl-defined policy if you edit its Tcl script without first suspending these policies.

Procedure

1.     Download the Tcl script file to the device by using FTP or TFTP.

For more information about using FTP and TFTP, see Fundamentals Configuration Guide.

2.     Create and enable a Tcl monitor policy.

a.     Enter system view.

system-view

b.     Create a Tcl-defined policy and bind it to the Tcl script file.

rtm tcl-policy policy-name tcl-filename

By default, no Tcl policies exist.

Make sure the script file is saved on all IRF member devices. This practice ensures that the policy can run correctly after a master/subordinate switchover occurs or the member device where the script file resides leaves the IRF.

Suspending monitor policies

About suspending monitor policies

This task suspends all CLI-defined and Tcl-defined monitor policies. If a policy is running when you perform this task, the system suspends the policy after it executes all the actions.

Restrictions and guidelines

To restore the operation of the suspended policies, execute the undo rtm scheduler suspend command.

Procedure

1.     Enter system view.

system-view

2.     Suspend monitor policies.

rtm scheduler suspend

Display and maintenance commands for EAA

Execute display commands except for the display this command in any view.

 

Task

Command

Display the running configuration of all CLI-defined monitor policies.

display current-configuration

Display user-defined EAA environment variables.

display rtm environment [ var-name ]

Display EAA monitor policies.

display rtm policy { active | registered [ verbose ] } [ policy-name ]

Display the running configuration of a CLI-defined monitor policy in CLI-defined monitor policy view.

display this

EAA configuration examples

Example: Configuring a CLI event monitor policy by using Tcl

Network configuration

As shown in Figure 2, use Tcl to create a monitor policy on the Device. This policy must meet the following requirements:

·     EAA sends the log message "rtm_tcl_test is running" when a command that contains the display this string is entered.

·     The system executes the command only after it executes the policy successfully.

Figure 2 Network diagram

Procedure

# Edit a Tcl script file (rtm_tcl_test.tcl, in this example) for EAA to send the message "rtm_tcl_test is running" when a command that contains the display this string is executed.

::platformtools::rtm::event_register cli sync mode execute pattern display this user-role network-admin

::platformtools::rtm::action syslog priority 1 facility local4 msg rtm_tcl_test is running

# Download the Tcl script file from the TFTP server at 1.2.1.1.

<Sysname> tftp 1.2.1.1 get rtm_tcl_test.tcl

# Create Tcl-defined policy test and bind it to the Tcl script file.

<Sysname> system-view

[Sysname] rtm tcl-policy test rtm_tcl_test.tcl

[Sysname] quit

Verifying the configuration

# Display information about the policy.

<Sysname> display rtm policy registered

Total number: 1

Type  Event             TimeRegistered       PolicyName

TCL   CLI               Jan 01 09:47:12 2019 test

# Enable the information center to output log messages to the current monitoring terminal.

<Sysname> terminal monitor

The current terminal is enabled to display logs.

<Sysname> system-view

[Sysname] info-center enable

Information center is enabled.

[Sysname] quit

# Execute the display this command. Verify that the system displays the rtm_tcl_test is running message and a message that the policy is being successfully executed.

<Sysname> display this

%Jan  1 09:50:04:634 2019 2013 Sysname RTM/1/RTM_ACTION: rtm_tcl_test is running

%Jan  1 09:50:04:636 2019 Sysname RTM/6/RTM_POLICY: TCL policy test is running successfully.

#

return

Example: Configuring a CLI event monitor policy from the CLI

Network configuration

Configure a policy from the CLI to monitor the event that occurs when a question mark (?) is entered at the command line that contains letters and digits.

When the event occurs, the system executes the command and sends the log message "hello world" to the information center.

Procedure

# Create CLI-defined policy test and enter its view.

<Sysname> system-view

[Sysname] rtm cli-policy test

# Add a CLI event that occurs when a question mark (?) is entered at any command line that contains letters and digits.

[Sysname-rtm-test] event cli async mode help pattern [a-zA-Z0-9]

# Add an action that sends the message "hello world" with a priority of 4 from the logging facility local3 when the event occurs.

[Sysname-rtm-test] action 0 syslog priority 4 facility local3 msg “hello world”

# Add an action that enters system view when the event occurs.

[Sysname-rtm-test] action 2 cli system-view

# Add an action that creates VLAN 2 when the event occurs.

[Sysname-rtm-test] action 3 cli vlan 2

# Set the policy action runtime to 2000 seconds.

[Sysname-rtm-test] running-time 2000

# Specify the network-admin user role for executing the policy.

[Sysname-rtm-test] user-role network-admin

# Enable the policy.

[Sysname-rtm-test] commit

Verifying the configuration

# Display information about the policy.

[Sysname-rtm-test] display rtm policy registered

Total number: 1

Type  Event      TimeRegistered       PolicyName

CLI   CLI        Jan  1 14:56:50 2019 test

# Enable the information center to output log messages to the current monitoring terminal.

[Sysname-rtm-test] return

<Sysname> terminal monitor

The current terminal is enabled to display logs.

<Sysname> system-view

[Sysname] info-center enable

Information center is enabled.

[Sysname] quit

# Enter a question mark (?) at a command line that contains a letter d. Verify that the system displays the "hello world" message and a policy successfully executed message on the terminal screen.

<Sysname> d?

  debugging

  delete

  diagnostic-logfile

  dir

  display

 

<Sysname>d%Jan  1 14:57:20:218 2019 Sysname RTM/4/RTM_ACTION: "hello world"

%Jan  1 14:58:11:170 2019 Sysname RTM/6/RTM_POLICY: CLI policy test is running successfully.

Example: Configuring a track event monitor policy from the CLI

Network configuration

As shown in Figure 3, Device A has established BGP sessions with Device D and Device E. Traffic from Device D and Device E to the Internet is forwarded through Device A.

Configure a CLI-defined EAA monitor policy on Device A to disconnect the sessions with Device D and Device E when Ten-GigabitEthernet 1/0/1 connected to Device C is down. In this way, traffic from Device D and Device E to the Internet can be forwarded through Device B.

Figure 3 Network diagram

Procedure

 

# Display BGP peer information for Device A.

<DeviceA> display bgp peer ipv4

 

 BGP local router ID: 1.1.1.1

 Local AS number: 100

 Total number of peers: 3                  Peers in established state: 3

 

  * - Dynamically created peer

  Peer                    AS  MsgRcvd  MsgSent OutQ PrefRcv Up/Down  State

 

  10.2.1.2                200       13       16    0       0 00:16:12 Established

  10.3.1.2                300       13       16    0       0 00:10:34 Established

  10.3.2.2                300       13       16    0       0 00:10:38 Established

# Create track entry 1 and associate it with the link state of Ten-GigabitEthernet 1/0/1.

<Device A> system-view

[Device A] track 1 interface ten-gigabitethernet 1/0/1

# Configure a CLI-defined EAA monitor policy so that the system automatically disables session establishment with Device D and Device E when Ten-GigabitEthernet 1/0/1 is down.

[Device A] rtm cli-policy test

[Device A-rtm-test] event track 1 state negative

[Device A-rtm-test] action 0 cli system-view

[Device A-rtm-test] action 1 cli bgp 100

[Device A-rtm-test] action 2 cli peer 10.3.1.2 ignore

[Device A-rtm-test] action 3 cli peer 10.3.2.2 ignore

[Device A-rtm-test] user-role network-admin

[Device A-rtm-test] commit

[Device A-rtm-test] quit

Verifying the configuration

# Shut down Ten-GigabitEthernet 1/0/1.

[Device A] interface ten-gigabitethernet 1/0/1

[Device A-Ten-GigabitEthernet1/0/1] shutdown

# Execute the display bgp peer ipv4 command on Device A to display BGP peer information. If no BGP peer information is displayed, Device A does not have any BGP peers.

Example: Configuring a CLI event monitor policy with EAA environment variables from the CLI

Network configuration

Define an environment variable to match the IP address 1.1.1.1.

Configure a policy from the CLI to monitor the event that occurs when a command line that contains loopback0 is executed. In the policy, use the environment variable for IP address assignment.

When the event occurs, the system performs the following tasks:

·     Creates the Loopback 0 interface.

·     Assigns 1.1.1.1/24 to the interface.

·     Sends the matching command line to the information center.

Procedure

# Configure an EAA environment variable for IP address assignment. The variable name is loopback0IP, and the variable value is 1.1.1.1.

<Sysname> system-view

[Sysname] rtm environment loopback0IP 1.1.1.1

# Create the CLI-defined policy test and enter its view.

[Sysname] rtm cli-policy test

# Add a CLI event that occurs when a command line that contains loopback0 is executed.

[Sysname-rtm-test] event cli async mode execute pattern loopback0

# Add an action that enters system view when the event occurs.

[Sysname-rtm-test] action 0 cli system-view

# Add an action that creates the interface Loopback 0 and enters loopback interface view.

[Sysname-rtm-test] action 1 cli interface loopback 0

# Add an action that assigns the IP address 1.1.1.1 to Loopback 0. The loopback0IP variable is used in the action for IP address assignment.

[Sysname-rtm-test] action 2 cli ip address $loopback0IP 24

# Add an action that sends the matching loopback0 command with a priority of 0 from the logging facility local7 when the event occurs.

[Sysname-rtm-test] action 3 syslog priority 0 facility local7 msg $_cmd

# Specify the network-admin user role for executing the policy.

[Sysname-rtm-test] user-role network-admin

# Enable the policy.

[Sysname-rtm-test] commit

[Sysname-rtm-test] return

<Sysname>

Verifying the configuration

# Enable the information center to output log messages to the current monitoring terminal.

<Sysname> terminal monitor

<Sysname> terminal log level debugging

<Sysname> system-view

[Sysname] info-center enable

# Execute the loopback0 command. Verify that the system displays the loopback0 message and a policy successfully executed message on the terminal screen.

[Sysname] interface loopback0

[Sysname-LoopBack0]%Jan  1 09:46:10:592 2019  Sysname RTM/7/RTM_ACTION: interface loopback0

%Jan  1 09:46:10:613 2019 Sysname RTM/6/RTM_POLICY: CLI policy test is running successfully.

# Verify that Loopback 0 has been created and assigned the IP address 1.1.1.1.

[Sysname-LoopBack0] display interface loopback brief

Brief information on interfaces in route mode:

Link: ADM - administratively down; Stby - standby

Protocol: (s) - spoofing

Interface            Link Protocol Primary IP         Description

Loop0                UP   UP(s)    1.1.1.1

 

<Sysname-LoopBack0>