17-Network Management and Monitoring Configuration Guide

HomeSupportConfigure & DeployConfiguration GuidesH3C SecPath M9000 Configuration Guide(V7)(R9900)-6W10017-Network Management and Monitoring Configuration Guide
14-Fast log output configuration
Title Size Download
14-Fast log output configuration 155.68 KB

Fast log output overview

About fast log output

The fast log output feature enables fast output of logs to log hosts. These logs are called fast output logs, or fast logs for short.

Typically, logs generated by a service module are first sent to the information center, which then outputs the logs to the specified destination (such as to log hosts). When fast log output is configured, logs of service modules are sent directly to log hosts instead of to the information center. Compared to outputting logs to the information center, fast log output saves system resources. For more information about the information center, see "Configuring the information center."

Logs are classified into eight severity levels from 0 through 7 in descending order.

Table 1 Log levels

Severity value

Level

Description

0

Emergency

The system is unusable. For example, the system authorization has expired.

1

Alert

Action must be taken immediately. For example, traffic on an interface exceeds the upper limit.

2

Critical

Critical condition. For example, the device temperature exceeds the upper limit, the power module fails, or the fan tray fails.

3

Error

Error condition. For example, the link state changes.

4

Warning

Warning condition. For example, an interface is disconnected, or the memory resources are used up.

5

Notification

Normal but significant condition. For example, a terminal logs in to the device, or the device reboots.

6

Informational

Informational message. For example, a command or a ping operation is executed.

7

Debugging

Debug message.

Log header formats

The log header formats of fast output logs are as follows:

Table 2 Log header formats

Log header types

Format

Standard format

<PRI> Timestamp AppName %%10 SN:sn VsysId:id

Example:

<134> Apr 28 15:35:32 2020 H3C %%10 SN:10056879 VsysId:1

Customized format

URL filtering UNICOM format:

PRI Version HostName Timestamp AppName MsgID HostName Len

Example:

142 1 100.0.0.1 2020 Apr 28 15:35:43 H3C NAT444:SessionU 57

NAT CMCC format:

<PRI> Version HostName Timestamp AppName ProcID MsgID

Example:

<142> 1 - 2020 Apr 28 15:35:32 H3C - NAT444:SessionA

NAT UNICOM format:

<PRI> Version HostName Timestamp AppName ProcID MsgID

Example:

<142> 1 100.0.0.1 2020 Apr 28 15:35:43 H3C - NAT444:SessionA

NAT TELECOM format:

<PRI> Version Timestamp HostName AppName ProcID MsgID

Example:

<134> 1 2020 Apr 28 15:35:38 100.0.0.1 H3C - NAT444:sessionbasedA

Log field description

Table 3 Log field description

Field

Description

PRI

Log type code.

·     Standard format and NAT UNICOM format: 134.

·     URL filtering UNICOM format, NAT CMCC format, and NAT TELECOM format: 142.

Version

Version information, version identifier of log information, which is 1.

Timestamp

Records the time when the log was generated. The timestamp is in the format of YYYY Mon DD hh:mm:ss.

AppName

Name of the device that generated the log.

%%10

Vendor of the device that generated the log.

SN

Serial number of the device that generated the log. To view the device serial number, see the DEVICE_SERIAL_NUMBE field in the output of the display device manuinfo command.

This field is available only when the device is configured to carry the serial number in fast output logs by using the customlog with-sn command.

VsysId

Virtual system that generated the log.

HostName

Source IPv4 address of the device that generated the log. This field is fixed at a hyphen (-) for the NAT CMCC format.

MsgID

Log type.

Len

Total length of the log header, in bytes.

ProcID

Hyphen (-).

vSystem support for features

Non-default vSystems do not support configuring fast output of logs to Kafka servers.

For information about the support of non-default vSystems for the commands, see fast log output command reference. For information about vSystem, see Virtual Technologies Configuration Guide.

Restrictions and guidelines: fast log output configuration

The device supports outputting logs from service modules to log hosts by using the following methods in descending order of priority:

1.     ‍Fast log output.

2.     Flow log. For more information about flow log and the service modules supported by flow log, see "Configuring flow log."

3.     Information center.

If you configure multiple log output methods for a service module, the service module outputs its logs in the method that has the highest priority.

To output NAT logs to a log host, you must specify the log format required by the log host in the customlog format and customlog host commands.

You can configure the device to carry VNI information in NAT logs only if you specify the TELECOM format. NAT logs that carry the VNI field use a new format different from the TELECOM format.

You cannot specify both the standard format and SGCC format for IPS logs. If you configure both formats, the last specified format takes effect. However, you can configure either of the two formats and the CMCC-Kafka format for IPS logs.

 

 

NOTE:

The device supports configuring fast log output using either v1 or v2 commands. As a best practice, use commands of v2 for more convenient maintenance of the types of fast logs to be sent.

 

You can specify a maximum of 10 log hosts for fast log output. The log host resources on the device are shared by v1 and v2 commands. If you have specified 10 log hosts using v1 commands, you cannot use v2 commands to specify additional log hosts, and vice versa.

You cannot specify the local host as a log host.

Configuring fast log output (v2) (recommended)

Configuring fast log output

About this task

To fast output logs from specific modules to specific log hosts, create log hosts for fast log output on the device, and then enable fast log output for the modules in log host view. This setting effectively centralizes the storage and management of logs generated by the device, making it easier for you to monitor and analyze the device's operational status in real time. Thereby, you can better understand and resolve issues on network devices.

Procedure

1.     Enter system view.

system-view

2.     Configure a log host for fast log output and enter its view.

customlog host v2 [ vpn-instance vpn-instance-name ] { hostname | ipv4-address | ipv6 ipv6-address } [ port port-number ]

By default, no log hosts are configured for fast log output.

3.     Enable fast log output from the specified module to the log host.

¡     Enable fast log output from the AFT module to the log host.

module aft [ cmcc | unicom | telecom ]

¡     Enable fast log output from the anti-virus module to the log host.

module anti-virus

¡     Enable fast log output from the attack detection and prevention module to the log host.

module attack-defense

¡     Enable fast log output from the application audit and management module to the log host.

module audit

¡     Enable fast log output from the data filtering module to the log host.

module data-filter

¡     Enable fast log output from the DGA detection module to the log host.

module dga

¡     Enable fast log output from the file filtering module to the log host.

module file-filter

¡     Enable fast log output from the device access control module for IoT device security management to the log host.

module iot-access-control

¡     Enable fast log output from the standard traffic control module for IoT device security management to the log host.

module iot-flow-control

¡     Enable fast log output from the standard format check module for IoT device security management to the log host.

module iot-format-check

¡     Enable fast log output from the sensitive signal control module for IoT device security management to the log host.

module iot-signal-control

¡     Enable fast log output from the intrusion prevention system (IPS) module to the log host.

module ips [ sgcc { policy-hit | signature-update } ]

¡     Enable fast log output from the keepalive module to the log host in SGCC format.

module keepalive sgcc

¡     Enable fast log output from LB modules to the log host.

module loadbalance [ global-intelligent-dns | local-intelligent-dns | outbound-link-lb | server-lb | transparent-dns-proxy ] *

¡     Enable fast log output from the NAT module to the log host.

module nat [ cmcc | telecom | telecom-vni | unicom ]

¡     Enable fast log output from the NetShare control module to the log host.

module netshare

¡     Enable fast log output from the IP reputation, domain reputation, and URL reputation module to the log host.

module reputation

¡     Enable fast log output from the sandbox module to the log host.

module sandbox

¡     Enable fast log output from the server connection detection module to the log host.

module scd

¡     Enable fast log output of security policy packet matching logs to the log host.

module security-policy [ sgcc ]

¡     Enable fast log output of security policy configuration logs to the log host.

module security-policy-config sgcc

¡     Enable fast log output from the session management module logs to the log host.

module session

¡     Enable fast log output from the SSL VPN module logs to the log host.

module sslvpn

¡     Enable fast log output from the terminal identification module to the log host.

module terminal

¡     Enable fast log output from the bandwidth management module to the log host.

module traffic-policy

¡     Enable fast log output from the IAM trusted access control module to the log host.

module trusted-access iam [ authorization | notification ] *

¡     Enable fast log output from the URL filtering module to the log host.

module url-filter [ unicom ]

¡     Enable fast log output from the WAF module logs to the log host.

module waf

By default, fast log output from a specific module to a log host is disabled.

4.     Return to system view.

quit

5.     (Optional.) Configure the source IP address for fast log output.

customlog host source interface-type interface-number

By default, the source IP address of fast output logs is the primary IP address of the outgoing interface.

After this command is executed, the device always uses the primary IP address of the specified interface as the source IP address of logs, regardless of the actual physical interface used to send the logs. Execute this command when you need to filter logs by source IP address on the log host.

6.     (Optional.) Configure the timestamp of fast output logs to be the system time.

customlog timestamp localtime

By default, the timestamp of fast output logs is the Greenwich Mean Time (GMT).

7.     (Optional.) Configure the device to carry its serial number in fast output logs.

customlog with-sn

By default, the device does not carry its serial number in fast output logs.

8.     (Optional.) Specify a language for fast log output.

customlog language { chinese | english }

By default, fast logs are output in English.

Only some fields in the fast logs of certain service modules can be output in Chinese. For example, in session logs, only the Application and Category fields support Chinese. For more information about the supported fields in service module logs, see the command reference.

Configuring fast log output to use the UTF-8 encoding

About this task

The fast log output module and the log host must use the same character set encoding. If they use different encodings, the log host cannot correctly display Chinese characters in the log messages received from the fast log output module. By default, fast log output uses the GB18030 encoding. You can perform this task to configure fast log output to use the UTF-8 encoding.

Procedure

1.     Enter system view.

system-view

2.     Configure fast log output to use UTF-8 encoding.

customlog character-encoding utf-8

By default, fast log output uses the GB18030 encoding.

Configuring fast output of logs to Kafka servers

About this task

The device supports outputting fast logs in Kafka format to a Kafka log server. When you have deployed a Kafka log server in the network, created a Kafka server on the device, and enabled output of fast logs to the Kafka server, the device will send fast logs in Kafka format to the Kafka log server.

A broker is a member of a Kafka server cluster. After you configure the IP address and port of a broker for receiving logs on the device side, the device will send logs in Kafka format to the specified address.

Restrictions and guidelines

The customlog kafka-server export command takes effect only when you have enabled fast log output for the corresponding modules using the customlog format command.

Procedure

1.     Enter system view.

system-view

2.     Create a Kafka server and enter its view.

kafka-server server-name

By default, no Kafka server exists.

3.     Specify a Kafka broker.

broker { hostname | ipv4-address | ipv6 ipv6-address } [ port port-number]

By default, no Kafka broker is specified.

4.     Associate a VPN instance with the Kafka server.

vpn-instance vpn-instance-name

By default, the Kafka server is associated with the public network.

5.     Return to system view.

quit

6.     Enable output of fast logs to the Kafka server.

customlog kafka-server server-name topic topic-name export dpi ips

By default, output of fast logs to the Kafka server is disabled.

Testing fast log output

About this task

After a log host is configured for fast log output, use this feature to generate a specific type of test logs and check if the log host can receive these test logs normally.

The log host does not reply to the device after it receives the test logs. You need to check the test result on the log host.

Procedure

1.     Enter system view.

system-view

2.     Send a specified number and type of test logs.

In standalone mode:

customlog host v2 test count number { aft | anti-virus | attack-defense | audit | data-filter | dga | file-filter | iot-access-control | iot-flow-control | iot-format-check | iot-signal-control | ips | keepalive | loadbalance { global-intelligent-dns | local-intelligent-dns | outbound-link-lb | server-lb | transparent-dns-proxy } | nat | sandbox | scd | security-policy | security-policy-config | session | sslvpn | traffic-policy | trusted-access iam { authorization | notification } | url-filter | waf } [ slot slot-number [ cpu cpu-number ] [ kernel ]

In IRF mode:

customlog host v2 test count number { aft | anti-virus | attack-defense | audit | data-filter | dga | file-filter | iot-access-control | iot-flow-control | iot-format-check | iot-signal-control | ips | keepalive | loadbalance { global-intelligent-dns | local-intelligent-dns | outbound-link-lb | server-lb | transparent-dns-proxy } | nat | sandbox | scd | security-policy | security-policy-config | session | sslvpn | traffic-policy | trusted-access iam { authorization | notification } | url-filter | waf } [ chassis chassis-number slot slot-number [ cpu cpu-number ] [ kernel ]

Verifying and maintaining fast log output

Execute display commands in any view to verify the operation of fast log output.

Execute reset commands in any view to clear fast log output statistics.

Table 4 Verifying and maintaining fast log output

Task

Command

Display running kernel information for a log host.

In standalone mode:

display customlog host v2 kernel [ vpn-instance vpn-instance-name ] { hostname | ipv4-address | ipv6 ipv6-address } [ port port-number ]  [ slot slot-number [ cpu cpu-number ] ]

In IRF mode:

display customlog host v2 kernel [ vpn-instance vpn-instance-name ] { hostname | ipv4-address | ipv6 ipv6-address } [ port port-number ] [ chassis chassis-number slot slot-number [ cpu cpu-number ] ]

Display the statistics for fast log output.

In standalone mode:

display customlog host v2 [ send-failed ] statistics [ slot slot-number [ cpu cpu-number ] ]

In IRF mode:

display customlog host v2 [ send-failed ] statistics [ chassis chassis-number slot slot-number [ cpu cpu-number ] ]

Clear the statistics for fast log output.

In standalone mode:

reset customlog host v2 statistics [ slot slot-number [ cpu cpu-number ] ]

In IRF mode:

reset customlog host v2 statistics [ chassis chassis-number slot slot-number [ cpu cpu-number ] ]

 

Configuring fast log output (v1)

Configuring fast output of logs to log hosts

Restrictions and guidelines

The customlog format and customlog host commands (v1) provide a set of parameters for all service modules. However, whether the configured parameters can take effect on a specific service module depends on whether the device supports that service module. For example, if the device does not support NAT, the device will not generate NAT-related fast logs, even if NAT fast log output parameters have been configured in the commands.

Procedure

1.     ‍Enter system view.

system-view

2.     Enable fast log output.

customlog format { aft | aft-cmcc | aft-telecom | aft-unicom | attack-defense | cntm | dns | dpi [ anti-virus | audit | data-filter | file-filter | iot-access-control | iot-flow-control | iot-format-check | iot-signal-control | ips [ sgcc { policy-hit | signature-update } | cmcc-kafka ] | netshare | reputation | sandbox | terminal | traffic-policy | url-filter [ unicom ] | waf | dga ] | keepalive sgcc | lb [ dns-proxy | gslb | inbound | outbound | slb ] | nat { cmcc | telecom [ with-vni ] | unicom } | packet-filter [ sgcc ] | scd | security-policy sgcc | session | trusted-access { csap | iam [ authorization | notification ] } | wlan }

By default, fast log output is disabled.

3.     Configure fast log output parameters.

customlog host [ vpn-instance vpn-instance-name ] { hostname | ipv4-address | ipv6 ipv6-address } [ port port-number ][ facility local-number ] export { aft | attack-defense | cmcc-sessionlog | cmcc-userlog | cntm | dns | dpi [ anti-virus | audit | data-filter | file-filter | iot-access-control | iot-flow-control | iot-format-check | iot-signal-control | ips | netshare | reputation | sandbox | terminal | traffic-policy | url-filter | waf | dga ] * | keepalive | lb [ dns-proxy | gslb | inbound | outbound | slb ] * | packet-filter | scd | security-policy | session | telecom-sessionlog | telecom-userlog | trusted-access { csap | iam [ authorization | notification ] } * | unicom-sessionlog | unicom-userlog } *

By default, no fast log output parameters are configured.

The value for the port-number argument must be the same as the port number configured on the log host. Otherwise, the log host cannot receive logs.

The facility local-number option takes effect only on the logs output in standard format from each service module as well as the AFT and NAT logs output in a carrier-customized format. If you do not specify this option, the value for the Facility field in the log headers output by each service module is used as the logging facility. For the AFT and NAT modules, logs output in CMCC and UNICOM formats will have a facility value of Local1, while logs output in TELECOM format will have a facility value of Local0.

4.     (Optional.) Specify the source IP address for fast log output.

customlog host source interface-type interface-number

By default, the source IP address of fast output logs is the primary IP address of the outgoing interface.

If this command is configured, the primary IP address of the specified interface is used as the source IP address of fast output logs regardless of the outgoing interface.

Configure this command when you need to filter logs by source IP address on the log host.

5.     (Optional.) Configure the timestamp of fast output logs to show the system time.

customlog timestamp localtime

By default, the timestamp of fast output logs shows the Greenwich Mean Time (GMT).

6.     (Optional.) Specify a language for fast log output.

customlog language { chinese | english }

By default, fast logs are output in English.

Only some fields in the fast logs of certain service modules can be output in Chinese. For example, only the Application and Category fields in session logs support fast output in Chinese. For more information about the supported fields in service module logs, see the command reference.

Configuring fast log output to use the UTF-8 encoding

About this task

The fast log output module and the log host must use the same character set encoding. If they use different encodings, the log host cannot correctly display Chinese characters in the log messages received from the fast log output module. By default, fast log output uses the GB18030 encoding. You can perform this task to configure fast log output to use the UTF-8 encoding.

Procedure

1.     Enter system view.

system-view

2.     Configure fast log output to use UTF-8 encoding.

customlog character-encoding utf-8

By default, fast log output uses the GB18030 encoding.

Configuring fast output of logs to Kafka servers

About this task

The device supports outputting fast logs in Kafka format to a Kafka log server. When you have deployed a Kafka log server in the network, created a Kafka server on the device, and enabled output of fast logs to the Kafka server, the device will send fast logs in Kafka format to the Kafka log server.

A broker is a member of a Kafka server cluster. After you configure the IP address and port of a broker for receiving logs on the device side, the device will send logs in Kafka format to the specified address.

Restrictions and guidelines

The customlog kafka-server export command takes effect only when you have enabled fast log output for the corresponding modules using the customlog format command.

Procedure

1.     Enter system view.

system-view

2.     Create a Kafka server and enter its view.

kafka-server server-name

By default, no Kafka server exists.

3.     Specify a Kafka broker.

broker { hostname | ipv4-address | ipv6 ipv6-address } [ port port-number]

By default, no Kafka broker is specified.

4.     Associate a VPN instance with the Kafka server.

vpn-instance vpn-instance-name

By default, the Kafka server is associated with the public network.

5.     Return to system view.

quit

6.     Enable output of fast logs to the Kafka server.

customlog kafka-server server-name topic topic-name export dpi ips

By default, output of fast logs to the Kafka server is disabled.

Fast log output configuration examples

Example: Configuring fast log output to a log host

Network configuration

As shown in Figure 1, configure fast log output on the device to send session logs to the log server.

Figure 1 Network diagram

Procedure

1.     Assign IP addresses to interface GigabitEthernet 1/0/2.

<Device> system-view

[Device] interface gigabitethernet 1/0/2

[Device-GigabitEthernet1/0/2] ip address 1.1.0.1 255.255.0.0

[Device-GigabitEthernet1/0/2] quit

2.     Configure settings for routing.

This example configures a static route, and the next hop in the router is 1.1.0.2.

[Device] ip route-static 1.2.0.0 16 1.1.0.2

3.     Add interface GigabitEthernet 1/0/2 to security zone untrust.

[Device] security-zone name untrust

[Device-security-zone-Untrust] import interface gigabitethernet 1/0/2

[Device-security-zone-Untrust] quit

4.     Configure a security policy:

# Configure a rule named loglocalout to allow the device to send fast log output messages to the server.

[Device] security-policy ip

[Device-security-policy-ip] rule name loglocalout

[Device-security-policy-ip-1-loglocalout] source-zone local

[Device-security-policy-ip-1-loglocalout] destination-zone untrust

[Device-security-policy-ip-1-loglocalout] source-ip-host 1.1.0.1

[Device-security-policy-ip-1-loglocalout] destination-ip-host 1.2.0.1

[Device-security-policy-ip-1-loglocalout] action pass

[Device-security-policy-ip-1-loglocalout] quit

[Device-security-policy-ip] quit

5.     Configure fast log output. Enable fast log output, configure log output to the log server, and enable logging for session creation and deletion. Enable IPv4 session logging in the inbound direction of the interface connected to the internal network.

[Device] customlog format session

[Device] customlog host 1.2.0.1 port 1000 export session

[Device] session log flow-begin

[Device] session log flow-end

[Device] interface gigabitethernet 1/0/1

[Device-GigabitEthernet1/0/1] session log enable ipv4 inbound

Verifying the configuration

On the server, verify that logs are received from the device successfully.

  • Cloud & AI
  • InterConnect
  • Intelligent Computing
  • Intelligent Storage
  • Security
  • SMB Products
  • Intelligent Terminal Products
  • Product Support Services
  • Technical Service Solutions
All Services
  • Resource Center
  • Policy
  • Online Help
  • Technical Blogs
All Support
  • Become A Partner
  • Partner Policy & Program
  • Global Learning
  • Partner Sales Resources
  • Partner Business Management
  • Service Business
All Partners
  • Profile
  • News & Events
  • Online Exhibition Center
  • Contact Us
All About Us
新华三官网