20-Network Management and Monitoring Configuration Guide

HomeSupportConfigure & DeployConfiguration GuidesH3C WX2800X&WSG1800X Configuration Guides(R5605P11)-6W10020-Network Management and Monitoring Configuration Guide
09-Fast log output configuration
Title Size Download
09-Fast log output configuration 146.68 KB

Fast log output overview

About fast log output

The fast log output feature enables fast output of logs to log hosts. These logs are called fast output logs, or fast logs for short.

Typically, logs generated by a service module are first sent to the information center, which then outputs the logs to the specified destination (such as to log hosts). When fast log output is configured, logs of service modules are sent directly to log hosts instead of to the information center. Compared to outputting logs to the information center, fast log output saves system resources. For more information about the information center, see System Management Configuration Guide.

Logs are classified into eight severity levels from 0 through 7 in descending order.

Table 1 Log levels

Severity value

Level

Description

0

Emergency

The system is unusable. For example, the system authorization has expired.

1

Alert

Action must be taken immediately. For example, traffic on an interface exceeds the upper limit.

2

Critical

Critical condition. For example, the device temperature exceeds the upper limit, the power module fails, or the fan tray fails.

3

Error

Error condition. For example, the link state changes.

4

Warning

Warning condition. For example, an interface is disconnected, or the memory resources are used up.

5

Notification

Normal but significant condition. For example, a terminal logs in to the device, or the device reboots.

6

Informational

Informational message. For example, a command or a ping operation is executed.

7

Debugging

Debug message.

Log header formats

The log header formats of fast output logs are as follows:

Table 2 Log header formats

Log header types

Format

Standard format

<PRI> Timestamp AppName %%10 SN:sn VsysId:id

Example:

<134> Apr 28 15:35:32 2020 H3C %%10 SN:10056879 VsysId:1

Customized format

URL filtering UNICOM format:

PRI Vision HostName Timestamp AppName MsgID HostName Len

Example:

142 1 100.0.0.1 2020 Apr 28 15:35:43 H3C NAT444:SessionU 57

NAT CMCC format:

<PRI> Vision HostName Timestamp AppName ProcID MsgID

Example:

<142> 1 100.0.0.1 2020 Apr 28 15:35:32 H3C - NAT444:SessionA

NAT UNICOM format:

<PRI> Vision HostName Timestamp AppName ProcID MsgID

Example:

<142> 1 100.0.0.1 2020 Apr 28 15:35:43 H3C - NAT444:SessionA

NAT TELECOM format:

<PRI> Vision Timestamp HostName AppName ProcID MsgID

Example:

<134> 1 2020 Apr 28 15:35:38 100.0.0.1 H3C - NAT444:sessionbasedA

Log field description

Table 3 Log field description

Field

Description

PRI

Log type code.

·     Standard format and NAT UNICOM format: 134.

·     URL filtering UNICOM format, NAT CMCC format, and NAT TELECOM format: 142.

Timestamp

Records the time when the log was generated. The timestamp is in the format of YYYY Mon DD hh:mm:ss.

AppName

Name of the device that generated the log.

%%10

Vendor of the device that generated the log.

SN

Serial number of the device that generated the log. To view the device serial number, see the DEVICE_SERIAL_NUMBE field in the output of the display device manuinfo command.

This field is available only when the device is configured to carry the serial number in fast output logs by using the customlog with-sn command.

VsysId

Virtual system that generated the log.

HostName

Source IPv4 address of the device that generated the log.

MsgID

Log type.

Len

Total length of the log header, in bytes.

ProcID

Hyphen (-).

Restrictions and guidelines: fast log output configuration

The device supports outputting logs from service modules to log hosts by using the following methods in descending order of priority:

1.     Fast log output.

2.     Flow log. For more information about flow log and the service modules supported by flow log, see "Configuring flow log."

3.     Information center.

If you configure multiple log output methods for a service module, the service module outputs its logs in the method that has the highest priority.

To output NAT logs to a log host, you must specify the log format required by the log host in the customlog format and customlog host commands.

You can configure the device to carry VNI information in NAT logs only if you specify the TELECOM format. NAT logs that carry the VNI field use a new format different from the TELECOM format.

You cannot specify both the standard format and SGCC format for IPS logs. If you configure both formats, the last specified format takes effect. However, you can configure either of the two formats and the CMCC-Kafka format for IPS logs.

 

 

NOTE:

The device supports configuring fast log output using either v1 or v2 commands. As a best practice, use commands of v2 for more convenient maintenance of the types of fast logs to be sent.

 

You can specify a maximum of eight log hosts for fast log output. The log host resources on the device are shared by v1 and v2 commands. If you have specified eight log hosts using v1 commands, you cannot use v2 commands to specify additional log hosts, and vice versa.

Configuring fast log output (v2) (recommended)

Configuring fast log output

About this task

To fast output logs from specific modules to specific log hosts, create log hosts for fast log output on the device, and then enable fast log output for the modules in log host view. This setting effectively centralizes the storage and management of logs generated by the device, making it easier for you to monitor and analyze the device's operational status in real time. Thereby, you can better understand and resolve issues on network devices.

Procedure

1.     Enter system view.

system-view

2.     Configure a log host for fast log output and enter its view.

customlog host v2 { hostname | ipv4-address | ipv6 ipv6-address } [ port port-number ]

By default, no log hosts are configured for fast log output.

3.     Enable fast log output from the specified module to the log host.

¡     Enable fast log output from the anti-virus module to the log host.

module anti-virus

¡     Enable fast log output from the attack detection and prevention module to the log host.

module attack-defense

¡     Enable fast log output from the application audit and management module to the log host.

module audit

¡     Enable fast log output from the data filtering module to the log host.

module data-filter

¡     Enable fast log output from the file filtering module to the log host.

module file-filter

¡     Enable fast log output from the intrusion prevention system (IPS) module to the log host.

module ips [ sgcc { policy-hit | signature-update } ]

¡     Enable fast log output from the keepalive module to the log host in SGCC format.

module keepalive sgcc

¡     Enable fast log output from LB modules to the log host.

module loadbalance [ global-intelligent-dns | local-intelligent-dns | outbound-link-lb | server-lb | transparent-dns-proxy ] *

¡     Enable fast log output from the NAT module to the log host.

module nat [ cmcc | telecom | telecom-vni | unicom ]

¡     Enable fast log output of security policy packet matching logs to the log host.

module security-policy [ sgcc ]

¡     Enable fast log output of security policy configuration logs to the log host.

module security-policy-config sgcc

¡     Enable fast log output from the session management module logs to the log host.

module session

¡     Enable fast log output from the SSL VPN module logs to the log host.

module sslvpn

¡     Enable fast log output from the bandwidth management module to the log host.

module traffic-policy

¡     Enable fast log output from the URL filtering module to the log host.

module url-filter [ unicom ]

By default, fast log output from a specific module to a log host is disabled.

4.     Return to system view.

quit

5.     (Optional.) Configure the source IP address for fast log output.

customlog host source interface-type interface-number

By default, the source IP address of fast output logs is the primary IP address of the outgoing interface.

After this command is executed, the device always uses the primary IP address of the specified interface as the source IP address of logs, regardless of the actual physical interface used to send the logs. Execute this command when you need to filter logs by source IP address on the log host.

6.     (Optional.) Configure the timestamp of fast output logs to be the system time.

customlog timestamp localtime

By default, the timestamp of fast output logs is the Greenwich Mean Time (GMT).

7.     (Optional.) Configure the device to carry its serial number in fast output logs.

customlog with-sn

By default, the device does not carry its serial number in fast output logs.

8.     (Optional.) Specify a language for fast log output.

customlog language { chinese | english }

By default, fast logs are output in English.

Only some fields in the fast logs of certain service modules can be output in Chinese. For example, in session logs, only the Application and Category fields support Chinese. For more information about the supported fields in service module logs, see the command reference.

Configuring a logging facility for a log host

About this task

A logging facility is primarily used for marking different log sources on a log host, and for searching and filtering logs from corresponding log sources.

Restrictions and guidelines

The configured facility takes effect only on the logs output in standard format from each service module as well as the NAT logs output in a carrier-customized format.

If you do not configure a facility for a log host, the value for the Facility field in the log headers output by each service module is used as the logging facility. For the NAT module, logs output in CMCC and UNICOM formats will have a facility value of Local1, while logs output in TELECOM format will have a facility value of Local0.

Procedure

1.     Enter system view.

system-view

2.     Configure a log host for fast log output and enter its view.

customlog host v2 { hostname | ipv4-address | ipv6 ipv6-address } [ port port-number ]

By default, no log host is configured for fast log output.

3.     Configure a logging facility for the log host.

facility local-number

By default, no logging facility is configured for a log host.

Configuring fast log output to use the UTF-8 encoding

About this task

The fast log output module and the log host must use the same character set encoding. If they use different encodings, the log host cannot correctly display Chinese characters in the log messages received from the fast log output module. By default, fast log output uses the GB18030 encoding. You can perform this task to configure fast log output to use the UTF-8 encoding.

Procedure

1.     Enter system view.

system-view

2.     Configure fast log output to use UTF-8 encoding.

customlog character-encoding utf-8

By default, fast log output uses the GB18030 encoding.

Configuring fast output of logs to Kafka servers

About this task

The device supports outputting fast logs in Kafka format to a Kafka log server. When you have deployed a Kafka log server in the network, created a Kafka server on the device, and enabled output of fast logs to the Kafka server, the device will send fast logs in Kafka format to the Kafka log server.

A broker is a member of a Kafka server cluster. After you configure the IP address and port of a broker for receiving logs on the device side, the device will send logs in Kafka format to the specified address.

Restrictions and guidelines

The customlog kafka-server export command takes effect only when you have enabled fast log output for the corresponding modules using the customlog format command.

Procedure

1.     Enter system view.

system-view

2.     Create a Kafka server and enter its view.

kafka-server server-name

By default, no Kafka server exists.

3.     Specify a Kafka broker.

broker { hostname | ipv4-address | ipv6 ipv6-address } [ port port-number]

By default, no Kafka broker is specified.

4.     Return to system view.

quit

5.     Enable output of fast logs to the Kafka server.

customlog kafka-server server-name topic topic-name export dpi ips

By default, output of fast logs to the Kafka server is disabled.

Testing fast log output

About this task

After a log host is configured for fast log output, use this feature to generate a specific type of test logs and check if the log host can receive these test logs normally.

The log host does not reply to the device after it receives the test logs. You need to check the test result on the log host.

Procedure

1.     ‍Enter system view.

system-view

2.     Send a specified number and type of test logs.

customlog host v2 test count number { anti-virus | attack-defense | audit | data-filter | file-filter | ips | keepalive | loadbalance { global-intelligent-dns | local-intelligent-dns | outbound-link-lb | server-lb | transparent-dns-proxy } | nat | security-policy | security-policy-config | session | sslvpn | traffic-policy  | url-filter } [ kernel ]

 

 

Verifying and maintaining fast log output

Execute display commands in any view to verify the operation of fast log output.

Execute reset commands in any view to clear fast log output statistics.

Table 4 Verifying and maintaining fast log output

Task

Command

Display running kernel information for a log host.

display customlog host v2 kernel { hostname | ipv4-address | ipv6 ipv6-address } [ port port-number ]

Display the statistics for fast log output.

display customlog host v2 [ send-failed ] statistics

Clear the statistics for fast log output.

reset customlog host v2 statistics

 

Configuring fast log output (v1)

Configuring fast output of logs to log hosts

Restrictions and guidelines

The customlog format and customlog host commands (v1) provide a set of parameters for all service modules. However, whether the configured parameters can take effect on a specific service module depends on whether the device supports that service module. For example, if the device does not support NAT, the device will not generate NAT-related fast logs, even if NAT fast log output parameters have been configured in the commands.

Procedure

1.     Enter system view.

system-view

2.     Enable fast log output.

customlog format { attack-defense | cntm | dns | dpi [ anti-virus | audit | data-filter | file-filter | ips [ sgcc { policy-hit | signature-update } | cmcc-kafka ] | traffic-policy | url-filter [ unicom ] ] | keepalive sgcc | lb [ dns-proxy | gslb | inbound | outbound | slb ] | nat { cmcc | telecom | unicom } | packet-filter [ sgcc ] | security-policy sgcc | session | wlan }

By default, fast log output is disabled.

3.     Configure fast log output parameters.

customlog host { hostname | ipv4-address | ipv6 ipv6-address } [ port port-number ][ facility local-number ] export { attack-defense | cmcc-sessionlog | cmcc-userlog | cntm | dns | dpi [ anti-virus | audit | data-filter | file-filter | ips | traffic-policy | url-filter ] * | keepalive | lb [ dns-proxy | gslb | inbound | outbound | slb ] * | packet-filter | security-policy | session | telecom-sessionlog | telecom-userlog | unicom-sessionlog | unicom-userlog } *

By default, no fast log output parameters are configured.

The value for the port-number argument must be the same as the port number configured on the log host. Otherwise, the log host cannot receive logs.

The facility local-number option takes effect only on the logs output in standard format from each service module as well as the AFT and NAT logs output in a carrier-customized format. If you do not specify this option, the value for the Facility field in the log headers output by each service module is used as the logging facility. For the AFT and NAT modules, logs output in CMCC and UNICOM formats will have a facility value of Local1, while logs output in TELECOM format will have a facility value of Local0.

4.     (Optional.) Specify the source IP address for fast log output.

customlog host source interface-type interface-number

By default, the source IP address of fast output logs is the primary IP address of the outgoing interface.

If this command is configured, the primary IP address of the specified interface is used as the source IP address of fast output logs regardless of the outgoing interface.

Configure this command when you need to filter logs by source IP address on the log host.

5.     (Optional.) Configure the timestamp of fast output logs to show the system time.

customlog timestamp localtime

By default, the timestamp of fast output logs shows the Greenwich Mean Time (GMT).

6.     (Optional.) Configure the device to carry its serial number in fast output logs.

customlog with-sn

By default, the device does not carry its serial number in fast output logs.

7.     (Optional.) Specify a language for fast log output.

customlog language { chinese | english }

By default, fast logs are output in English.

Only some fields in the fast logs of certain service modules can be output in Chinese. For example, only the Application and Category fields in session logs support fast output in Chinese. For more information about the supported fields in service module logs, see the command reference.

Configuring fast log output to use the UTF-8 encoding

About this task

The fast log output module and the log host must use the same character set encoding. If they use different encodings, the log host cannot correctly display Chinese characters in the log messages received from the fast log output module. By default, fast log output uses the GB18030 encoding. You can perform this task to configure fast log output to use the UTF-8 encoding.

Procedure

1.     Enter system view.

system-view

2.     Configure fast log output to use UTF-8 encoding.

customlog character-encoding utf-8

By default, fast log output uses the GB18030 encoding.

Configuring fast output of logs to Kafka servers

About this task

The device supports outputting fast logs in Kafka format to a Kafka log server. When you have deployed a Kafka log server in the network, created a Kafka server on the device, and enabled output of fast logs to the Kafka server, the device will send fast logs in Kafka format to the Kafka log server.

A broker is a member of a Kafka server cluster. After you configure the IP address and port of a broker for receiving logs on the device side, the device will send logs in Kafka format to the specified address.

Restrictions and guidelines

The customlog kafka-server export command takes effect only when you have enabled fast log output for the corresponding modules using the customlog format command.

Procedure

1.     Enter system view.

system-view

2.     Create a Kafka server and enter its view.

kafka-server server-name

By default, no Kafka server exists.

3.     Specify a Kafka broker.

broker { hostname | ipv4-address | ipv6 ipv6-address } [ port port-number]

By default, no Kafka broker is specified.

4.     Return to system view.

quit

5.     Enable output of fast logs to the Kafka server.

customlog kafka-server server-name topic topic-name export dpi ips

By default, output of fast logs to the Kafka server is disabled.

Fast log output configuration examples

Example: Configuring fast log output to a log host

Network configuration

As shown in Figure 1, configure fast log output on the AC to send NAT444 user logs to the log host in CMCC format.

Figure 1 Network diagram

Procedure

1.     Make sure the AC and the log host can reach each other. (Details not shown.)

2.     Configure the AC:

# Enable fast log output in CMCC format for the NAT module.

<AC> system-view

[AC] customlog format nat cmcc

# Output NAT444 user logs in CMCC format to the log host at 1.2.0.1/16.

[AC] customlog host 1.2.0.1 port 1000 export cmcc-userlog

# Enable NAT444 user log.

[AC] nat log enable

[AC] nat log port-block-assign

[AC] nat log port-block-withdraw

3.     Configure the log host:

The log host configuration varies by log host. For more information, see related document of the log host.

Verifying the configuration

On the host, verify that logs are received from the AC successfully.

  • Cloud & AI
  • InterConnect
  • Intelligent Computing
  • Intelligent Storage
  • Security
  • SMB Products
  • Intelligent Terminal Products
  • Product Support Services
  • Technical Service Solutions
All Services
  • Resource Center
  • Policy
  • Online Help
  • Technical Blogs
All Support
  • Become A Partner
  • Partner Policy & Program
  • Global Learning
  • Partner Sales Resources
  • Partner Business Management
  • Service Business
All Partners
  • Profile
  • News & Events
  • Online Exhibition Center
  • Contact Us
All About Us
新华三官网