- Table of Contents
-
- 17-Network Management and Monitoring Configuration Guide
- 00-Preface
- 01-EAA configuration
- 02-Event MIB configuration
- 03-Flow log configuration
- 04-NetStream configuration
- 05-NTP configuration
- 06-RMON configuration
- 08-Mirroring configuration
- 09-Track configuration
- 10-Process placement configuration
- 11-Interface collaboration configuration
- 12-SNMP configuration
- 13-NQA configuration
- 14-Fast log output configuration
- 15-System maintenance and debugging configuration
- 16-Information center configuration
- 17-NETCONF configuration
- 18-Process monitoring and maintenance configuration
- 19-BFD configuration
- Related Documents
-
Title | Size | Download |
---|---|---|
14-Fast log output configuration | 155.68 KB |
Restrictions and guidelines: fast log output configuration
Configuring fast log output (v2) (recommended)
Configuring fast log output to use the UTF-8 encoding
Configuring fast output of logs to Kafka servers
Verifying and maintaining fast log output
Configuring fast log output (v1)
Configuring fast output of logs to log hosts
Configuring fast log output to use the UTF-8 encoding
Configuring fast output of logs to Kafka servers
Fast log output configuration examples
Example: Configuring fast log output to a log host
Fast log output overview
About fast log output
The fast log output feature enables fast output of logs to log hosts. These logs are called fast output logs, or fast logs for short.
Typically, logs generated by a service module are first sent to the information center, which then outputs the logs to the specified destination (such as to log hosts). When fast log output is configured, logs of service modules are sent directly to log hosts instead of to the information center. Compared to outputting logs to the information center, fast log output saves system resources. For more information about the information center, see "Configuring the information center."
Logs are classified into eight severity levels from 0 through 7 in descending order.
Severity value |
Level |
Description |
0 |
Emergency |
The system is unusable. For example, the system authorization has expired. |
1 |
Alert |
Action must be taken immediately. For example, traffic on an interface exceeds the upper limit. |
2 |
Critical |
Critical condition. For example, the device temperature exceeds the upper limit, the power module fails, or the fan tray fails. |
3 |
Error |
Error condition. For example, the link state changes. |
4 |
Warning |
Warning condition. For example, an interface is disconnected, or the memory resources are used up. |
5 |
Notification |
Normal but significant condition. For example, a terminal logs in to the device, or the device reboots. |
6 |
Informational |
Informational message. For example, a command or a ping operation is executed. |
7 |
Debugging |
Debug message. |
Log header formats
The log header formats of fast output logs are as follows:
Table 2 Log header formats
Log header types |
Format |
Standard format |
Example: |
Customized format |
URL filtering UNICOM format: Example: NAT CMCC format: Example: NAT UNICOM format: Example: NAT TELECOM format: Example: |
Log field description
Table 3 Log field description
Field |
Description |
PRI |
Log type code. · Standard format and NAT UNICOM format: 134. · URL filtering UNICOM format, NAT CMCC format, and NAT TELECOM format: 142. |
Version |
Version information, version identifier of log information, which is 1. |
Timestamp |
Records the time when the log was generated. The timestamp is in the format of YYYY Mon DD hh:mm:ss. |
AppName |
Name of the device that generated the log. |
%%10 |
Vendor of the device that generated the log. |
SN |
Serial number of the device that generated the log. To view the device serial number, see the DEVICE_SERIAL_NUMBE field in the output of the display device manuinfo command. This field is available only when the device is configured to carry the serial number in fast output logs by using the customlog with-sn command. |
VsysId |
Virtual system that generated the log. |
HostName |
Source IPv4 address of the device that generated the log. This field is fixed at a hyphen (-) for the NAT CMCC format. |
MsgID |
Log type. |
Len |
Total length of the log header, in bytes. |
ProcID |
Hyphen (-). |
vSystem support for features
Non-default vSystems do not support configuring fast output of logs to Kafka servers.
For information about the support of non-default vSystems for the commands, see fast log output command reference. For information about vSystem, see Virtual Technologies Configuration Guide.
Restrictions and guidelines: fast log output configuration
1. Fast log output.
2. Flow log. For more information about flow log and the service modules supported by flow log, see "Configuring flow log."
3. Information center.
If you configure multiple log output methods for a service module, the service module outputs its logs in the method that has the highest priority.
To output NAT logs to a log host, you must specify the log format required by the log host in the customlog format and customlog host commands.
You can configure the device to carry VNI information in NAT logs only if you specify the TELECOM format. NAT logs that carry the VNI field use a new format different from the TELECOM format.
You cannot specify both the standard format and SGCC format for IPS logs. If you configure both formats, the last specified format takes effect. However, you can configure either of the two formats and the CMCC-Kafka format for IPS logs.
|
NOTE: The device supports configuring fast log output using either v1 or v2 commands. As a best practice, use commands of v2 for more convenient maintenance of the types of fast logs to be sent. |
You can specify a maximum of 10 log hosts for fast log output. The log host resources on the device are shared by v1 and v2 commands. If you have specified 10 log hosts using v1 commands, you cannot use v2 commands to specify additional log hosts, and vice versa.
You cannot specify the local host as a log host.
Configuring fast log output (v2) (recommended)
Configuring fast log output
About this task
To fast output logs from specific modules to specific log hosts, create log hosts for fast log output on the device, and then enable fast log output for the modules in log host view. This setting effectively centralizes the storage and management of logs generated by the device, making it easier for you to monitor and analyze the device's operational status in real time. Thereby, you can better understand and resolve issues on network devices.
Procedure
1. Enter system view.
system-view
2. Configure a log host for fast log output and enter its view.
customlog host v2 [ vpn-instance vpn-instance-name ] { hostname | ipv4-address | ipv6 ipv6-address } [ port port-number ]
By default, no log hosts are configured for fast log output.
3. Enable fast log output from the specified module to the log host.
¡ Enable fast log output from the AFT module to the log host.
module aft [ cmcc | unicom | telecom ]
¡ Enable fast log output from the anti-virus module to the log host.
module anti-virus
¡ Enable fast log output from the attack detection and prevention module to the log host.
module attack-defense
¡ Enable fast log output from the application audit and management module to the log host.
module audit
¡ Enable fast log output from the data filtering module to the log host.
module data-filter
¡ Enable fast log output from the DGA detection module to the log host.
module dga
¡ Enable fast log output from the file filtering module to the log host.
module file-filter
¡ Enable fast log output from the device access control module for IoT device security management to the log host.
module iot-access-control
¡ Enable fast log output from the standard traffic control module for IoT device security management to the log host.
module iot-flow-control
¡ Enable fast log output from the standard format check module for IoT device security management to the log host.
module iot-format-check
¡ Enable fast log output from the sensitive signal control module for IoT device security management to the log host.
module iot-signal-control
¡ Enable fast log output from the intrusion prevention system (IPS) module to the log host.
module ips [ sgcc { policy-hit | signature-update } ]
¡ Enable fast log output from the keepalive module to the log host in SGCC format.
module keepalive sgcc
¡ Enable fast log output from LB modules to the log host.
module loadbalance [ global-intelligent-dns | local-intelligent-dns | outbound-link-lb | server-lb | transparent-dns-proxy ] *
¡ Enable fast log output from the NAT module to the log host.
module nat [ cmcc | telecom | telecom-vni | unicom ]
¡ Enable fast log output from the NetShare control module to the log host.
module netshare
¡ Enable fast log output from the IP reputation, domain reputation, and URL reputation module to the log host.
module reputation
¡ Enable fast log output from the sandbox module to the log host.
module sandbox
¡ Enable fast log output from the server connection detection module to the log host.
module scd
¡ Enable fast log output of security policy packet matching logs to the log host.
module security-policy [ sgcc ]
¡ Enable fast log output of security policy configuration logs to the log host.
module security-policy-config sgcc
¡ Enable fast log output from the session management module logs to the log host.
module session
¡ Enable fast log output from the SSL VPN module logs to the log host.
module sslvpn
¡ Enable fast log output from the terminal identification module to the log host.
module terminal
¡ Enable fast log output from the bandwidth management module to the log host.
module traffic-policy
¡ Enable fast log output from the IAM trusted access control module to the log host.
module trusted-access iam [ authorization | notification ] *
¡ Enable fast log output from the URL filtering module to the log host.
module url-filter [ unicom ]
¡ Enable fast log output from the WAF module logs to the log host.
module waf
By default, fast log output from a specific module to a log host is disabled.
4. Return to system view.
quit
5. (Optional.) Configure the source IP address for fast log output.
customlog host source interface-type interface-number
By default, the source IP address of fast output logs is the primary IP address of the outgoing interface.
After this command is executed, the device always uses the primary IP address of the specified interface as the source IP address of logs, regardless of the actual physical interface used to send the logs. Execute this command when you need to filter logs by source IP address on the log host.
6. (Optional.) Configure the timestamp of fast output logs to be the system time.
customlog timestamp localtime
By default, the timestamp of fast output logs is the Greenwich Mean Time (GMT).
7. (Optional.) Configure the device to carry its serial number in fast output logs.
customlog with-sn
By default, the device does not carry its serial number in fast output logs.
8. (Optional.) Specify a language for fast log output.
customlog language { chinese | english }
By default, fast logs are output in English.
Only some fields in the fast logs of certain service modules can be output in Chinese. For example, in session logs, only the Application and Category fields support Chinese. For more information about the supported fields in service module logs, see the command reference.
Configuring fast log output to use the UTF-8 encoding
About this task
The fast log output module and the log host must use the same character set encoding. If they use different encodings, the log host cannot correctly display Chinese characters in the log messages received from the fast log output module. By default, fast log output uses the GB18030 encoding. You can perform this task to configure fast log output to use the UTF-8 encoding.
Procedure
1. Enter system view.
system-view
2. Configure fast log output to use UTF-8 encoding.
customlog character-encoding utf-8
By default, fast log output uses the GB18030 encoding.
Configuring fast output of logs to Kafka servers
About this task
The device supports outputting fast logs in Kafka format to a Kafka log server. When you have deployed a Kafka log server in the network, created a Kafka server on the device, and enabled output of fast logs to the Kafka server, the device will send fast logs in Kafka format to the Kafka log server.
A broker is a member of a Kafka server cluster. After you configure the IP address and port of a broker for receiving logs on the device side, the device will send logs in Kafka format to the specified address.
Restrictions and guidelines
The customlog kafka-server export command takes effect only when you have enabled fast log output for the corresponding modules using the customlog format command.
Procedure
1. Enter system view.
system-view
2. Create a Kafka server and enter its view.
kafka-server server-name
By default, no Kafka server exists.
3. Specify a Kafka broker.
broker { hostname | ipv4-address | ipv6 ipv6-address } [ port port-number]
By default, no Kafka broker is specified.
4. Associate a VPN instance with the Kafka server.
vpn-instance vpn-instance-name
By default, the Kafka server is associated with the public network.
5. Return to system view.
quit
6. Enable output of fast logs to the Kafka server.
customlog kafka-server server-name topic topic-name export dpi ips
By default, output of fast logs to the Kafka server is disabled.
Testing fast log output
About this task
After a log host is configured for fast log output, use this feature to generate a specific type of test logs and check if the log host can receive these test logs normally.
The log host does not reply to the device after it receives the test logs. You need to check the test result on the log host.
Procedure
1. Enter system view.
system-view
2. Send a specified number and type of test logs.
In standalone mode:
customlog host v2 test count number { aft | anti-virus | attack-defense | audit | data-filter | dga | file-filter | iot-access-control | iot-flow-control | iot-format-check | iot-signal-control | ips | keepalive | loadbalance { global-intelligent-dns | local-intelligent-dns | outbound-link-lb | server-lb | transparent-dns-proxy } | nat | sandbox | scd | security-policy | security-policy-config | session | sslvpn | traffic-policy | trusted-access iam { authorization | notification } | url-filter | waf } [ slot slot-number [ cpu cpu-number ] [ kernel ]
In IRF mode:
customlog host v2 test count number { aft | anti-virus | attack-defense | audit | data-filter | dga | file-filter | iot-access-control | iot-flow-control | iot-format-check | iot-signal-control | ips | keepalive | loadbalance { global-intelligent-dns | local-intelligent-dns | outbound-link-lb | server-lb | transparent-dns-proxy } | nat | sandbox | scd | security-policy | security-policy-config | session | sslvpn | traffic-policy | trusted-access iam { authorization | notification } | url-filter | waf } [ chassis chassis-number slot slot-number [ cpu cpu-number ] [ kernel ]
Verifying and maintaining fast log output
Execute display commands in any view to verify the operation of fast log output.
Execute reset commands in any view to clear fast log output statistics.
Table 4 Verifying and maintaining fast log output
Task |
Command |
Display running kernel information for a log host. |
In standalone mode: display customlog host v2 kernel [ vpn-instance vpn-instance-name ] { hostname | ipv4-address | ipv6 ipv6-address } [ port port-number ] [ slot slot-number [ cpu cpu-number ] ] In IRF mode: display customlog host v2 kernel [ vpn-instance vpn-instance-name ] { hostname | ipv4-address | ipv6 ipv6-address } [ port port-number ] [ chassis chassis-number slot slot-number [ cpu cpu-number ] ] |
Display the statistics for fast log output. |
In standalone mode: display customlog host v2 [ send-failed ] statistics [ slot slot-number [ cpu cpu-number ] ] In IRF mode: display customlog host v2 [ send-failed ] statistics [ chassis chassis-number slot slot-number [ cpu cpu-number ] ] |
Clear the statistics for fast log output. |
In standalone mode: reset customlog host v2 statistics [ slot slot-number [ cpu cpu-number ] ] In IRF mode: reset customlog host v2 statistics [ chassis chassis-number slot slot-number [ cpu cpu-number ] ] |
Configuring fast log output (v1)
Configuring fast output of logs to log hosts
Restrictions and guidelines
The customlog format and customlog host commands (v1) provide a set of parameters for all service modules. However, whether the configured parameters can take effect on a specific service module depends on whether the device supports that service module. For example, if the device does not support NAT, the device will not generate NAT-related fast logs, even if NAT fast log output parameters have been configured in the commands.
Procedure
1. Enter system view.
system-view
2. Enable fast log output.
customlog format { aft | aft-cmcc | aft-telecom | aft-unicom | attack-defense | cntm | dns | dpi [ anti-virus | audit | data-filter | file-filter | iot-access-control | iot-flow-control | iot-format-check | iot-signal-control | ips [ sgcc { policy-hit | signature-update } | cmcc-kafka ] | netshare | reputation | sandbox | terminal | traffic-policy | url-filter [ unicom ] | waf | dga ] | keepalive sgcc | lb [ dns-proxy | gslb | inbound | outbound | slb ] | nat { cmcc | telecom [ with-vni ] | unicom } | packet-filter [ sgcc ] | scd | security-policy sgcc | session | trusted-access { csap | iam [ authorization | notification ] } | wlan }
By default, fast log output is disabled.
3. Configure fast log output parameters.
customlog host [ vpn-instance vpn-instance-name ] { hostname | ipv4-address | ipv6 ipv6-address } [ port port-number ][ facility local-number ] export { aft | attack-defense | cmcc-sessionlog | cmcc-userlog | cntm | dns | dpi [ anti-virus | audit | data-filter | file-filter | iot-access-control | iot-flow-control | iot-format-check | iot-signal-control | ips | netshare | reputation | sandbox | terminal | traffic-policy | url-filter | waf | dga ] * | keepalive | lb [ dns-proxy | gslb | inbound | outbound | slb ] * | packet-filter | scd | security-policy | session | telecom-sessionlog | telecom-userlog | trusted-access { csap | iam [ authorization | notification ] } * | unicom-sessionlog | unicom-userlog } *
By default, no fast log output parameters are configured.
The value for the port-number argument must be the same as the port number configured on the log host. Otherwise, the log host cannot receive logs.
The facility local-number option takes effect only on the logs output in standard format from each service module as well as the AFT and NAT logs output in a carrier-customized format. If you do not specify this option, the value for the Facility field in the log headers output by each service module is used as the logging facility. For the AFT and NAT modules, logs output in CMCC and UNICOM formats will have a facility value of Local1, while logs output in TELECOM format will have a facility value of Local0.
4. (Optional.) Specify the source IP address for fast log output.
customlog host source interface-type interface-number
By default, the source IP address of fast output logs is the primary IP address of the outgoing interface.
If this command is configured, the primary IP address of the specified interface is used as the source IP address of fast output logs regardless of the outgoing interface.
Configure this command when you need to filter logs by source IP address on the log host.
5. (Optional.) Configure the timestamp of fast output logs to show the system time.
customlog timestamp localtime
By default, the timestamp of fast output logs shows the Greenwich Mean Time (GMT).
6. (Optional.) Specify a language for fast log output.
customlog language { chinese | english }
By default, fast logs are output in English.
Only some fields in the fast logs of certain service modules can be output in Chinese. For example, only the Application and Category fields in session logs support fast output in Chinese. For more information about the supported fields in service module logs, see the command reference.
Configuring fast log output to use the UTF-8 encoding
About this task
The fast log output module and the log host must use the same character set encoding. If they use different encodings, the log host cannot correctly display Chinese characters in the log messages received from the fast log output module. By default, fast log output uses the GB18030 encoding. You can perform this task to configure fast log output to use the UTF-8 encoding.
Procedure
1. Enter system view.
system-view
2. Configure fast log output to use UTF-8 encoding.
customlog character-encoding utf-8
By default, fast log output uses the GB18030 encoding.
Configuring fast output of logs to Kafka servers
About this task
The device supports outputting fast logs in Kafka format to a Kafka log server. When you have deployed a Kafka log server in the network, created a Kafka server on the device, and enabled output of fast logs to the Kafka server, the device will send fast logs in Kafka format to the Kafka log server.
A broker is a member of a Kafka server cluster. After you configure the IP address and port of a broker for receiving logs on the device side, the device will send logs in Kafka format to the specified address.
Restrictions and guidelines
The customlog kafka-server export command takes effect only when you have enabled fast log output for the corresponding modules using the customlog format command.
Procedure
1. Enter system view.
system-view
2. Create a Kafka server and enter its view.
kafka-server server-name
By default, no Kafka server exists.
3. Specify a Kafka broker.
broker { hostname | ipv4-address | ipv6 ipv6-address } [ port port-number]
By default, no Kafka broker is specified.
4. Associate a VPN instance with the Kafka server.
vpn-instance vpn-instance-name
By default, the Kafka server is associated with the public network.
5. Return to system view.
quit
6. Enable output of fast logs to the Kafka server.
customlog kafka-server server-name topic topic-name export dpi ips
By default, output of fast logs to the Kafka server is disabled.
Fast log output configuration examples
Example: Configuring fast log output to a log host
Network configuration
As shown in Figure 1, configure fast log output on the device to send session logs to the log server.
Procedure
1. Assign IP addresses to interface GigabitEthernet 1/0/2.
<Device> system-view
[Device] interface gigabitethernet 1/0/2
[Device-GigabitEthernet1/0/2] ip address 1.1.0.1 255.255.0.0
[Device-GigabitEthernet1/0/2] quit
2. Configure settings for routing.
This example configures a static route, and the next hop in the router is 1.1.0.2.
[Device] ip route-static 1.2.0.0 16 1.1.0.2
3. Add interface GigabitEthernet 1/0/2 to security zone untrust.
[Device] security-zone name untrust
[Device-security-zone-Untrust] import interface gigabitethernet 1/0/2
[Device-security-zone-Untrust] quit
4. Configure a security policy:
# Configure a rule named loglocalout to allow the device to send fast log output messages to the server.
[Device] security-policy ip
[Device-security-policy-ip] rule name loglocalout
[Device-security-policy-ip-1-loglocalout] source-zone local
[Device-security-policy-ip-1-loglocalout] destination-zone untrust
[Device-security-policy-ip-1-loglocalout] source-ip-host 1.1.0.1
[Device-security-policy-ip-1-loglocalout] destination-ip-host 1.2.0.1
[Device-security-policy-ip-1-loglocalout] action pass
[Device-security-policy-ip-1-loglocalout] quit
[Device-security-policy-ip] quit
5. Configure fast log output. Enable fast log output, configure log output to the log server, and enable logging for session creation and deletion. Enable IPv4 session logging in the inbound direction of the interface connected to the internal network.
[Device] customlog format session
[Device] customlog host 1.2.0.1 port 1000 export session
[Device] session log flow-begin
[Device] session log flow-end
[Device] interface gigabitethernet 1/0/1
[Device-GigabitEthernet1/0/1] session log enable ipv4 inbound
Verifying the configuration
On the server, verify that logs are received from the device successfully.