H3C SR6600&SR6600-X Routers
Troubleshooting Guide

SR6600 Router Series

SR6600-X Router Series

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Software version: R7606

Document version: 6W101-20181015

 

Contents

Introduction· 1

General guidelines· 1

Collecting log and operating information· 1

Collecting common log messages· 2

Collecting diagnostic log messages· 3

Collecting operating statistics· 3

Contacting technical support 5

Troubleshooting CPOS interfaces· 6

CPOS interface and serial interface down· 6

Symptom·· 6

Solution· 6

Data link layer protocol flapping or traffic forwarding failure on serial interface· 7

Symptom·· 7

Solution· 7

Related commands· 7

Troubleshooting FIP-600 modules· 9

Failure to ping a directly connected device· 9

Symptom·· 9

Solution· 9

Failure to forward traffic· 10

Symptom·· 10

Solution· 10

Packet loss· 11

Symptom·· 11

Solution· 11

Related commands· 11

Troubleshooting POS interfaces· 13

Interface physically down· 13

Symptom·· 13

Solution· 13

Data link layer down· 13

Symptom·· 13

Solution· 13

Related commands· 14

Troubleshooting SAP modules· 15

Failure of an interface to come up· 15

Symptom·· 15

Solution· 15

Failure to forward traffic· 15

Symptom·· 15

Solution· 15

Packet loss in data forwarding· 16

Symptom·· 16

Solution· 16

Related commands· 16

Troubleshooting WAN-IRF· 17

IRF fabric setup failure· 17

Symptom·· 17

Solution· 17

Cross-chassis traffic forwarding failure· 18

Symptom·· 18

Solution· 18

Cross-chassis packet loss problem·· 19

Symptom·· 19

Solution· 19

IRF split 19

Symptom·· 19

Solution· 19

Related commands· 20

Troubleshooting cards· 22

MPU or SR6602-X device startup failure· 22

Symptom·· 22

Solution· 22

LPU startup failure· 26

Symptom·· 26

Solution· 26

Related commands· 28

Troubleshooting transceiver modules· 28

Interface down· 28

Symptom·· 28

Solution· 28

Transceiver module type not recognized· 29

Symptom·· 29

Solution· 29

Interface or transceiver module alarm·· 29

Symptom·· 29

Solution· 29

Related commands· 29

Troubleshooting ports· 30

Failure to ping the directly connected device· 30

Symptom·· 30

Solution· 30

Failure to forward packets· 31

Symptom·· 31

Solution· 31

Packet dropping· 32

Symptom·· 32

Solution· 32

Related commands· 32

Troubleshooting active/standby MPU switchover 33

Exceptional standby MPU reboot 33

Symptom·· 33

Solution· 33

Exceptional active/standby MPU switchover 34

Symptom·· 34

Solution· 34

Related commands· 34

Troubleshooting interface modules· 34

Interface information not displayed· 35

Symptom·· 35

Solution· 35

Interface module information not displayed· 35

Symptom·· 35

Solution· 35

Related commands· 36

 

Introduction

This document provides information about troubleshooting common software and hardware problems with H3C SR6600/SR6600-X routers.

General guidelines

	IMPORTANT: To prevent a problem from causing loss of configuration, save the configuration each time you finish configuring a feature. For configuration recovery, regularly back up the configuration to a remote server.

 

When you troubleshoot the router, follow these general guidelines:

·     To help identify the cause of the problem, collect system and configuration information, including:

¡     Symptom, time of failure, and configuration.

¡     Network topology information, including the network diagram, port connections, and points of failure.

¡     Log messages and diagnostic information. For more information about collecting this information, see "Collecting log and operating information."

¡     Physical evidence of failure:

-     Photos of the hardware.

-     Status of the card, power, and fan status LEDs.

¡     Steps you have taken, such as reconfiguration, cable swapping, and reboot.

¡     Output from the commands executed during the troubleshooting process.

·     To ensure safety, wear an ESD wrist strap when you replace or maintain a hardware component.

·     If hardware replacement is required, use the release notes to verify the hardware and software compatibility.

Collecting log and operating information

	IMPORTANT: By default, the information center is enabled. If the feature is disabled, you must use the info-center enable command to enable the feature for collecting log messages.

 

Table 1 shows the types of files that the system uses to store operating log and status information. You can export these files by using FTP, TFTP, or USB.

These files are stored on the active MPU (in standalone mode) or global active MPU (in IRF mode). Multiple MPUs will have log files if active/standby or master/subordinate switchovers have occurred. You must collect log files from all these MPUs. To more easily locate log information, use a consistent rule to categorize and name files. For example, save log information files to a separate folder for each MPU, and include their chassis and slot numbers in the folder names.

Table 1 Log and operating information

Category	File name format	Content
Common log	logfileX.log	Command execution and operational log messages.
Diagnostic log	diagfileX.log	Diagnostic log messages about device operation, including the following items: ·     Parameter settings in effect when an error occurs. ·     Information about a card startup error. ·     Handshaking information between the MPU and LPU when a communication error occurs.
Operating statistics	file-basename.gz	Current operating statistics for feature modules, including the following items: ·     Device status. ·     CPU status. ·     Memory status. ·     Configuration status. ·     Software entries. ·     Hardware entries.

 

	NOTE: For common and diagnostic log files, the system automatically compresses them into .gz files when they are full.

 

Collecting common log messages

1.     Save common log messages from the log buffer to a log file.

By default, log files are saved in the logfile directory of the CF card on the active MPU (in standalone mode) or global active MPU (in IRF mode).

<Sysname> logfile save

The contents in the log file buffer have been saved to the file cfa0:/logfile/logfile8.log

2.     Identify the log files on each MPU:

# Display the log files on the active MPU (in standalone mode) or global active MPU (in IRF mode).

<Sysname> dir cfa0:/logfile/

Directory of cfa0:/logfile

   0 -rw-       21863 Jul 11 2013 16:00:37   logfile8.log

 

1021104 KB total (421552 KB free)

# Display the log files on each standby MPU:

¡     In standalone mode, display the log files on the standby MPU.

<Sysname> dir slot1#cfa0:/logfile/

Directory of slot1#cfa0:/logfile

   0 -rw-       21863 Jul 11 2013 16:00:37   logfile8.log

 

1021104 KB total (421552 KB free)

¡     In IRF mode, display the log files on each standby MPU.

<Sysname> dir chassis2#slot0#cfa0:/logfile/

Directory of chassis2#slot0#cfa0:/logfile

   0 -rw-       21863 Jul 11 2013 16:00:37   logfile8.log

 

1021104 KB total (421552 KB free)

 

	NOTE: If a subordinate chassis has two MPUs, make sure you identify and export the log files on both MPUs.

 

3.     Transfer the files to the desired destination by using FTP, TFTP, or USB. (Details not shown.)

Collecting diagnostic log messages

1.     Save diagnostic log messages from the diagnostic log file buffer to a diagnostic log file.

By default, diagnostic log files are saved in the diagfile directory of the CF card on each MPU.

<Sysname> diagnostic-logfile save

The contents in the diagnostic log file buffer have been saved to the file cfa0:/diagfile/diagfile18.log

2.     Identify the diagnostic log files on each MPU:

# Display the diagnostic log files on the active MPU (in standalone mode) or global active MPU (in IRF mode).

<Sysname> dir cfa0:/diagfile/

Directory of cfa0:/diagfile

   0 -rw-      161321 Jul 11 2013 16:16:00   diagfile18.log

 

1021104 KB total (421416 KB free)

# Display the diagnostic log files on each standby MPU:

¡     In standalone mode, display the diagnostic log files on the standby MPU.

<Sysname> dir slot1#cfa0:/diagfile/

Directory of slot1#cfa0:/diagfile

   0 -rw-      161321 Jul 11 2013 16:16:00   diagfile18.log

 

1021104 KB total (421416 KB free)

¡     In IRF mode, display the diagnostic log files on each standby MPU.

<Sysname> dir chassis2#slot0#cfa0:/diagfile/

Directory of chassis2#slot0#cfa0:/diagfile

   0 -rw-      161321 Jul 11 2013 16:16:00   diagfile18.log

 

1021104 KB total (421416 KB free)

 

	NOTE: If a subordinate chassis has two MPUs, make sure you identify and export the diagnostic log files on both MPUs.

 

3.     Transfer the files to the desired destination by using FTP, TFTP, or USB. (Details not shown.)

Collecting operating statistics

You can collect operating statistics by saving the statistics to a file or displaying the statistics on the screen.

When you collect operating statistics, follow these guidelines:

·     Log in to the device through a network port or management port instead of the console port, if possible. Network and management ports are faster than the console port.

·     Do not execute commands while operating statistics are being collected.

·     H3C recommends saving operating statistics to a file to retain the information.

To collect operating statistics:

1.     Disable pausing between screens of output if you want to display operating statistics on the screen. Skip this step if you are saving statistics to a file.

<Sysname> screen-length disable

2.     Collect operating statistics for multiple feature modules.

<Sysname> display diagnostic-information

Save or display diagnostic information (Y=save, N=display)? [Y/N] :

3.     At the prompt, choose to save or display operating statistics:

# To save operating statistics, enter y at the prompt and then specify the destination file path.

Save or display diagnostic information (Y=save, N=display)? [Y/N] :y

Please input the file name(*.tar.gz)[flash:/diag.tar.gz] :cfa0:/diag.tar.gz

Diagnostic information is outputting to cfa0:/diag.tar.gz.

Please wait...

Save successfully.

<Sysname> dir cfa0:/

Directory of cfa0:

…

   6 -rw-      898180 Jun 26 2013 09:23:51   diag.tar.gz

 

1021808 KB total (259072 KB free)

# To display operating statistics on the monitor terminal, enter n at the prompt.

Save or display diagnostic information (Y=save, N=display)? [Y/N] :N

===========================================================

  ===============display alarm===============

No alarm information.

=========================================================

  ===============display boot-loader===============

Software images on slot 0:

Current software images:

  cfa0:/SR6600-X-CMW710-BOOT-R7328_mrpnc.bin

  cfa0:/SR6600-X-CMW710-SYSTEM-R7328_mrpnc.bin

Main startup software images:

  cfa0:/SR6600-X-CMW710-BOOT-R7328_mrpnc.bin

  cfa0:/SR6600-X-CMW710-SYSTEM-R7328_mrpnc.bin

Backup startup software images:

  None

=========================================================

  ===============display counters inbound interface===============

Interface         Total (pkts)   Broadcast (pkts)   Multicast (pkts)  Err (pkts)

BAGG1                        0                  0                  0           0

GE4/0/1                      0                  0                  0           0

GE4/0/2                      2                  2                  0           0

GE4/0/3                      0                  0                  0           0

GE4/0/4                      0                  0                  0           0

GE4/0/5                      0                  0                  0           0

GE4/0/6                      0                  0                  0           0

GE4/0/7                      0                  0                  0           0

GE4/0/8                      0                  0                  0           0

GE4/0/9                      0                  0                  0           0

GE4/0/10                     0                  0                  0           0

……

Contacting technical support

If you cannot resolve a problem after using the troubleshooting procedures in this document, contact H3C Support. When you contact an authorized H3C support representative, be prepared to provide the following information:

·     Information described in "General guidelines."

·     Product serial numbers.

This information will help the support engineer assist you as quickly as possible.

The following is the contact information for H3C Support:

·     Telephone number—400-810-0504.

·     E-mail—service@h3c.com

Troubleshooting CPOS interfaces

This section provides troubleshooting information for common CPOS interface problems.

CPOS interface and serial interface down

Symptom

A CPOS interface directly connected to another device is physically down. The serial interface created through channelization on the CPOS interface is physically down, and the data link layer protocol state of the serial interface is down.

Solution

To resolve the problem:

1.     Execute the display controller cpos interface-name command to view the status information of the CPOS interface.

2.     If the CPOS interface is down and an LOS or LOF error has been generated, verify the following items:

¡     The optical fiber is connected correctly.

¡     The two ends use different clock mode settings (master at one end and slave at the other end).

3.     If the CPOS interface is up but the serial interface is physically down, execute the display controller cpos interface-name command. Determine whether the channel to which the serial interface belongs has alarms generated.

If the channel to which the serial interface belongs has no alarms generated, the shutdown command might have been executed for the serial interface. Execute the display interface serial interface-number command to view the status of the serial interface. If the serial interface status is DOWN (Administratively), execute the undo shutdown command to bring up the serial interface.

4.     If the serial interface is physically up but the data link layer protocol state is down, execute the display interface serial interface-number command. Verify that the HDLC chip of the CPOS interface collects packet statistics correctly.

5.     If the problem persists, collect the following statistics and contact H3C Support:

¡     Execute the debugging physical packet command to enable packet debugging on both devices to verify that they can correctly send and receive packets. Collect packet sending and receiving statistics.

¡     Execute the display hardware internal module cpos interface-number statistics command in probe view to collect statistics of the CPOS interface.

¡     Execute the display hardware internal module cpos interface-number reg 0 command in probe view to collect the CPLD information of the CPOS interface.

¡     Execute the display hardware internal module cpos interface-number reg 1 command in probe view to collect the FPGA information of the CPOS interface.

¡     Collect the chip register information of the CPOS interface by executing the following commands in probe view:

-     display hardware internal module cpos interface-number reg 2.

-     display hardware internal module cpos interface-number reg 3.

-     display hardware internal module cpos interface-number reg 4.

¡     Execute the display hardware internal module serial interface-number statistics command in probe view to collect statistics of the serial interface, including the CPU statistics.

¡     Execute the display hardware internal module cpos interface-number statistics command in probe view to collect the FPGA information of the interface module.

Data link layer protocol flapping or traffic forwarding failure on serial interface

Symptom

The data link layer protocol state flaps or the traffic forwarding fails on the serial interface.

Solution

To resolve the problem:

1.     If the serial interface is physically down, follow the steps described in "CPOS interface and serial interface down" to resolve the problem.

2.     If the problem persists, collect the following statistics and contact H3C Support:

¡     Execute the display hardware internal module interface-number command in probe view to determine whether the serial interface has received or sent error packets and collect packet statistics.

¡     Use the debugging physical packet command to enable packet debugging on both devices to verify that they can correctly send and receive packets. Collect packet sending and receiving statistics.

¡     Execute the display hardware internal module cpos interface-number statistics command in probe view to collect statistics of the CPOS interface.

¡     Execute the display hardware internal module cpos interface-number reg 0 command in probe view to collect the CPLD information of the CPOS interface.

¡     Execute the display hardware internal module cpos interface-number reg 1 command in probe view to collect the FPGA information of the CPOS interface.

¡     Collect the chip register information of the CPOS interface by executing the following commands in probe view:

-     display hardware internal module cpos interface-number reg 2.

-     display hardware internal module cpos interface-number reg 3.

-     display hardware internal module cpos interface-number reg 4.

¡     Execute the display hardware internal module serial interface-number statistics command in probe view to collect statistics of the serial interface, including the CPU statistics.

¡     Execute the display hardware internal module cpos interface-number statistics command in probe view to collect the FPGA information of the interface module.

Related commands

This section lists the commands that you might use for troubleshooting CPOS interfaces.

 

Command	Description
clock master	Sets the clock mode of the CPOS interface to master.
debugging physical packet	Enables packet debugging on the device.
display controller cpos interface-name	Displays status information for CPOS interfaces, including alarms and errors that occurred to the regenerator section, multiplex section, and higher-order path.
display hardware internal module interface-name interface-number reg { 0 | 1 | 2 |3 | 4 }	Displays hardware register information about CPOS interfaces.
display hardware internal module interface-name interface-number statistics	Displays maintenance information about CPOS interfaces.
display interface serial interface-number	Displays information about serial interfaces.
undo shutdown	Brings up the CPOS interface.

 

Troubleshooting FIP-600 modules

This section provides troubleshooting information for common FIP-600 module problems.

Failure to ping a directly connected device

Symptom

A FIP-600 module fails to ping a directly connected device.

Solution

To resolve the problem:

1.     Use the display interface command to view information about the output interface of the ping packets and verify the following items:

¡     The interface is up.

¡     The interface counters are collecting packet statistics correctly. For example, if packet error statistics are correctly collected, cable and interface problems can be ruled out.

2.     Use the display arp all command to verify that the router has an ARP entry for the directly connected interface.

If no such ARP entry exists, execute the debugging arp packet command on both devices to determine whether they can correctly receive and send ARP packets.

3.     In probe view, use the debugging hardware internal fdp cdat slot slot-num debug { ingress | egress } command to view debugging information of packets sent and received by the CPU data channel interface.

The first 32 bytes represent the header fields, followed by 32 bytes of payload. You can filter the packets to be displayed based on criteria such as interface or packet length.

To learn the reason why the packets are sent to the CPU, contact H3C Support.

4.     In probe view, execute the display hardware internal fdp cdat slot slot-num statistics command to view the packet statistics of the CPU data channel. To view detailed statistics such as vCPU-based statistics, execute the display hardware internal fdp cdat slot slot-num statistics 4 command.

5.     In probe view, execute the display hardware internal fdp cdat slot slot-num statistics 8 command to view the packet drop statistics of the CPU and the logic module. Contact H3C Support to learn the causes for the packet drop.

6.     In probe view, execute the display hardware internal fdp cdat slot slot-num statistics 80 command to view statistics of the logic module. Contact H3C Support to learn the causes if packet drop exists.

7.     In probe view, execute the display hardware internal fdp cdat slot slot-num statistics 100 command to view statistics about the CPU data channel interface.

8.     In probe view, execute the display hardware internal fdp flow slot slot-num statistic command to view statistics about the Layer 3 packets sent to the CPU. If packet error or loss occurs, contact H3C Support to learn the causes.

9.     In probe view, execute the display hardware internal fdp cdat slot slot-num status 80 command to view statistics of the logic module. If the FIFO status is not empty when there is no traffic, the FIFO queue is congested. Contact H3C Support to diagnose the problem.

10.     If the logic module and the CPU can correctly send and receive packets, use the software forwarding diagnostic techniques to diagnose the problem. For example, verity that the correct routing entry exists in the routing table.

11.     Execute the display hardware internal nae slot slot-num freein command in probe view to collect hardware information and contact H3C Support to locate the problem.

Failure to forward traffic

Symptom

A router installed with FIP-600 modules fails to forward traffic as an intermediate device.

Solution

To resolve the problem:

1.     Ping a directly connected device. If the ping operation fails, follow the procedure described in "Failure to ping a directly connected device" to resolve the problem.

2.     In probe view, execute the display hardware internal fdp cdat slot slot-num statistics command to view the packet statistics of the CPU data channel. The following conditions might exist:

¡     If the packet output rate of the CPU decreases significantly, the packets are dropped or transparently sent to the MPU.

¡     If the packet output rate and input rate are approximately the same, the packets are sent to the CPU for normal forwarding.

¡     If the packet input rate is significantly lower than the packet output rate, the packets are not sent to the CPU.

3.     If the packets are sent to the CPU, execute the following command in probe view to view the debugging information of packets sent and received by the CPU data channel interface:

debugging hardware internal fdp cdat slot slot-num debug { ingress | egress }

The first 32 bytes represent the header fields, followed by 32 bytes of payload. You can filter the packets to be displayed based on criteria such as interface or packet length.

To learn the causes why the packets are sent to the CPU, contact H3C Support.

4.     If the packets are dropped, perform the following tasks:

¡     Execute the display hardware internal fdp cdat slot slot-num statistics 8 command in probe view to view the packet loss statistics. Contact H3C Support to learn the causes for the packet loss.

¡     Execute the display hardware internal fdp flow slot slot-num statistic command to view statistics about the Layer 3 packets sent to the CPU. If packet error or drop exists, contact H3C Support to diagnose the problem.

5.     In probe view, execute the display hardware internal fdp cdat slot slot-num statistics 8 command to view the packet loss statistics of the CPU and the logic module. Contact H3C Support to learn the causes for the packet loss.

6.     In probe view, execute the debugging hardware internal fdp cdat slot slot-num bypass interface command to enable bypass logic on the interface where the packets to be forwarded are received. If traffic can be forwarded correctly after the feature is enabled, an error has occurred to the logic module. Contact H3C Support to diagnose the problem.

7.     In probe view, execute the display hardware internal fdp cdat slot slot-num statistics 80 command to view statistics of the logic module. If packet drops exist, contact H3C Support to learn the causes.

8.     In probe view, execute the display hardware internal fdp cdat slot slot-num status 80 command to view statistics of the logic module. If the FIFO status is not empty when there is no traffic, the FIFO queue is congested. Contact H3C Support to diagnose the problem.

9.     If the logic module and the CPU can correctly send and receive packets, use the software forwarding diagnostic techniques to diagnose the problem. For example, verity that the correct routing entry exists in the routing table.

10.     If the problem persists, contact H3C Support.

Packet loss

Symptom

A large number of packets are dropped by a FIP-600 module during the forwarding process.

Solution

To resolve the problem:

1.     In probe view, execute the display hardware internal fdp cdat slot slot-num statistics command to view the packet statistics of the CPU data channel. The following conditions might exist:

¡     If the packet output rate and input rate are approximately the same, the packets are forwarded by the logic module to the CPU for normal forwarding.

¡     If the packet input rate is significantly lower than the output rate, the packets are not sent to the CPU.

2.     If the packets are sent to the CPU, execute the following command in probe view to view the debugging information of packets sent and received by the CPU data channel interface:

debugging hardware internal fdp cdat slot slot-num debug { ingress | egress }

The first 32 bytes represent the header fields, followed by 32 bytes of payload. You can filter the packets to be displayed based on criteria such as interface or packet length.

To learn the causes why the packets are sent to the CPU, contact H3C Support.

3.     If the packets are dropped by the logic module before being sent to the CPU, execute the display hardware internal fdp cdat slot slot-num statistics 8 command in probe view to view the packet loss statistics of the logic module. Contact H3C Support to learn the causes for the packet loss.

4.     In probe view, execute the display hardware internal fdp cdat slot slot-num statistics 80 command to view statistics of the logic module. If packet drops exist, contact H3C Support to learn the causes for the packet drops.

5.     If the problem persists, contact H3C Support.

Related commands

This section lists the commands that you might use for troubleshooting FIP-600 modules.

 

Command	Description
debugging hardware internal fdp cdat slot slot-num bypass	Enables the bypass logic feature.
debugging hardware internal fdp cdat slot slot-num debug	Enables packet debugging for the CPU data channel.
display hardware internal fdp cdat slot slot-num statistics	Displays the CPU data channel statistics.
display hardware internal fdp cdat slot slot-num status	Displays the CPU data channel status information.
display hardware internal fdp flow slot slot-num statistic	Displays statistics about received Layer 3 packets.
display hardware internal nae slot slot-num freein	Displays information about the CPU hardware buffer pools.
reset hardware internal fdp cdat slot slot-num statistics	Clears the CPU data channel statistics.
reset hardware internal fdp flow slot slot-num statistic	Clears statistics about received Layer 3 packets.

 

Troubleshooting POS interfaces

This section provides troubleshooting information for common POS interface problems.

Interface physically down

Symptom

The physical state of the POS interface is down.

Solution

To resolve the problem:

1.     Verify that the two ends use different clock mode settings (master at one end and slave at the other end).

2.     Verify that the POS interface and its optical module have matching transmission rate.

3.     Verify that the two ends use the same framing format.

4.     Use the display interface pos interface-number command to view the alarms generated on the interface.

¡     If an AIS alarm was generated, verify that the interface has correct overhead bytes settings.

¡     If an LOS or LOF error was generated, verify that the optical module on the two ends and the optical fiber are working correctly.

5.     If the problem persists, contact H3C Support.

Data link layer down

Symptom

The physical state of the POS interface is up, but the data link layer state is down.

Solution

This symptom might occur due to negotiation failures caused by packet loss.

To resolve the problem:

1.     Execute the debugging physical packet all interface pos command on both devices to identify the faulty device.

2.     On the faulty device, perform the following tasks:

a.     Execute the reset counters interface pos interface-number command to clear statistics for the POS interface.

b.     In probe view, execute the display hardware internal module pos interface-number statistics command to view packet statistics.

c.     Perform the following tasks:

-     Use the display hardware internal module pos interface-number reg 3 command in probe view to verify that the physical layer configuration is correct.

-     Use the display hardware internal module pos interface-number reg 1 command in probe view to verify that the FPGA configuration is correct.

3.     If the problem persists, contact H3C Support.

Related commands

This section lists the commands that you might use for troubleshooting POS interfaces.

 

Command	Description
debugging physical packet	Enables packet debugging on the device.
display hardware internal module interface-name interface-number reg 1	Displays FPGA information about POS interfaces.
display hardware internal module interface-name interface-number reg 3	Displays physical layer information about POS interfaces.
display hardware internal module interface-name interface-number statistics	Displays maintenance information about POS interfaces.
reset counters interface pos interface-number	Clears POS interface statistics.

 

Troubleshooting SAP modules

This section provides troubleshooting information for common SAP module problems.

Failure of an interface to come up

Symptom

An interface on a SAP module fails to come up.

Solution

To resolve the problem:

1.     Verify that the SAP module has started up correctly.

2.     Verify that the interface is not shut down manually. If it is shut down manually, execute the undo shutdown command to bring up the interface.

3.     Check the transceiver module and fiber for damages and connection faults. Adjust the fiber connector or reconnect the fiber connector to the transceiver module.

4.     Verify that the transceiver module and the fiber are compatible.

5.     If the problem persists, contact H3C Support.

Failure to forward traffic

Symptom

A SAP module fails to forward traffic.

Solution

To resolve the problem:

1.     Verify that the device where the SAP module resides is operating correctly.

2.     Verify that the traffic forwarding configuration on the SAP module is correct.

3.     Verify that both the inbound and outbound interfaces are up. If an interface is not up, see "Failure of an interface to come up."

4.     Execute the display interface command in system view to determine whether the interfaces are sending and receiving packets correctly.

¡     If the interface is receiving packets correctly, verify that the directly connected device is operating correctly. If the peer device is not operating correctly, troubleshoot the peer device.

¡     If the interface fails to send packets or packet loss occurs in data transmitting, execute the debugging physical packet command to display information about packets sent to the CPU.

-     If the command displays packet information, the packets are sent to the CPU correctly.

-     If the command does not display packet information, packets are not sent to the CPU. Enter probe view and execute the bcm slot chip show/counter command to display statistics about packet loss and contact H3C Support.

5.     If the packets are discarded after being sent to the CPU, contact H3C Support.

6.     Execute the bcm slot chip l3/defip/show command in probe view to determine whether the routing entries are correct and whether the packets match the routing entries. If not, contact H3C Support

7.     If the problem persists, contact H3C Support.

Packet loss in data forwarding

Symptom

Packet loss occurs in data forwarding.

Solution

To resolve the problem, see "Failure to forward traffic."

Related commands

This section lists the commands that you might use for troubleshooting SAP modules.

 

Command	Description
bcm slot chip l3/defip/show	Displays information about routing entries.
bcm slot chip l3/l3table/show	Displays ARP entries.
bcm slot chip show/counter	Displays traffic statistics and data rate for an interface.
debugging physical packet [all | input | output ] interface interface-number	Displays information about the packets sent to the CPU.
display counters { inbound | outbound } interface	Displays traffic statistics on an interface.
display interface interface-number	Displays information about an interface.
reset counters interface	Clears traffic statistics on an interface.

 

Troubleshooting WAN-IRF

This section provides troubleshooting information for common WAN-IRF problems. In this section, WAN-IRF is referred to as IRF.

IRF fabric setup failure

Symptom

Two routers cannot form an IRF fabric.

Solution

To resolve the problem:

1.     Execute the display device command on each device to verify that the devices are operating in IRF mode and have unique IRF member IDs:

¡     If the Chassis field is not available for a device, use the chassis convert mode irf command to enable IRF mode for the device.

¡     If the Chassis field is available, check the value for this field. Make sure each device has a unique value for this field and the value must be 1 or 2. You can use the irf member renumber command to configure the value (IRF member ID). The value takes effect after a reboot.

2.     Execute the display irf configuration command to verify that IRF physical interfaces have been bound to IRF-port 1 on one device and to IRF-port 2 on the other device.

Make sure the IRF port bindings meet the binding requirements in the virtual technologies configuration guide for the router.

3.     Verify that the IRF physical interfaces are connected correctly:

 

	IMPORTANT: When you connect two neighboring IRF members, connect the physical interfaces of IRF-port 1 on one member to the physical interfaces of IRF-port 2 on the other.

 

a.     Verify that no intermediate devices are deployed between the member devices.

b.     Verify that the recommended connection media are used to connect IRF physical interfaces:

-     Use straight-through or crossover copper Ethernet cables to connect copper Ethernet ports for a short-distance connection.

-     Use transceiver modules and fibers to connect fiber Ethernet ports for a long-distance connection.

-     Make sure the transceiver modules at the two ends of an IRF link are the same type. For more information about transceiver modules, see the device installation guide.

c.     Verify that the physical IRF connections are consistent with the IRF port bindings. If there are inconsistencies, reconfigure the IRF port bindings or reconnect the IRF physical interfaces.

4.     Execute the display irf link command to verify that a minimum of one IRF link is up for the devices.

5.     In probe view, execute the display hardware internal wanirf ipc pkt-info chassis chassis-number slot slot-number interface-type interface-number command for each IRF physical interface. This command displays statistics about IPC packets sent and received by an interface.

¡     If the command does not display any packet statistics, verify that the irf-port-configuration active command has been executed.

¡     If the command displays packet loss statistics, contact H3C Support to identify the cause.

6.     In probe view, execute the display hardware internal wanirf topoinfo chassis chassis-number slot slot-number command to check the status of each slot that provides IRF physical interfaces. Make sure a minimum of one slot is active. If no slot is active, contact H3C Support to identify the cause.

7.     In probe view, execute the display hardware internal wanirf portinfo chassis chassis-number slot slot-number command to collect the IRF physical interface status and outgoing interface information. If any abnormalities occur, contact H3C Support to identify the cause.

8.     In probe view, execute the display hardware internal wanirf stm debugging chassis chassis-number slot slot-number all on command. Use this command to collect the STM packet debugging information on each slot that provides IRF physical interfaces. Send the collected information to H3C Support to verify that the packet format and contents are correct.

9.     In probe view, execute the display hardware internal wanirf stm pkt-info chassis chassis-number slot slot-number command to display STM packet statistics.

¡     (SR6602-X routers.) Use this command to verify that STM packets can be forwarded from one member device to the other member device.

¡     (SR6600 and SR6600-X routers except for SR6602-X.) Use this command to verify that STM packets can be forwarded from the active MPU on a member device to the active MPU on the other member device.

10.     If the problem persists, contact H3C Support.

Cross-chassis traffic forwarding failure

Symptom

Traffic cannot be forwarded across chassis in an IRF fabric.

Solution

To resolve the problem:

1.     Verify that the device configuration is correct and the IRF ports are up.

2.     Execute the display ip statistics chassis chassis-number slot slot-number command to verify that the source slot has sent out the traffic successfully.

If the traffic forwarding fails, contact H3C Support to identify the cause.

3.     In probe view, execute the display hardware internal ibd pkt-info chassis chassis-number slot slot-number verbose command for the slots on the traffic forwarding path. Verify that traffic from the source slot can be forwarded to the specified slots. If traffic forwarding fails, contact H3C Support to identify the cause.

4.     In probe view, execute the display hardware internal wanirf ibd pkt-info chassis chassis-number slot slot-number interface-type interface-number command for each IRF physical interface of the member devices. Use this command to verify that an IRF physical interface can send and receive IBD packets. If a physical interface cannot send or receive packets, contact H3C Support to identify the cause.

5.     In probe view, execute the display hardware internal wanirf ibd debugging chassis chassis-number slot slot-number all on command to collect IBD packet debugging information for IRF physical interfaces. Verify that the packet format and contents are correct.

6.     If the peer IRF physical interfaces can send and receive IBD packets, execute the command in step 3. Use the command to verify that the local traffic can be forwarded to the destination slot.

7.     In probe view, execute the display hardware internal module interface-name statistics command to check the outgoing interface statistics for each IRF physical interface. If the statistics are incorrect, contact H3C Support to identify the cause.

Cross-chassis packet loss problem

Symptom

Packet loss occurs for cross-chassis traffic.

Solution

To resolve the problem:

1.     Use the methods described in "Cross-chassis traffic forwarding failure" to identify the packet loss issue and remove the issue.

2.     If the problem persists, contact H3C Support.

IRF split

Symptom

An IRF fabric splits. An IRF fabric splits if the member devices do not receive packets from each other within 20 seconds.

Solution

To resolve the problem:

1.     Execute the display irf link command to verify that the IRF links are up. If an IRF link is down, check the cable, or the transceiver modules and fiber, and identify whether the slots that provide the IRF physical interfaces have rebooted.

2.     In probe view, execute the display hardware internal wanirf portinfo chassis chassis-number slot slot-number command to check the IRF physical interface status and outgoing interface information. If any abnormalities occur, contact H3C Support to identify the cause.

3.     Execute the display cpu-usage command to check the CPU usage. Verify that the CPUs are not overwhelmed.

4.     In probe view, execute the display hardware internal wanirf ipc pkt-info chassis chassis-number slot slot-number interface-type interface-number command to identify whether an IRF physical interface has IPC packet loss statistics. If an IRF physical interface has packet loss statistics, contact H3C Support to identify the cause.

5.     In probe view, execute the display hardware internal wanirf ipc debugging chassis chassis-number slot slot-number all on command to check IPC packet debugging information for IRF physical interfaces. Verify that the packet format and contents are correct.

6.     In probe view, execute the display hardware internal wanirf ipc sendpkt chassis chassis-number slot slot-number unicast chassis chassis-number slot slot-number pkt-length pkt-number command.

¡     (SR6602-X routers.) Use this command to verify that IPC packets can be forwarded between two member devices.

¡     (SR6600 and SR6600-X routers except for SR6602-X.) Use this command to verify that IPC packets can be forwarded between the active MPU and a slot that provides IRF physical interfaces.

7.     In probe view, execute the debug stack show globalvariable slot slot-number command, and collect and send the output information to H3C Support.

¡     (SR6602-X routers.) Use this command to display IRF packet information for the member devices.

¡     (SR6600 and SR6600-X routers except for SR6602-X.) Use this command to display IRF packet information for the MPUs.

8.     In probe view, execute the display hardware internal wanirf stm pkt-info chassis chassis-number slot slot-number command.

¡     (SR6602-X routers.) Use this command to verify that STM packets can be forwarded from one member device to the other member device.

¡     (SR6600 and SR6600-X routers except for SR6602-X.) Use this command to verify that STM packets can be forwarded from the active MPU on a member device to the active MPU on the other member device.

9.     If the problem persists, contact H3C Support.

Related commands

This section lists the commands that you might use for troubleshooting WAN-IRF.

 

Command	Remarks
debug stack show globalvariable slot slot-number	(SR6602-X routers.) Displays IRF packet information for a member device. (SR6600 and SR6600-X routers except for SR6602-X.) Displays IRF packet information for an MPU.
display hardware internal wanirf ibd debugging chassis chassis-number slot slot-number { receive | send | all on | off }	Displays IBD packet debugging information of IRF physical interfaces for a slot.
display hardware internal wanirf ibd pkt-info chassis chassis-number slot slot-number interface-type interface-number	Displays IBD packet statistics for an IRF physical interface.
display hardware internal wanirf ibd sendpkt chassis chassis-number slot slot-number unicast chassis chassis-number slot slot-number pkt-len pkt-number	Specifies any two slots to send IBD test packets to each other.
display hardware internal wanirf ipc debugging chassis chassis-number slot slot-number { receive | send | all on | off }	Displays IPC packet debugging information of IRF physical interfaces for a slot.
display hardware internal wanirf ipc pkt-info chassis chassis-number slot slot-number interface-type interface-number	Displays IPC packet statistics for an IRF physical interface.
display hardware internal wanirf ipc sendpkt chassis chassis-number slot slot-number unicast chassis chassis-number slot slot-number pkt-len pkt-number	Specifies any two slots to send IPC test packets to each other.
display hardware internal wanirf portinfo chassis chassis-number slot slot-number	Displays IRF port information for a slot.
display hardware internal wanirf stm debugging chassis chassis-number slot slot-number { receive | send | all on | off }	Displays STM packet debugging information of IRF physical interfaces for a slot.
display hardware internal wanirf stm pkt-info chassis chassis-number slot slot-number	(SR6602-X routers.) Displays STM packet statistics on a member device. (SR6600 and SR6600-X routers except for SR6602-X.) Displays STM packet statistics on an MPU.
display hardware internal wanirf topoinfo chassis chassis-number slot slot-number	Displays IRF topology information for a slot.
reset hardware internal wanirf ibd pkt-info chassis chassis-number slot slot-number interface-type interface-number	Clears IBD packet statistics for an IRF physical interface.
reset hardware internal wanirf ipc pkt-info chassis chassis-number slot slot-number interface-type interface-number	Clears IPC packet statistics for an IRF physical interface.
reset hardware internal wanirf stm pkt-info chassis chassis-number slot slot-number	(SR6602-X routers.) Clears STM packet statistics for a member device. (SR6600 and SR6600-X routers except for SR6602-X.) Clears STM packet statistics for an MPU.

 

Troubleshooting cards

This section provides troubleshooting information for common card problems.

MPU or SR6602-X device startup failure

Symptom

The MPU (RSE-X3 or RPE-X3) or an SR6602-X cannot start up.

Solution

To resolve the problem:

1.     Power cycle the device, and observe the RUN LED on the MPU or device while the device is starting up.

¡     If the LED flashes at the frequency of 8 Hz, or the LED flashes over 5 seconds and then goes off, the device has started up correctly.

¡     If the LED flashes at the frequency of 1 Hz or is always on after you power cycle the device, the device has hardware problems. Go to step 3.

¡     If the LED is always off after you power cycle the device, the device has not been powered on or the basic BootWare segment is corrupt. Go to step 2.

2.     Verify that the device has been powered on:

a.     Check whether the fans are rotating. If the fans are rotating, the device has been powered on.

b.     (For routers except for SR6602-X.) If the device is not the fans are not rotating, wait for a while and remove the MPU. Identify whether the temperature of the CPU cooler on the MPU is higher than the room temperature. If the temperature of the CPU cooler is higher than the room temperature, the device has been powered on.

c.     If the device has not been powered on, perform the following tasks:

-     Verify that the power source is providing power correctly.

-     Verify that the power cord is connected firmly.

-     Check the installed power modules. If one or more power modules have problems, replace the power modules.

3.     If the device has been powered on, check the information displayed on the console terminal.

¡     If the following system startup information appears, the basic BootWare segment is running correctly:

System is starting...

Press Ctrl+D to access BASIC-BOOTWARE MENU...

Press Ctrl+T to start memory test

Booting Normal Extended BootWare

The Extended BootWare is self-decompressing........Done.

****************************************************************************

*                                                                          *

*                    H3C SR66 BootWare, Version 2.05                       *

*                                                                          *

****************************************************************************

Copyright (c) 2004-2016 Hangzhou H3C Technologies Co., Ltd.

Compiled Date       : Jan 18 2016

CPU Type            : P2020

CPU L1 Cache        : 32KB

CPU Clock Speed     : 1000MHz

Memory Type         : DDR3 SDRAM

Memory Size         : 2048MB

Memory Speed        : 800MHz

BootWare Size       : 1024KB

Flash Size          : 8MB

NVRAM Size          : 128KB

BASIC CPLD Version  : 3.0

EXTENDED CPLD Version: 2.0

PCB Version         : Ver.B

¡     If the system startup information is not displayed, there might be memory or CPU problems. If the device is using an RSE-X3 or RPE-X3 MPU, remove the memory module, and check the console terminal for the following information:

RAM initialization failed

Fatal error! Please reboot the board.

-     If the RAM initialization failure alert is not displayed, there might be CPU problems. Contact H3C Support for further troubleshooting.

-     If the RAM initialization failure alert is displayed, contact H3C Support to replace the memory module.

¡     If the information like the following appears, the memory self-test failed.

readed value is 75555555 , expected value is 55555555

DRAM test fails at: 5ff80020

Fatal error! Please reboot the board.

Reboot the device. If the memory controller is not restored to the normal state, the memory or the memory channel hardware circuits might have problems. Contact H3C Support.

¡     If the following information appears, both the extended BootWare segment and backup extended BootWare segment have problems. You must manually upgrade the extended BootWare segment.

System start booting...

Boot ROM program does not exist.

Now start to download program.

 

======================<BASIC-BOOTWARE MENU(Ver 1.12)>=====================

|<1> Modify Serial Interface Parameter                                   |

|<2> Update Extend BootWare                                              |

|<3> Update Full BootWare                                                |

|<4> Boot Extend BootWare                                                |

|<5> Boot Backup Extend BootWare                                         |

|<0> Reboot                                                              |

==========================================================================

Enter your choice(0-5):

¡     If no information is displayed after "Booting Normal Extend BootWare... ", contact H3C Support.

System start booting...

Booting Normal Extend BootWare........

4.     Verify that the Comware image files have been correctly loaded.

¡     If the following information is displayed, the Comware image files have been loaded and decompressed successfully:

****************************************************************************

*                                                                          *

*                    H3C SR66 BootWare, Version 2.05                       *

*                                                                          *

****************************************************************************

Copyright (c) 2004-2016 Hangzhou H3C Technologies Co., Ltd.

Compiled Date       : Jan 18 2016

CPU Type            : P2020

CPU L1 Cache        : 32KB

CPU Clock Speed     : 1000MHz

Memory Type         : DDR3 SDRAM

Memory Size         : 2048MB

Memory Speed        : 800MHz

BootWare Size       : 1024KB

Flash Size          : 8MB

NVRAM Size          : 128KB

BASIC CPLD Version  : 3.0

EXTENDED CPLD Version: 2.0

PCB Version         : Ver.B

 

 

BootWare Validating...

Press Ctrl+B to access EXTENDED-BOOTWARE MENU...

Loading the main image files...

Loading file flash:/SR6600-cmw710-system-test.bin........................

............................................................................

............................................................................

.........................Done.

Loading file flash:/SR6600-cmw710-boot-test.bin..........................

....Done.

 

Image file flash:/SR6600-cmw710-boot-test.bin is self-decompressing......

....................................................Done.

System image is starting...

Line aux1 is available.

 

 

Press ENTER to get started.

¡     If the following information is displayed, the Comware image files do not exist and you must download the files:

****************************************************************************

*                                                                          *

*                    H3C SR66 BootWare, Version 2.05                       *

*                                                                          *

****************************************************************************

Copyright (c) 2004-2016 Hangzhou H3C Technologies Co., Ltd.

 

Compiled Date       : Jan 18 2016

CPU Type            : P2020

CPU L1 Cache        : 32KB

CPU Clock Speed     : 1000MHz

Memory Type         : DDR3 SDRAM

Memory Size         : 2048MB

Memory Speed        : 800MHz

BootWare Size       : 1024KB

Flash Size          : 8MB

NVRAM Size          : 128KB

BASIC CPLD Version  : 3.0

EXTENDED CPLD Version: 2.0

PCB Version         : Ver.B

 

BootWare Validating...

Application program does not exist.

Please input BootWare password:

¡     If the following information is displayed, the Comware image files have checksum errors and you must download the files to the storage medium again:

****************************************************************************

*                                                                          *

*                    H3C SR66 BootWare, Version 2.05                       *

*                                                                          *

****************************************************************************

Copyright (c) 2004-2016 Hangzhou H3C Technologies Co., Ltd.

Compiled Date       : Jan 18 2016

CPU Type            : P2020

CPU L1 Cache        : 32KB

CPU Clock Speed     : 1000MHz

Memory Type         : DDR3 SDRAM

Memory Size         : 2048MB

Memory Speed        : 800MHz

BootWare Size       : 1024KB

Flash Size          : 8MB

NVRAM Size          : 128KB

BASIC CPLD Version  : 3.0

EXTENDED CPLD Version: 2.0

PCB Version         : Ver.B

 

BootWare Validating...

Press Ctrl+B to enter extended boot menu...

Starting to get the main application file--cfa0:/system.bin!................

..........................................................................

..........................................................................

..........................................................................

Something wrong with the file.

5.     Observe the Comware image file startup process.

¡     If the following information is displayed, the system image is missing. You must manually download the system image.

Loading the main image files...

Loading file cfa0:/boot.bin.................................................

...................................Done.

<boot>

¡     If one of the following symptoms appears, contact H3C Support:

-     The system stops at the "System image is starting..." message.

-     After displaying the "System image is starting..." message, the system does not enter the CLI but repeatedly reboots.

-     The system displays the "Press ENTER to get started" message but you cannot enter the CLI by pressing Enter.

-     You can enter the CLI but the system reboots after a while.

LPU startup failure

Symptom

The LPU cannot start up correctly.

Solution

To resolve the problem:

1.     Check the RUN LED on the LPU.

¡     If the LED flashes at the frequency of 8 Hz, or the LED flashes over 5 seconds and then goes off, the LPU has started up correctly.

¡     If the LED flashes at the frequency of 1 Hz or is always on, the LPU has hardware problems. Go to step 3.

¡     If the LED is always off, the LPU has not been powered on or the LPU has a problem. Go to step 2.

2.      If the LED is always off, verify that the LPU has been powered on:

a.     Identify whether the LPU has been powered on.

<System>display device

Slot No.  Board type    Status       Primary    SubSlots

 ---------------------------------------------------------------------

 0         RSE-X3        Startup      Standby    0

 1         RSE-X3        Normal       Master     0

 2         N/A           Absent       N/A        N/A

 3         FIP-300       Wait         N/A        1

The values for the Status field include:

-     Normal—The LPU is operating correctly.

-     Startup—The LPU has been powered on and is starting up.

-     Wait—The LPU has not been powered on.

b.     If the LPU has not been powered on, identify whether the available power is sufficient.

[System-probe]display hardware internal sysm power-management

System Power Total       : 650  watts

System Power Used        : 150  watts

System Power Available   : 300  watts

System Power Per Unit    : 650  watts

System Power Reserved    : 200  watts

System Power AlarmFlag   : 0x00000000

 

Slot  Board Type  Watts        Priority 

------------------------------------------------------------

0     RSE-X3      0(0)         0(0)     

1     RSE-X3      0(0)         0(0)     

2     NA          0(0)         0(0)     

3     FIP-300     150(0)*      5(-1)    

c.     If the available power is sufficient, display detailed LPU status information.

[System-probe]display hardware internal sysm fip

Slot No.  State   Errcode   Flags   HwFlags

--------------------------------------------------------------

2         Off     0         0x0     0x0

3         enable  0         0x3     0x43

 

Flags :

    bit0-PowerOn bit1-Present

    bit2-ManuOn  bit3-ManuOff

    bit4-AutoOff bit5-Inserting

bit6-Enable

-     If the State field displays enable, the LPU might have hardware problems. Replace the LPU.

-     If the State field displays Fault, the LPU might have a BOM code error or the device does not support the LPU. Update the BOM code or replace the LPU.

-     If the State field displays Disable, the LPU might have been unmounted. Use the undo remove slot command to mount the LPU.

3.     If the LPU has been powered on, the basic BootWare segment might have been corrupted. Contact H3C Support.

4.     Reboot the device.

5.     Identify whether the Comware image files have been correctly loaded:

¡     If the following information is displayed repeatedly, the IPC channel is not available.

Press Ctrl+B to enter extended boot menu......

Failed. No response received from the active MPU.

 GDSYNC_SendRequest: Start

 GDSYNC_SendRequest: Start

 GDSYNC_SendRequest: Start

 GDSYNC_SendRequest: Start

 GDSYNC_SendRequest: Start

 GDSYNC_Start failed!

 SYNC failed.

Replace the LPU with an LPU that can operate correctly. Install the original module in another slot.

-     If neither of the LPU can start up correctly either, the MPU or LPU might have problems. Contact H3C Support.

-     If the new LPU can start up correctly but the original LPU cannot, the original LPU has problems.

-     If the new LPU cannot start up correctly, the IPC channel of the slot has problems. Contact H3C Support.

¡     If the following information is displayed repeatedly, the Comware image files are not correctly loaded.

%Jul 17 14:01:48:947 2014 H3C DEV/3/LOAD_FAILED: -MDC=1; Board in slot 3 failed to load software images.

%Jul 17 14:01:48:948 2014 H3C DEV/3/LOAD_FAILED: -MDC=1; Board in slot 3 failed to load software images.

Perform the following tasks:

-     Verify that the Comware software is an officially released version.

-     Identify whether the version supports the LPU.

6.     Identify whether the Comware image files have been correctly started. If any of the following symptoms appear, contact H3C Support:

¡     The system stops at the System image is starting... message.

¡     The system stops responding after displaying the System image is starting... message and some other information.

¡     The system reboots after displaying the System image is starting... message. This symptom occurs repeatedly.

¡     After The system displays the System image is starting... message and the Change to Normal message for the LPU, the LPU reboots repeatedly.

Related commands

This section lists the commands that you might use for troubleshooting card problems:

 

Command	Description
display device	Displays device information.

 

Troubleshooting transceiver modules

This section provides troubleshooting information for common transceiver module problems.

Interface down

Symptom

An interface has a transceiver module installed and an optical fiber is connected to the transceiver module correctly, but the physical or line protocol state of the interface is down.

Solution

To resolve the problem:

1.     Execute the display hardware internal module interface-type interface-number reg 6 command in probe view to verify that the transceiver module is present.

2.     Execute the display transceiver information interface command in probe view to verify that the transceiver modules at the two ends are the same type, and the transceiver module is compatible with the interface.

3.     Verify that the fiber type is compatible with the transceiver module.

4.     Execute the display hardware internal module interface-type interface-number reg 0 command to view the CPLD. Determine whether a transmit or receive failure or a hardware failure has occurred.

5.     If the problem persists, contact H3C Support.

Transceiver module type not recognized

Symptom

The system generates a message that reports unknown transceiver module type.

Solution

To resolve the problem:

1.     Execute the display hardware internal module interface-type interface-number reg 0 command in probe view to verify that the transceiver module is not a counterfeit and has an electronic label.

If the transceiver module is a counterfeit, the transceiver module type fails to be displayed correctly and the display transceiver manuinfo interface and display transceiver diagnosis interface commands are not available.

2.     Execute the display hardware internal transceiver register interface command in probe view to verify that the system can read the values on the registers in the transceiver module.

3.     If the problem persists, contact H3C Support.

Interface or transceiver module alarm

Symptom

An interface alarm is generated, or messages that report transceiver module insertion and removal are output repeatedly.

Solution

To resolve the problem:

1.     Verify that the CPLD version of the interface module is updated to the most recent version.

2.     Execute the display transceiver alarm interface command to determine whether an alarm was generated and whether the alarm is consistent with the output messages.

3.     If the problem persists, contact H3C Support.

Related commands

This section lists the commands that you might use for troubleshooting the transceiver modules.

 

Command	Description
display hardware internal module interface-name interface-number reg 6	Displays configuration and operating information about a transceiver module.
display hardware internal transceiver register interface interface-name interface-number device device-index address length length	Displays register values on a transceiver module.
display transceiver alarm interface interface-name interface-number	Displays alarm information for transceiver modules.
display transceiver diagnosis interface interface-name interface-number	Displays the current values of the digital diagnosis parameters on transceiver modules.
display transceiver information interface interface-name interface-number	Displays detailed transceiver information.
display transceiver interface interface-name interface-number	Displays the key parameters of transceiver modules.
display transceiver manuinfo interface-name interface-number	Displays electronic label information for transceiver modules.

 

Troubleshooting ports

Failure to ping the directly connected device

Symptom

An Ethernet interface cannot ping the directly connected device.

Solution

To resolve the problem:

1.     Execute the display interface command and perform the following tasks:

¡     Verify that the interface is up.

¡     Verify that the speed and duplex mode of the local end match the speed and duplex mode of the peer end.

¡     Verify that the cable and interface are operating correctly by viewing the packet statistics of the interface.

¡     Verify that the transceiver module of the local end matches the transceiver module of the peer end if the interface is a fiber port.

2.     Use the display arp all command to verify that the interface has learned the ARP entry of the directly connected interface. If not, use the debugging arp packet command to verify that the interface can send and receive ARP packets correctly.

3.     Use the debugging ip packet command to verify that the interface can send and receive IP packets correctly.

4.     Use the debugging ip icmp command to verify that the interface can send and receive ICMP packets correctly.

5.     If the problem persists, collect the following information for the interface:

¡     Execute the display hardware internal module interface-name interface-number statistics command in probe view to collect the interface statistics.

¡     Execute the display hardware internal module interface-name interface-number status command in probe view to collect the status information.

¡     Execute the display hardware internal module interface-name interface-number reg 1 command in probe view to collect the FPGA information.

¡     Execute the display hardware internal module interface-name interface-number reg 2 command in probe view to collect the MAC information.

¡     Execute the display hardware internal module interface-name interface-number reg 3 command in probe view to collect the PHY information.

¡     For interfaces on a FIP-300, FIP-310, or SAP-16EXP LPU, execute the display hardware internal nae slot slot-number freein command in probe view to collect the hardware buffer information.

¡     For interfaces on a FIP-240 or SAP-4EXP LPU, execute the display hardware internal dpaa slot slot-number bman pool-info command in probe view to collect the hardware buffer information.

6.     Contact H3C Support.

Failure to forward packets

Symptom

An Ethernet interface on an intermediate router cannot forward packets.

Solution

To resolve the problem:

1.     Verify that the Ethernet interface can ping the directly connected device. If the ping operation fails, see "Failure to ping the directly connected device."

2.     Use the debugging ip packet command to verify that the interface can transmit IP packets correctly.

3.     If the problem persists, collect the following information for the interface:

¡     Execute the display hardware internal module interface-name interface-number statistics command in probe view to collect the interface statistics.

¡     Execute the display hardware internal module interface-name interface-number status command in probe view to collect the status information.

¡     Execute the display hardware internal module interface-name interface-number reg 1 command in probe view to collect the FPGA information.

¡     Execute the display hardware internal module interface-name interface-number reg 2 command in probe view to collect the MAC information.

¡     Execute the display hardware internal module interface-name interface-number reg 3 command in probe view to collect the PHY information.

¡     For interfaces on a FIP-300, FIP-310, or SAP-16EXP LPU, execute the display hardware internal nae slot slot-number freein command in probe view to collect the hardware buffer information.

¡     For interfaces on a FIP-240 or SAP-4EXP LPU, execute the display hardware internal dpaa slot slot-number bman pool-info command in probe view to collect the hardware buffer information.

4.     Contact H3C Support.

Packet dropping

Symptom

An Ethernet interface drops packets.

Solution

To resolve the problem:

1.     Verify that the two ends are always in up state. Use the reset counter interface command to clear the traffic statistics for the interface.

2.     Use the display interface command to view the traffic statistics for the interface to determine whether the inbound and outbound traffic statistics have increased. Check the inbound and outbound traffic statistics for the peer interface. Determine whether the input and output errors for the interface have increased constantly.

3.     If the problem persists, collect the following information for the interface:

¡     For interfaces on a FIP-300, FIP-310, or SAP-16EXP LPU, execute the display hardware internal poe slot slot-number statistics command in probe view to collect the interface statistics.

¡     For interfaces on a HIM of a FIP-240 LPU, execute the display hardware internal himadp slot slot-number cnt command in probe view to collect the interface statistics.

¡     Execute the display hardware internal module interface-name interface-number statistics command in probe view to collect the interface statistics.

¡     Execute the display hardware internal module interface-name interface-number status command in probe view to collect the status information.

¡     Execute the display hardware internal module interface-name interface-number reg 1 command in probe view to collect the FPGA information.

¡     Execute the display hardware internal module interface-name interface-number reg 2 command in probe view to collect the MAC information.

¡     Execute the display hardware internal module interface-name interface-number reg 3 command in probe view to collect the PHY information.

¡     Execute the display hardware internal ibd pkt-info slot slot-number slot-number command in probe view to collect inter-card packet statistics.

4.     Contact H3C Support.

Related commands

This section lists the commands that you might use for troubleshooting ports.

 

Command	Description
debugging arp packet	Enables ARP debugging.
debugging ip icmp	Enables ICMP debugging.
debugging ip packet	Enables IP debugging.
display arp all	Displays all ARP entries.
display counters rate inbound interface	Displays inbound traffic rate statistics.
display counters rate outbound interface	Displays outbound traffic rate statistics.
display hardware internal dpaa slot slot-number bman pool-info	Displays hardware buffer information for interfaces on a FIP-240 or SAP-4EXP LPU.
display hardware internal himadp slot slot-number cnt	Displays statistics for interfaces on a HIM of a FIP-240 LPU.
display hardware internal ibd pkt-info slot slot-number slot-number	Displays packet statistics transmitted between interface modules.
display hardware internal module interface-name interface-number statistics	Displays Ethernet interface statistics.
display hardware internal module interface-name interface-number status	Displays status information for an Ethernet interface.
display hardware internal module interface-name interface-number message	Displays configuration information for an Ethernet interface.
display hardware internal module interface-name interface-number reg 1	Displays FPGA information for an Ethernet interface.
display hardware internal module interface-name interface-number reg 2	Displays MAC information for an Ethernet interface.
display hardware internal module interface-name interface-number reg 3	Displays PHY information for an Ethernet interface.
display hardware internal nae slot slot-number freein	Displays hardware buffer information for interfaces on a FIP-300, FIP-310, or SAP-16EXP LPU.
display hardware internal poe slot slot-number statistics	Displays statistics for interfaces on a FIP-300, FIP-310, or SAP-16EXP LPU.
display interface	Displays Ethernet interface information.

 

Troubleshooting active/standby MPU switchover

This section provides troubleshooting information for common active/standby MPU switchover problems.

Exceptional standby MPU reboot

Symptom

The standby MPU also reboots when the reboot command is used to reboot the active MPU.

Solution

To resolve the problem:

1.     After the original active MPU starts up, use FTP or TFTP to transfer the up-to-date log files in the logfile directory on the MPU to the file server.

2.     Search the log file for the reboot log message, for example, Command is reboot slot 0.

3.     Search the log file for the most recent system restart log message, for example, SYSLOG_RESTART: System restarted.

4.     Search the log messages between the two log messages for a log message like Batch backup of standby board in slot 1 has finished.

¡     If no log message like the specified message is found, the original standby MPU was starting up when you executed the reboot command. No action is required. Next time you want to use the reboot slot command to reboot the active MPU, make sure the standby MPU has started up.

¡     If a log message like the specified message is found, contact H3C Support.

Exceptional active/standby MPU switchover

Symptom

While the device is operating correctly, the active MPU reboots and the standby MPU becomes the new active MPU.

Solution

To resolve the problem:

1.     In probe view, use the display hardware internal util slot slot-num rbinfo 5 detail command to display reboot records.

2.     If the most recent record is slave-rob, the standby MPU became the new active MPU because it did not receive packets from the original active MPU. Wait for the original active MPU to start up and then contact H3C Support.

3.     If no slave-rob record is found, an exception occurred on the original active MPU. Execute the following commands to collect diagnostic information and then contact H3C Support:

¡     display hardware internal util slot slot-num rbinfo 5 detail

¡     display kernel exception 2 verbose slot slot-num

Related commands

This section lists the commands that you might use for troubleshooting active/standby MPU switchover.

 

Command	Description
display kernel exception number slot slot-num	Displays exception information.
display hardware internal util slot slot-num rbinfo	Displays reboot information.
display hardware internal mss slot slot-num information	Displays driver switchover information.
set hardware internal mss slot slot-num heart-beat rob { disable | enable }	Enables or disables the standby MPU to become the active MPU forcibly.

 

Troubleshooting interface modules

This section provides troubleshooting information for common interface module problems.

Interface information not displayed

Symptom

An interface module is installed on the device, but no information about its interfaces is displayed in the display interface brief command output.

Solution

To resolve the problem:

1.     Execute the display device verbose command to identify the interface module type and state.

2.     See the interface module manual to determine whether the LPU supports the interface module.

3.     If the problem persists, execute the display hardware internal util slot slot-num nvlog start num command to collect log messages and contact H3C Support.

Interface module information not displayed

Symptom

An interface module is installed on the device, but no information about the interface module is displayed in the display device verbose command output.

Solution

To resolve the problem:

1.     See the interface module manual to determine whether the LPU supports the interface module.

2.     Execute the display hardware internal pci device slot slot-num command in probe view.

For example, if the interface module is installed in slot 5, execute the command as follows:

[System-probe]display hardware internal pci device slot 5

… … … …

<02:05.00> pex8624 unit 1 port 5

<09:00.00> tsi384 linking subslot 2

<10:00.00> pci device in subslot 2

<02:06.00> pex8624 unit 1 port 6

<14:00.00> fpga for subslot 2

… … … … 

¡     If the command output does not contain the pci device in subslot x message, contact H3C Support.

¡     If the command output contains the message, go to step 3.

3.     Locate the pci device in subslot x message in the command output and copy the text in the square brackets (<>) on the left of the message. Then execute the display hardware internal pci config x slot y command, where x represents the copied text and y represents the interface module slot number.

Use the command output in step 2 as an example. The copied text is 10:00.00.

[System-probe]display hardware internal pci config 10:00.00 slot 5

<04:00.00> configuration space:

0000: dc 18 01 48 06 00 a0 02 12 00 80 02 00 ff 00 00

0010: 08 00 00 ec 08 00 00 ef 08 00 00 ed 00 00 00 ee

0020: 00 00 00 00 00 00 00 00 00 00 00 00 dc 18 01 48

… … … …   

If the first line in the command output is 0000: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff, the PCI bus of the interface module is faulty. Replace the interface module. If the first line is not 0000: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff, contact H3C Support.

4.     If the problem persists, contact H3C Support.

Related commands

This section lists the commands that you might use for troubleshooting the interface modules.

 

Command	Description
display hardware internal pci config bus:dev.func slot slot-num	Displays information about the registers on the PCI nodes.
display hardware internal pci device slot slot-num	Displays information about the PCI nodes.
display hardware internal util slot slot-num nvlog start num	Displays log messages in the nvlog.