SSH Troubleshooting Manual

Chapter 1  SSH Troubleshooting

&  Note:

This document is not restricted to a specific software/hardware version.

1.1  Password Authentication Failure (H3C Device Is Server)

1.1.1  Symptoms

The H3C device acts as the SSH server, and the SSH client is configured correctly, but the user of the client cannot pass password authentication to log in to the server.

1.1.2  Troubleshooting Flow

Figure 1-1 User fails password authentication when H3C device is the server

1.1.3  Troubleshooting Procedures

I. Check that the network connection is OK

From the client, ping the server’s IP address to check that the network between the client and server is available.

II. Check that SSH server is enabled on the H3C device

If the SSH server function is not enabled on the H3C device, the client cannot log in to the server. Use the following command to check whether SSH server is enabled on the H3C device:

<Server> display ssh server status

SSH server: Disable

SSH version : 1.99

SSH authentication-timeout : 60 second(s)

SSH server key generating interval : 0 hour(s)

SSH authentication retries : 3 time(s)

SFTP server: Disable

SFTP server Idle-Timeout: 10 minute(s)

If the SSH server field has a value of Disable, use the ssh server enable command on the H3C device to enable SSH server.

If the client wants to use SFTP to log in, also check that SFTP service is enabled on the H3C device. You can use the sftp server enable command on the device to enable SFTP service.

III. Check that the versions match

If the server and client cannot negotiate an SSH version successfully, the expected connection cannot be established. You can use the display ssh server status command to check the SSH version of the server.

<Server> display ssh server status

SSH server: Disable

SSH version : 1.99

SSH authentication-timeout : 60 second(s)

SSH server key generating interval : 0 hour(s)

SSH authentication retries : 3 time(s)

SFTP server: Disable

SFTP server Idle-Timeout: 10 minute(s)

If the SSH server is using SSH2.0 but the client is using SSH1.5 or lower, the version negotiation will fail.

If the client must use SSH1.5, you can configure the ssh server compatible-ssh1x enable command on the server to make the server compatible with SSH1.5. In this case, the SSH version is displayed as 1.99 on the server. If the client can use SSH2.0, configure the client to use SSH2.0 to enjoy higher security.

IV. Check that the VTY user interfaces are configured correctly

After the SSH server function is enabled on the device, if the client tries to log in to the server but is refused without any prompt about whether the client has authenticated the server, check whether there is any debugging information on the server. If not, the VTY user interfaces may not be configured correctly:

1)         For an SSH user to log in, there must be VTY user interfaces using the authentication mode of scheme.

Use the following command to check the VTY user interface configuration:

[Server-ui-vty0-4] display this

#

user-interface con 0

user-interface vty 0 4

#

return

By default, the authentication mode of a VTY user interface is not scheme, and you need to configure the following command in VTY user interface view.

[Server-ui-vty0-4] authentication-mode scheme

2)         For an SSH user to log in, there must be VTY user interfaces supporting SSH.

Use the following command to check the VTY user interface configuration:

[Server-ui-vty0-4] display this

#

user-interface con 0

user-interface vty 0 4

 authentication-mode scheme

 protocol inbound telnet

#

return

The above configuration information shows that the VTY user interfaces supports only Telnet. Configure the following command to solve this problem:

[Server-ui-vty0-4] protocol inbound all

By default, VTY user interfaces support all protocols.

V. Check that the client is configured with the server’s public key

To prevent server spoofing, when an SSH client tries to log in to an SSH server, the client authenticates the identity of the server by its digital signature. If the client is not configured with the public key of the server or the configured one is not correct, the server identity authentication will fail and the client will not be able to log in to the server. Therefore, you need to generate a key pair on the server and configure the public key of the server on the client correctly.

If the user of the client sees no server identity authentication information after entering the username, the reason may be that no key pair has been generated on the server.

You can use the following command to check whether the server has a public key:

<Server> display public-key local dsa public

Or:

<Server> display public-key local rsa public

If no DSA or RSA public key is displayed, the server does not have any public key. You can solve the problem in this way:

1)         Execute the public-key local create command on the server to create a key pair.

2)          Execute the public-key local export command to export the public key to a file.

3)         Upload the file to the client through FTP or TFTP and configure the client to use the public key file for authentication of the server.

VI. Check that the user is configured on the server

If the user of a client is repeatedly prompted to enter the password during login and the user enters the password as prompted but always fails the authentication, the reason may be that the user has not been configured on the server.

To prevent attackers from guessing usernames, the SSH server never gives prompts for non-existent usernames. You can use the display local-user command on the server to check whether there is such a local user configured or, if a remote AAA server is used for user authentication, check whether there is such a user configured on the remote authentication server.

If no such user is configured on the remote authentication server, configure an account for the user on the server.

VII. Check that the user passes the authentication

If a user is repeatedly prompted to enter the password during login and the user enters the password as prompted but always fails the authentication, the reason may also be:

1)         The password is not correct.

A user must input the same username and password that are configured on the server to log in. If the user forgets the password, you can send the correct password to the user, or configure a new password for the user on the server and notify the user of the new password.

2)         The user has been blacklisted by the server.

On some devices, after the password management feature is enabled, if a user fails authentication consecutively, the IP address and username will be added to a blacklist and all subsequent login requests will be denied.

You can use the display password-control blacklist command on the server to view the blacklist, checking whether the user is on the list. If yes, the user cannot log in to the server.

You can use the reset password-control blacklist user-name command to remove the user from the blacklist.

3)         The user has not been authorized to use SSH service.

For an SSH user to log in to the server, you need to authorize the user to use SSH service.

You can use the display local-user command to check whether the user has been authorized to use SSH service. For remote AAA authentication, check the configuration on the remote authentication server.

<Server> display local-user user-name test

The contents of local user test:

 State:                    Active

 ServiceType:              ssh

 Access-limit:             Disable           Current AccessNum: 0

 User-group:               system

 Bind attributes:

 Authorization attributes:

Total 1 local user(s) matched.

If the user is not permitted to use SSH service, use the service-type ssh command in local user view on the SSH server to authorize the user to use SSH service, or authorize the user to use SSH service on the remote server.

VIII. Check that the correct SSH service type is configured for the user

An SSH user can use Stelnet service, SFTP service, or both. If the service that an SSH user uses at login is not that configured for the user on the server, the user cannot log in.

You can use the display ssh user-information command on the server to view the configurations of the SSH user:

<Server> display ssh user-information

 Total ssh users:1

Username    Authentication-type   User-public-key-name     Service-type

test        password              null                     sftp

If the service that the user needs to use is not configured on the server, use the ssh user username service-type command to change the service type configuration for the user.

IX. Check whether the user has been timed out

If the following debugging information appears on the server, the user has been timed out because the user has not logged in within the specified period:

*Jan 24 14:40:12:100 2008 Server SSH/7/Server_ERROR: VTY[0]:Login Error:

SSH user user failed to login because of authentication timeout.

Such a problem is usually caused by a too short timeout time setting.

You can use the display ssh server status command to view the timeout time configured on the server:

<Server> display ssh server status

 SSH server: Enable

 SSH version : 2.0

 SSH authentication-timeout : 20 second(s)

 SSH server key generating interval : 0 hour(s)

 SSH authentication retries : 3 time(s)

 SFTP server: Disable

 SFTP server Idle-Timeout: 10 minute(s)

The key exchange and calculation process during SSH login may take a relatively longer period of time. If the performance of the device is not high enough, you need to give the authentication timeout time a greater value.

Use the ssh server authentication-timeout to change the setting of the authentication timeout time.

X. Check that the SFTP working directory is configured correctly

For an SSH user using SFTP, if the SFTP working directory configured for the user on the server is incorrect, the user cannot log in.

In this case, the server will display such debugging information:

Jan 30 17:12:34:891 2008 Server SSH/7/Server_MESSAGE: VTY[0]:SSH_Channel:

  Send Message SSH2_MSG_CHANNEL_FAILURE(100):RemoteID=1

*Jan 30 17:12:34:891 2008 Server SSH/7/Server_ERROR: SFTPS Error:

  Invalid sftp service or maximum sftp sessions exceeded.

For an SFTP user using only password authentication, the working directory that the user can use is the one authorized for the user. For a local authentication user, use the following method to check whether the working directory configured for the user is correct:

[Server-luser-test] display this

#

local-user test

 password simple 123

 authorization-attribute work-directory cf:/

 service-type ssh

#

return

If the working directory configuration is incorrect, use the authorization-attribute work-directory command in local user view to configure the correct working directory for the local user.

For a remote authentication user, configure the correct working directory for the user on the remote authentication server.

1.2  Public Key Authentication Failure (H3C Device Is Server)

1.2.1  Symptoms

The H3C device acts as the SSH server, and the SSH client is configured correctly, but the user cannot pass public key authentication to log in to the server.

1.2.2  Troubleshooting Flow

Figure 1-2 User fails public key authentication when H3C device is the server

1.2.3  Troubleshooting Procedures

The troubleshooting procedures for public key authentication failure when the H3C device acts as the server are similar to those for password authentication failure when the H3C device acts as the server. The differences mainly lie in two procedures:

l           Check that the user passes the authentication: Public key authentication uses the RSA or DSA algorithm and authenticates users by their digital signatures, while password authentication authenticates users by comparing passwords. Therefore, this troubleshooting procedure varies by the authentication method used.

l           Check that the SFTP working directory is configured correctly: Public key authentication users and password authentication users use different working directories. The working directory of a password authentication user is authorized by the AAA authorization server, while that of a public key authentication user is configured by using the ssh user command. Therefore, this troubleshooting procedure varies by the authentication method used.

The following presents only the two different procedures. For other procedures, refer to Troubleshooting Procedures.

I. Check that the user passes the public key authentication

If a user fails the public key authentication when logging in to the SSH server, such debugging information will be displayed on the server:

*Jan 30 10:12:28:703 2008 Server SSH/7/Server_ERROR: VTY[0]:Process AuthPK Error:  Failed to vertify public key!

*Jan 30 10:12:28:703 2008 Server SSH/7/Server_EVENT: VTY[0]:LOGIN Failed:

SSH user user failed to login from 192.168.1.2(0000-5e28-b700) on VTY0.

*Jan 30 10:22:28:875 2008 Server SSH/7/Server_EVENT: VTY[0]:SSH user[user] have reached the max permit retry times[3].

Possible reasons include:

1)         The server is not configured with the correct public key of the user.

Use the display ssh user-information command on the server to check the name of the public key for the user:

<Server> display ssh user-information

 Total ssh users:1

 Username            Authentication-type  User-public-key-name  Service-type

 test1               publickey            user_key3             stelnet|sftp

Use the display public-key peer command to view the details about the public key of the user:

<Server> display public-key peer name user_key3

=====================================

  Key Name  : user_key3

  Key Type  : RSA

  Key Module: 1024

=====================================

Key Code:

30819F300D06092A864886F70D010101050003818D0030818902818100E31EE84BB745DA6F
9ACCF5B6CA7BC4C0599A45F65281BE869AD90A192D37BBCE5683F7A68F0C95A0399A3022C7
F66EC52DE951630874CD8EE11AFA876ABE3937E86BF9A83EFC0C8E68144FE38DFAEDC12E48
654F8D15703455C96E9BCE026A91A2435E675BB7420D5A4AB5CE2520A213405AFED986F8F5
9101B745D9BF9BBA4D0203010001

Display the public key of the user on the client and compare it with that saved on the server. If the two do not match, obtain and configure the public key of the user on the server again. To do so, use FTP or TFTP to upload the public key file of the user in binary to the server at first. Then, on the server, use the public-key peer import sshkey command to import the public key from the public key file and then use the ssh user command to associate the public key with the user.

 

&  Note:

The way to display a user’s public key varies by SSH client.

 

2)         The public key configured on the server and the private key used by the user do not match:

An SSH client may support multiple public key algorithms, each of which corresponds to a different key pair. A user can pass authentication only when the public key configured on the server matches the private key used by the user for login. For example, if you configure the DSA public key of a user on the server, but the user uses an RSA key private key, instead of the DSA private key, to log in to the server, the user will fail the authentication.

To correct such a problem, make sure that the public key configured on the server matches the private key that the user will use for login.

 

&  Note:

The way for specifying the private key to be used by a user varies by SSH client. Some client software provides some command keywords, while some allows you to specify the private key file.

 

II. Check that the SFTP working directory is configured correctly

For an SSH user using SFTP, if the SFTP working directory configured for the user on the server is incorrect, the user cannot log in.

In this case, the server will display such debugging information:

Jan 30 17:12:34:891 2008 Server SSH/7/Server_MESSAGE: VTY[0]:SSH_Channel:

  Send Message SSH2_MSG_CHANNEL_FAILURE(100):RemoteID=1

*Jan 30 17:12:34:891 2008 Server SSH/7/Server_ERROR: SFTPS Error:

  Invalid sftp service or maximum sftp sessions exceeded.

The working directory of a public key authentication user is that specified by the work-directory keyword in the ssh user command. Use the following method to check whether the working directory is configured correctly:

<Server> display current-configuration | include ssh user

 ssh user test service-type sftp authentication-type publickey assign publickey test work-directory cf:/

If the working directory configuration is incorrect, use the ssh user command to specify the correct working directory.

1.3  Password Authentication Failure (H3C Device Is Client)

1.3.1  Symptoms

The H3C device acts as the SSH client, and the SSH server is configured correctly, but the user cannot pass password authentication to log in to the server.

1.3.2  Troubleshooting Flow

Figure 1-3 User fails password authentication when H3C device is the client

1.3.3  Troubleshooting Procedures

I. Check that the network connection is OK

From the client, ping the server’s IP address to check that the network between the client and server is available.

II. Check that the versions match

If the server and client cannot negotiate an SSH version successfully, the expected connection cannot be established.

When acting as the client, the H3C device supports SSH2.0. If the server is using SSH1.5 or lower, the version negotiation will fail. In this case, you need to configure the server to support SSH2.0.

III. Check that the client is configured with the correct public key of the server

To prevent server spoofing, when an SSH client tries to log in to an SSH server, the client authenticates the identity of the server by its digital signature. If the client is not configured with the public key of the server or the configured one is not correct, the server identity authentication will fail and the client will not be able to log in to the server.

When acting as the client, the H3C device supports the first-time authentication feature. That is, when the device is not configured with the server’s public key, a user trying to use the client to log in to the server for the first time can choose to continue to access the server and save the server’s public key locally. When accessing the server again, the client will use the locally saved server public key to authenticate the server. If you configure the client to support the first-time authentication feature, you do not need to configure the server’s public key on the client in advance. Otherwise, you need to configure the server’s public key on the client correctly in advance. By default, the client supports first-time authentication.

1)         The client does not support first-time authentication and is not configured with the server’s public key in advance.

In this case, when trying to log in to the server, the client will be logged out immediately after the user enters the username:

<Client> ssh2 192.168.1.1

Username: test

Trying 192.168.1.1 ...

Press CTRL+K to abort

Connected to 192.168.1.1 ...

Connection closed.

If such information appears on a client, it may be because that the client does not support first-time authentication and is not configured with the server’s public key in advance.

You can use the display current-configuration command to view whether the undo ssh client first-time command is configured. If yes, the client does not support first-time authentication.

Use either of the following methods to solve the problem:

l           Upload the public key file of the server in binary to the client through FTP or TFTP at first. Then, on the client, use the public-key peer import sshkey command to import the public key from the public key file and use the ssh client authentication server command to associate the public key with the server.

l           Configure the ssh client first-time enable command on the client to make the client support first-time authentication.

 

  Caution:

With the ssh client first-time enable command configured, a client cannot ensure the security of the connection.

 

2)         The server side public key that the client maintains is not that of the server.

In this case, when the client tries to log in to the server, the following information will be displayed on the client:

<Client> ssh2 192.168.40.182

Username: test

Trying 192.168.40.182 ...

Press CTRL+K to abort

Connected to 192.168.40.182 ...

 

The server's host key does not match the local cached key. Either the server administrator has changed the host key, or you connected to another server pretending to be this server. Please remove the local cached key, before logging in!

Connection closed.

If such information appears on a client, it may be because the server side public key that the client maintains is not that of the server. Use the following method to check whether the server side public key that the client maintains is exactly that of the server:

First, view the name of the server side public key that the client maintains.

<Client> display ssh server-info

Server Name(IP)                                   Server public key name

_________________________________________________________________________

192.168.40.182                                    server-key-name

Then, view the information of the server side public key on the client.

<Client> display public-key peer name server-key-name

Finally, compare the content of the server side public key maintained on the client with that of the server’s public key.

 

&  Note:

l      The way to display a user’s public key varies by SSH server.

l      If both the SSH client and SSH server are H3C devices, the client uses DSA to authenticate the identity of the server and you need to configure the SSH server’s DSA public key on the client.

 

To solve this problem, execute the following command on the client to cancel the incorrect public key-server association at first:

[Client] undo ssh client authentication server 192.168.40.182 assign publickey

Then, use either of these two methods to configure the public key of the server on the client:

l           Upload the public key file of the server in binary to the client through FTP or TFTP at first. Then, on the client, use the public-key peer import sshkey command to import the public key from the public key file and use the ssh client authentication server command to associate the public key with the server.

l           Configure the ssh client first-time enable command on the client to make the client support first-time authentication, so that the client will obtain the server’s public key and establish the public key-server association automatically when logging in to the server next time.

IV. Check that the username and password are correct

If a user is repeatedly prompted to enter the password during login and the user enters the password as prompted but always fails the authentication, the reason may be that the provided username or password is incorrect. Ensure that the user obtains the correct username and password and uses the correct username and password to log in.

1.4  Public Key Authentication Failure (H3C Device Is Client)

1.4.1  Symptoms

The H3C device acts as the SSH client, and the SSH server is configured correctly, but the user cannot pass public key authentication to log in to the server.

1.4.2  Troubleshooting Flow

Figure 1-4 User fails public key authentication when H3C device is the client

1.4.3  Troubleshooting Procedures

The troubleshooting procedures for public key authentication failure when the H3C device acts as the client are similar to those for password authentication failure when the H3C device acts as the client. The only difference is that public key authentication uses the RSA or DSA algorithm and authenticates users by their digital signatures, while password authentication authenticates users by comparing passwords. Therefore, the last troubleshooting procedure for public key authentication failure is different from that for password authentication failure. The following describes only the different procedure. For information about other procedures, refer to Troubleshooting Procedures.

I. Check that the client is configured to use the correct public key algorithm

When acting as the SSH client, the H3C device supports two public key algorithms: RSA and DSA. For a user to log in to the server successfully, the public key algorithm used by the client, identified by identity-key keyword in the ssh2 or sftp command, must match the type of the client’s public key that is configured on the server. For example, if you configure the DSA public key of a user on the server, but the user uses an RSA key private key, instead of the DSA private key, to log in to the server, the user will fail the authentication.

To solve this problem, when using the ssh2 or sftp command on a client to log in to the server, a user needs to use the identity-key keyword to specify the same public key algorithm that is configured on the server. By default, when acting as the SSH client, the H3C device uses the DSA algorithm.

1.5  Troubleshooting Commands

1.5.1  SSH Server Commands

Command

Description

display local-user

Display information about local users.

display password-control blacklist

Display information about blacklisted users after a user fails the authentication.

display public-key local

Display the public key information of the local key pair.

display public-key peer

Display information about the locally saved public keys of peers.

display ssh server status

On an SSH server, display the SSH server status information.

display ssh user-information

On an SSH server, display the SSH server session information.

debugging ssh server

Enable SSH server debugging.

 

1.5.2  SSH Client Commands

Command

Description

display public-key local

Display the public key information of the local key pair.

display public-key peer

Display information about the locally saved public keys of peers.

display ssh server-info

On a client, display the mappings between SSH servers and their host public keys saved on the client.

debugging ssh client

Enable SSH client debugging.

Copyright ©2009 Hangzhou H3C Technologies Co., Ltd. All rights reserved.

No part of this manual may be reproduced or transmitted in any form or by any means without prior written consent of Hangzhou H3C Technologies Co., Ltd.

The information in this document is subject to change without notice.