The Right Way to Secure SSH Access on Your Linux Server

Secure Shell (SSH) is the default way to access a Linux server remotely but it is in danger of brute-force attacks and misconfigurations. It allows us to connect to and from Linux devices, UNIX servers, network appliances and sometimes even Windows. If your server is exposed to the internet, securing SSH is not optional but it’s critical. In this step-by-step tutorial, you are going to learn how to secure SSH access like a pro.

 What You’ll Learn

1.         Backup the config file

2.         Set a banner message

3.         Prevent empty password

4.         How to switch from password to SSH keys

5.         How to disable root login

6.         How to change the default SSH port

7.         How to limit SSH access by user

8.         Time’s UP!

Prerequisites

•                     A Linux server (Red Hat, Ubuntu, Debian, etc.)

•                     A regular user account with sudo privileges

•                     A local machine (Linux, macOS, Windows with OpenSSH)

1.         Backup the config file

Best practice is always take back up the configuration file before making major changes. This is a common bit of advice, but it's a real one. It's easy, takes only a moment, and protects you in case of a mistake when editing the file and who hasn't made a mistake in Vim?

That’s it, now you save to work or required changes in the config file.

Suggestion: Always take a backup of configuration files before making critical changes.

2.         Set a banner message

This is a professional approach that every tech should follow, this setting only takes a moment. You can actually provide some pretty good information in banner messages. First, we'll write the banner message in the /etc/issue.net file by using Vim. Then we'll open the sshd_config file and tell it to use the content of issue.net as the banner.

Remove the information that is already written in /etc/issue.net and write this message.

Next, tell SSH to use the banner message. Open the sshd_config file in Vim and find the line that reads Banner. You do remember that you can use the forward-slash character in Vim's Command mode to keyword-search a file, right? For example, /banner

Find the line that reads # no default banner path and then uncomment the next line (it says Banner)

Save your changes in Vim with: wq and then restart the SSH service:

Any time you make a change to the configuration file, you must restart the service.

3.         Prevent empty passwords

Empty passwords are clearly a bad idea. You may have other utilities, such as Pluggable Authentication Modules (PAM), regulating your regular passwords, but it's also a good idea to make sure SSH enforces responsible security settings.

Open the /etc/ssh/sshd_config file in Vim, and then find the line that reads PermitEmptyPasswords. Uncomment it, and replace the yes value with no.

That’s it.

4.         How to switch from passwords to SSH keys

One of the most common security settings for SSH these days is key-based authentication. Through the years, this authentication method has become more and more common. Key-based authentication uses asymmetric cryptography. That means there are two keys. One is private and never sent across the network. The other is public and may be transferred across the network. Because the keys are related, they can be used to confirm identities—identities such as SSH authentication attempts.

You'll need to generate the key pair on the local SSH client computer and then transfer the public key across the network to the destination SSH server. In other words, the keys will identify you on your admin workstation. Once this configuration is in place, you are no longer challenged for a password when you establish an SSH connection. The process only requires a few steps.

First, generate the key pair:

The keys are stored in your home directory in a hidden directory named .ssh and the default key names are id_rsa (private key) and id_rsa.pub (public key).

Next, send the user1 public key across the network to the destination SSH server located at 192.168.1.58: Finally, test the connection:

Notice that you are not challenged for a password. Since you have now embraced key-based authentication, you can edit the sshd_config file to prevent any logins based on passwords. Once you configure this setting, only key-based authentication will be accepted.

Edit these two lines in the file:

5.   How to disable root login

Allowing direct root login is risky. Disable it by editing the SSH configuration file to make it secure.

Vim /etc/ssh/sshd_config

Save and restart the SSH service with the command systemctl restart sshd.

6.   How to change the default SSH port

Another common change is to configure SSH to listen on a different port than the standard 22/tcp that we've all memorized. There's already an entry in the sshd_config file.

You can comment out the default port setting and add another line, as I've done below:

You must remember to append the new non-standard port number to your SSH connection attempts from this point on

7.  How to limit SSH access by user

As you've already prevented the use of the root user account across SSH, why not go one step further and explicitly state which users can connect to the server? Actually, you have a regular non-root admin account you use or one that is already configured with sudo privileges.

Add the following line in the SSH configuration file (it's not in there by default):

By the way, you can actually filter with all of the following settings: AllowUsers, DenyUsers, AllowGroups and DenyGroups. You can discover more information on the man page for sshd_config. 

    

  8.   Time’s UP

The next option is to set time-out connections. The clientaliveinterval manages idle SSH connections. After sending a message to the client and server waits for a response. The ClientAliveInterval is the space of time between the messages. The ClientAliveCountMax defines how many times the server will do this before deciding that the client isn't really active anymore. At that point, the connection is dropped.

Here is an example of configuration that checks every 60 seconds and will do so three times:

Edit these values that make sense for your environment.

Frequently Asked Questions (FAQ)

1. How do I secure SSH access on Linux?

To secure SSH access, disable root login, use key-based authentication, change the default SSH port, and install fail2ban to prevent brute-force attacks.

2. Is it safe to change the default SSH port?

Yes, changing the SSH port adds an extra layer of security by reducing automated attack attempts. However, it should complement — not replace — other hardening techniques.

3. How do I disable root login in SSH?

Open the SSH configuration file `/etc/ssh/sshd_config`, set `PermitRootLogin no`, then restart the SSH service using `sudo systemctl restart ssh`.

4. Can I use password authentication and key authentication together?

It’s possible, but not recommended. Using both methods increases risk. The best practice is to disable password-based authentication entirely and use SSH keys.

5. How can I recover SSH access if I misconfigure sshd_config?

If you lose access due to a misconfiguration, use local or console access to restore your backup configuration file (`sshd_config.bak`) and restart the SSH service.

Best Practices for Securing SSH Access on Linux

·         Always disable root login and create a separate sudo user.

·         Use SSH key authentication instead of passwords.

·         Change the default port (22) to a non-standard port to reduce automated scans.

·         Limit user access to only trusted accounts using the `AllowUsers` directive.

·         Enable fail2ban or other intrusion detection tools to block repeated failed logins.

·         Restrict SSH access to specific IP addresses or networks with firewall rules.

·         Regularly update your SSH package and Linux system to patch vulnerabilities.

·         Set idle session timeouts in `/etc/ssh/sshd_config` using `ClientAliveInterval` and `ClientAliveCountMax`.

·         Monitor login attempts via `/var/log/auth.log` or use audit tools for security tracking.

·         Backup and verify your SSH configuration before making major changes.

Final thoughts:

I have listed several common but effective SSH configurations to help you better secure your environment. With security, no one setting is likely to protect your devices. The goal is layers of security, the combination of which helps to mitigate security threats. I strongly recommend that you organize your keys carefully if you implement key-based authentication. You have to consider using a centralized /etc/ssh/sshd_config file to maintain consistent security configurations on your SSH servers. Whenever you make changes in the configuration file, always restart the SSH service.

Continue to visit seeklinux for more information and updates.