Fix: SSHD Not Starting On Boot In Arch Linux

by Henrik Larsen 45 views

Introduction

Hey guys! Ever run into that super annoying problem where your SSH daemon (sshd) just refuses to start when you boot up your Arch Linux box? Yeah, it's a classic head-scratcher. You've installed OpenSSH, you can even manually fire up sshd without a hitch, but it's a no-show on boot. Frustrating, right? This guide is here to walk you through the common culprits and how to get your SSH server automatically starting like it should. We'll dive into systemd, configs, and all those little details that can trip you up. So, let's get this sorted!

Understanding the Problem: SSHD Not Starting on Boot

So, you've got OpenSSH installed, and you've probably tried the usual suspects – you can manually start the sshd service with a quick sudo /usr/bin/sshd, and it fires up just fine. You might even check its status using sudo systemctl status sshd.service, and everything looks green when you've started it manually. But the moment you reboot, poof! No SSH love. The core issue here is that systemd, Arch Linux's system and service manager, isn't automatically starting sshd during the boot process. This could be due to a bunch of reasons, which we'll explore, but it boils down to systemd not being told, or not being able, to start the service when your system comes online. We're going to get systemd and sshd to play nicely together, ensuring that your SSH server is ready and waiting every time you boot up. It's all about digging into the configuration and making sure the right hooks are in place. Think of it like making sure the car keys are in the ignition – we need to ensure everything is set for an automatic start.

Common Causes and Troubleshooting Steps

Okay, let's get our hands dirty and troubleshoot this thing. There are several reasons why your sshd service might be playing hide-and-seek on boot. We'll go through the most common causes and the steps you can take to diagnose and fix them. Think of this as a detective's checklist – we'll investigate each suspect until we find the culprit.

1. Systemd Service Not Enabled

The most frequent offender is a simple one: the sshd service isn't enabled to start on boot. Systemd uses "units" to manage services, and just installing a service doesn't automatically mean it'll start at boot. You need to explicitly enable it. To check if sshd is enabled, run:

systemctl is-enabled sshd.service

If the output is disabled, that's our prime suspect! To enable the service, use:

sudo systemctl enable sshd.service

This command creates symbolic links in the systemd configuration directories, telling systemd to start sshd during boot. Now, reboot and see if sshd behaves. If it does, awesome! If not, let's move on to the next suspect.

2. Firewall Interference

Another common issue is your firewall blocking SSH connections during the boot process. If your firewall rules aren't set up correctly, sshd might start, but connections will be refused, making it seem like the service isn't running. Arch Linux often uses iptables or firewalld. Let's focus on firewalld for this example.

First, check if firewalld is running:

sudo systemctl status firewalld

If it's active, you need to ensure SSH traffic is allowed. To allow SSH, use:

sudo firewall-cmd --permanent --add-service=ssh
sudo firewall-cmd --reload

The --permanent option makes the rule persistent across reboots, and --reload applies the changes immediately. If you're using iptables, you'll need to adjust your rules accordingly. Remember to save your iptables rules to make them persistent across reboots. After adjusting your firewall, reboot and test your SSH connection.

3. Configuration Errors in sshd_config

Sometimes, the issue isn't with systemd or the firewall, but with the SSH daemon's configuration itself. The sshd_config file, usually located at /etc/ssh/sshd_config, contains settings that dictate how sshd behaves. Typos or incorrect settings can prevent sshd from starting.

Open the sshd_config file with your favorite text editor (like nano or vim) and carefully review the settings. Pay close attention to these common culprits:

  • Port: Ensure the port is set correctly (the default is 22). If you've changed it, make sure it's not conflicting with another service.
  • ListenAddress: If this is set, make sure it's configured correctly for your network interface. If it's misconfigured, sshd might not listen on the correct interface.
  • PermitRootLogin: While generally discouraged for security reasons, if you're using root login for testing, ensure this is set to yes.
  • PubkeyAuthentication: If you're using public key authentication (which you should!), make sure this is set to yes.

After making changes, save the file and try restarting sshd:

sudo systemctl restart sshd.service

If there are syntax errors in the config file, sshd might fail to start, and systemd will log the errors. You can check the logs using journalctl -u sshd.service (we'll dive deeper into logs later).

4. Host Key Issues

SSHD uses host keys to identify the server to clients. If these keys are missing or corrupted, sshd might refuse to start. Typically, these keys are generated automatically when OpenSSH is installed, but sometimes things go wrong.

Check for the existence of host key files in /etc/ssh/: ssh_host_rsa_key, ssh_host_dsa_key, ssh_host_ecdsa_key, and ssh_host_ed25519_key. If any are missing, you can regenerate them using:

sudo ssh-keygen -A

This command regenerates all missing host keys. After running it, restart sshd and see if the problem is resolved.

5. Network Configuration Problems

In some cases, sshd might fail to start on boot due to network configuration issues. If your network isn't fully initialized when sshd tries to start, it might not be able to bind to the network interface. This is more common in systems with complex network configurations or those relying on DHCP.

One solution is to ensure that the network is fully up before sshd starts. You can achieve this by adding a dependency to the sshd systemd unit. Create a directory for sshd service overrides:

sudo mkdir -p /etc/systemd/system/sshd.service.d

Then, create a file named dependencies.conf in that directory:

sudo nano /etc/systemd/system/sshd.service.d/dependencies.conf

Add the following content to the file:

[Unit]
After=network-online.target
Wants=network-online.target

This tells systemd that sshd should start after the network-online.target is reached, indicating that the network is up. Save the file and then run:

sudo systemctl daemon-reload

This reloads the systemd configuration. Now, restart sshd or reboot to test the changes.

Diving Deeper: Analyzing Logs for Clues

When troubleshooting, logs are your best friends. Systemd logs all service activity, and these logs can provide invaluable clues about why sshd is failing to start. We've briefly touched on this, but let's get into more detail. The primary tool for accessing systemd logs is journalctl.

To view logs specifically for the sshd service, use:

journalctl -u sshd.service

This will show you the logs for the current boot. If you want to see logs from previous boots, you can add the -b option followed by a boot ID, or use -b -1 for the previous boot, -b -2 for the boot before that, and so on.

journalctl -u sshd.service -b -1

Pay close attention to any error messages or warnings. These messages often pinpoint the exact cause of the problem. For example, you might see errors related to:

  • Configuration file syntax: A typo in sshd_config will usually generate an error message.
  • Key permissions: Incorrect permissions on the host key files can prevent sshd from accessing them.
  • Address binding: If sshd can't bind to the specified IP address and port, it will log an error.
  • Missing dependencies: If sshd depends on other services that aren't running, the logs will reflect this.

Another useful option is -f, which follows the logs in real-time. This can be helpful if you're trying to start sshd manually and want to see the log output as it happens:

journalctl -u sshd.service -f

By carefully analyzing the logs, you can often narrow down the problem and find the right solution. It's like being a detective, but instead of fingerprints, you're looking for error messages.

Advanced Troubleshooting Techniques

If you've tried the common solutions and you're still wrestling with sshd, it's time to bring out the big guns. Let's explore some advanced troubleshooting techniques that can help you dig deeper and uncover the root cause of the problem.

1. Running sshd in Debug Mode

One powerful technique is to run sshd in debug mode. This allows you to see detailed output from the daemon as it starts, which can reveal configuration issues or other problems that aren't immediately obvious. To run sshd in debug mode, you can use the -d option:

sudo /usr/sbin/sshd -d

This will start sshd in the foreground and print debug messages to your terminal. You might need to open a new terminal to run this command, as it will tie up your current terminal. The -d option can be used multiple times (e.g., -dd, -ddd) to increase the level of debugging output.

Carefully examine the output for any errors or warnings. Pay attention to messages related to configuration file parsing, key loading, and network binding. Debug mode can often highlight the exact line in your sshd_config file that's causing problems.

2. Temporarily Disabling Security Features

Sometimes, security features like SELinux or AppArmor can interfere with sshd's startup. These security systems enforce access control policies, and if they're misconfigured, they can prevent sshd from functioning correctly.

As a temporary troubleshooting step, you can try disabling these features to see if they're the cause of the problem. Be aware that disabling security features can make your system more vulnerable, so only do this for testing purposes and re-enable them as soon as possible.

To check the status of SELinux, use:

sestatus

If it's enabled, you can temporarily disable it with:

sudo setenforce 0

This puts SELinux in permissive mode, which means it will log policy violations but won't enforce them. To re-enable SELinux, use sudo setenforce 1. For AppArmor, you can check its status and disable it using systemd:

sudo systemctl status apparmor
sudo systemctl stop apparmor

To re-enable AppArmor, use sudo systemctl start apparmor. After disabling these features, try restarting sshd and see if the issue is resolved. If it is, you'll need to investigate your SELinux or AppArmor configuration to determine the specific policy that's interfering with sshd.

3. Checking for Conflicting Services

In rare cases, another service might be conflicting with sshd, preventing it from starting. This could be another SSH server or a service that's trying to use the same port (usually port 22). To check for conflicting services, you can use the netstat or ss command.

sudo netstat -tulnp | grep 22

Or, using the ss command:

sudo ss -tulnp | grep 22

These commands will show you any processes that are listening on port 22. If you see another service listed, you'll need to either disable it or reconfigure it to use a different port. Be careful when disabling services, as it could affect other parts of your system. Make sure you understand the purpose of the service before disabling it.

Conclusion: SSHD Success!

Alright, guys, we've been through the trenches, battled systemd demons, and wrestled with configuration files. Hopefully, by now, your sshd service is purring like a kitten and starting automatically on boot. We've covered a lot of ground, from basic troubleshooting steps like enabling the service and checking firewall rules to more advanced techniques like analyzing logs and running sshd in debug mode.

The key takeaway here is that troubleshooting is a process of elimination. Start with the most common causes and work your way through the list. Don't be afraid to dive into logs and experiment with different solutions. And remember, Google is your friend! There's a wealth of information online about SSH and systemd, so don't hesitate to search for specific error messages or issues you're encountering.

With a little persistence and the techniques we've discussed, you should be able to conquer any SSHD startup issue on Arch Linux. Now go forth and SSH securely!