Recently, we experienced a mild denial of service attack against several of our hosted servers. The attack primarily targeted SSH on one of our subnets. The origin of the attack was a single IP in Hong Kong.
We have had an SSH attack service check configured on our routers for quite some time, but the nature of this attack skirted what that check was designed to catch. As a result, our on call staff received several SSH alerts from various systems that we monitor. The service check output was simply “Server answer:”. At first blush it seemed like this was a problem with one of our virtual host servers, but we quickly ruled that out as alerts came in from physical machines as well.
Staff members at this point tried to identify a common point of contact between this diverse set of hosts. We were seeing alerts on physical servers, virtual machines on different hosts, Ubuntu servers, CentOS servers, 32 and 64 bit servers, etc. There seemed to be no commonality at all. However, as soon as we realized that this was an SSH attack, by checking /var/log/secure on an affected server and recognizing the traffic on the router, the commonality between the affected servers became clear: SSH was running on port 22 for all of these hosts.
The time it takes for a malicious script to scan thousands of ports and then carry out an attack on whichever ones respond turns out to be too high a barrier to be widely used. It is much easier for an attacker to simply hit port 22 directly on a wide number of hosts. Since so many people don't know or don't take the time to move SSH off of port 22, the attack is usually quite successful.
In this case, the attacker was simply hitting port 22 as hard as possible on any IP that would respond in this particular subnet. The solution was pretty simple on our end, null route the IP address on the router. The unfortunate part of the attack is that there is no reason why this attack should have had any affect.
There are at least 2 really easy steps that you can take to protect your server from ever hiccuping as a result of an SSH denial of service attack.
First, move SSH to a non-standard port. If you change SSH to start listening on port 10978, for instance, the number of malicious IPs that get an SSH login prompt will simply flatline. You can make use of ~.ssh/config or PuTTY's configuration to specify the random port that you have chosen and then it requires no more hassle on your end to connect.
Second, install denyhosts. Using this package is a fabulous method of stopping denial of service (or distrubuted denial of service) attacks right away. Denyhosts works by monitoring the number of failed login attempts from remote IPs and bans the IPs if they exceed the threshold. Different thresholds are configurable for the root user and non-privileged users. In addition, denyhosts maintains a list of blacklisted IPs that you can tell the software to check, which would prevent a known attacker from even trying to connect the first time.
If you use denyhosts in combination with SSH on a non-standard port then the odds of your server being compromised by an SSH related attack are very small indeed. Even if you keep SSH on port 22, but install denyhosts, this package will prevent you from being locked out of your system due to an SSH related attack. Especially if you do not use SSH public key authentication, make sure to add your IP address to /etc/hosts.allow, so that mistyping your password doesn't block your IP too! (Speaking of public key authentication, why aren't you using it!?)
The moral of the story: take some time and install denyhosts and move SSH to a random port.comments powered by Disqus