There are no hard and fast rules on how to deal with a server that’s under attack, but this article should give you some tips to get control of the situation.
When we say “attack”, this could:
- A rogue server downloading or doing requests to your server
- A true denial of service attack
- Some script kiddy loop that’s trying to hit your server for whatever reason
On point 3, perhaps someone is trying to break in, or repeatedly submit a form, guess a username or password, or whatever.
You will need to following to control this attack:
- SSH access
- Access to the server log files
- Some understanding of TCP/IP and networking
If you are suffering from a true denial of service attack, eg. someone has intentionally set many computers to attach your network, you might also need access to your core router. We’ll discuss that right at the end because there attacks although totally overwhelming, are also rare.
Step 1 – Determine where the attacks are coming from.
To do this, you have to find the originating IP of the attacks. The command to do this is netstat which has an overwhelming amount of options, but here are a few ways to see what’s happening:
netstat -anp |grep 'tcp\|udp' | awk '{print $5}' | cut -d: -f1 | sort | uniq -c | sort -n
netstat -anp |grep ':443' | awk '{print $5}' | cut -d: -f1 | sort | uniq -c | sort -nr | head
Both these commands check who is attaching to your server and provides value sorted information on who are the top talkers. Use a pen and paper or notepad to keep track of which I has the most connections open.
Step 2 – Block the IPs
Blocking the IPs depends on your operating system, e.g. if you’re using Ubuntu or CentOS you can use firewalld or iptables commands to block IPs.
firewalld example:
# firewall-cmd --add-rich-rule='rule family=ipv4 source address=10.x.x.x reject' --permanent # sudo firewall-cmd --reload
What to do if the server is not accessible due to too many connections?
If the server is too busy due to too many connections, you need to stop the attacker at router level. In this scenario you have to work with your provider, but if you are the provider, then we recommend a MikroTik router in bridge mode and torch.
How to see the timeout of MySQL
The references below provide additional information on how to increase timeouts for FCGI and MySQL, but be careful, just endless increasing these values might not be mitigating the origin of the problem. Make small incremental changes and clearly document these
How to see wait_timeout in my.cnf:
MariaDB [(none)]> SHOW SESSION VARIABLES LIKE "%timeout";
See Also
List of Useful Netstat Commands
References
- https://stackoverflow.com/questions/33811543/php-and-mod-fcgid-ap-pass-brigade-failed-in-handle-request-ipc-function
- https://support.plesk.com/hc/en-us/articles/213399589
- https://httpd.apache.org/mod_fcgid/mod/mod_fcgid.html#fcgidmaxrequestsperprocess
- https://talk.plesk.com/threads/fastcgi-config.332018/
- https://stackoverflow.com/questions/28111879/gearman-zf2-doctrine2-mysql-sqlstatehy000-general-error-2006-mysql-serve
- https://www.techrepublic.com/article/how-to-quickly-check-to-see-if-your-linux-server-is-under-a-ddos-attack-from-a-single-ip-address/