Table of Contents
Synopsis
At times cPanel servers will be overwhelmed and connection counts will bust the 100 limit. Taking it up to 150 or 200 as per their suggestion, or even 700 still shows problems. It became evident after many days and weeks that a harder firewall is required.
At first the author tried scripts and the default FirewallD
as per AlmaLinux but it was a mess. This roll you own
script is documented at the end. However, a custom script with protection doesn’t work so well because it will lead to extra maintenance and might not be supported by the upstream supplier. You could have deep invisible problems that only shine up after many weeks or months.
The next step was trying ConfigServer Firewall. But that also proved pointless. Be careful too, because when you install ConfigServer firewall it tells you to remove FirewallD:
Warning:
CSF does not function with the
firewalld
utility. If you install CSF, you must remove thefirewalld
utility. To do this, run theyum remove firewalld
command.
So now, by trying to fix one thing, you might have broken the default cPanel configuration. You do not want to break your cPanel security.
TLDR;
Here are the key settings apparently missing on a default cPanel server that is going to get attacked:
- Reverse DNS rDNS / PTR protection
- Connection Tracking Protection
It appears the default settings for Exim accepts connection requests without a valid PTR. This is incredulous but true.
Let’s see two ways to fix this:
Exim Configuration Manager / Advanced
acl_smtp_connect
is the correct ACL name for handling connections at the SMTP connect stage. This is the earliest and best stage to get rid of the fuckers.
Here’s the corrected approach:
Reject at SMTP Connect (Earliest Possible)
Super strict (all ports)
Only do this in a closed user group, not in a public facing server.
Add this to acl_smtp_connect
, but custom_begin_connect
section:
# In your custom_begin_connect section: # Accept IPv6 and IPv4 localhost for Roundcube accept hosts = <; 127.0.0.1 ; ::1 control = no_pipelining # Reject if no PTR record exists deny message = Connection rejected: No reverse DNS record condition = ${if eq{$sender_host_name}{}{yes}{no}} # Reject if PTR exists but verification fails deny message = Connection rejected: Reverse DNS verification failed !verify = reverse_host_lookup condition = ${if !eq{$sender_host_name}{}{yes}{no}}
Relaxed (only port 25)
# Accept IPv6 and IPv4 localhost for Roundcube accept hosts = <; 127.0.0.1 ; ::1 control = no_pipelining # Reject if no PTR record exists – only for port 25 deny message = Connection rejected: No reverse DNS record condition = ${if and{{eq{$sender_host_name}{}{}}{eq{$received_port}{25}}}{yes}{no}} # Reject if PTR exists but verification fails – only for port 25 deny message = Connection rejected: Reverse DNS verification failed !verify = reverse_host_lookup condition = ${if and{{!eq{$sender_host_name}{}{}}{eq{$received_port}{25}}}{yes}{no}}
Sequence for Testing Localhost
# /usr/sbin/exim -bh ::1 **** SMTP testing session as if from host ::1 **** but without any ident (RFC 1413) callback. **** This is not for real! >> host in hosts_connection_nolog? no (option unset) LOG: SMTP connection from [::1] >>> host in host_lookup? no (option unset) >>> host in host_reject_connection? no (option unset) >>> host in sender_unqualified_hosts? no (option unset) >>> host in recipient_unqualified_hosts? no (option unset) >>> host in helo_verify_hosts? no (option unset) >>> host in helo_try_verify_hosts? no (option unset) >>> host in helo_accept_junk_hosts? >>> list element: * >>> host in helo_accept_junk_hosts? yes (matched "*") >>> using ACL "acl_smtp_connect" >>> processing "accept" (/etc/exim.conf 443) >>> check hosts = ::1 >>> host in "::1"? >>> list element: :1 >>> host in "::1"? no (malformed IPv6 address or address mask: :1) >>> accept: condition test failed in ACL "acl_smtp_connect" >>> processing "accept" (/etc/exim.conf 446) >>> check hosts = 127.0.0.1 >>> host in "127.0.0.1"? >>> list element: 127.0.0.1 >>> host in "127.0.0.1"? no (end of list) >>> accept: condition test failed in ACL "acl_smtp_connect" >>> processing "deny" (/etc/exim.conf 449) >>> message: Connection rejected: No reverse DNS record >>> looking up host name for ::1 >>> IP address lookup yielded "localhost" >>> ╎check dnssec require list >>> ╎ localhost not in empty list (option unset? cannot trace name) >>> ╎check dnssec request list >>> ╎ localhost not in empty list (option unset? cannot trace name) >>> local host found for non-MX address >>> checking addresses for localhost >>> 127.0.0.1 >>> no IP address for localhost matched ::1 >>> ::1 does not match any IP address for localhost >>> check condition = ${if eq{$sender_host_name}{}{yes}{no}} >>> = yes >>> deny: condition test succeeded in ACL "acl_smtp_connect" >>> end of ACL "acl_smtp_connect": DENY 550 Connection rejected: No reverse DNS record LOG: H=[::1] rejected connection in "connect" ACL: Connection rejected: No reverse DNS record
ConfigServer Firewall
Be careful with this solution because you might have many new blocking problems and lots of customer support!
Don’t bother looking for “SMTP” using the UI. You’ll need CT_
items or PORTFLOOD
to fix.
# In /etc/csf/csf.conf PORTFLOOD = "25;tcp;10;300"
service csf restart; service lfd restart
Check the log, but no notifications won’t show you much:
tail -f /var/log/lfd.log
Watching attackers in real time
Threshold > 5 on ports 25
, 465
, and 587
:
watch -n 1 "netstat -plan \ | awk '\$4 ~ /:(25|465|587)\$/ { split(\$4,l,\":\"); split(\$5,r,\":\"); k=r[1]\" → \"l[2]; c[k]++ } \ END { for (k in c) if (c[k]>5) printf \"%s\tINCOMING\t%d connections\n\",k,c[k] }'"
Fun fact, set port flood to 10 as above and real time > 5 and see them go poof.
Roll you own
Only do this if you’re brave.
- Exclude some IPs or ranges (this is fruitless on a public facing server)
- Block servers who have a simultaneous threshold of X (10 in the example)
- This is done by Netstat and then looped over
- The loop checks for PTRs
- The loop skips excluded IPs
- The loop skips forward match reverse IPs
- The loop checks for PTRs
- This is done by Netstat and then looped over
- Create a log file every time there is a block
# cat scripts/top-talkers.sh #!/bin/bash THRESHOLD=10 LOGFILE="/root/scripts/smtp_conn_watch.log" NEW_ENTRIES=$(mktemp) EXCLUDED_IPS=("192.168.10.1" "1.1.1.1" "209.85") netstat -plan | grep ':25' | awk '{print $5}' | cut -d: -f1 | sort | uniq -c | awk -v t="$THRESHOLD" '$1 > t' | while read -r count ip; do [[ ! $ip =~ ^[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+$ ]] && continue ptr=$(dig +short -x "$ip" | sed 's/\.$//') line="$(date '+%Y-%m-%d %H:%M:%S') - $ip, ${ptr:-PTR_NOT_FOUND}, $count" echo "$line" >> "$LOGFILE" # <-- append # Skip excluded IPs for excluded in "${EXCLUDED_IPS[@]}"; do [[ "$ip" == "$excluded"* ]] && continue 2 done # Skip FCrDNS-valid hosts if [[ -n "$ptr" && "$(dig +short "$ptr" | sed 's/\.$//')" == "$ip" ]]; then continue fi # Drop if not already blocked if ! /usr/sbin/iptables -C INPUT -s "$ip" -j DROP 2>/dev/null; then /usr/sbin/iptables -A INPUT -s "$ip" -j DROP echo "$line" >> "$NEW_ENTRIES" fi done # Mail new blocks, if any if [ -s "$NEW_ENTRIES" ]; then mail -s "Top Talkers" -r "[email protected]" \ [email protected] < "$NEW_ENTRIES" fi rm -f "$NEW_ENTRIES"
Maintenance
nft -a list chain filter INPUT
for i in {91..132}; do nft delete rule ip filter INPUT handle $i; done
cPanel “alternatives” document
References
- https://kb.vander.host/knowledgebase/control-panels/how-to-configure-a-firewall-for-whm-cpanel-to-say-block-port-25-spammers/
- https://support.cpanel.net/hc/en-us/articles/360052384014-Why-do-email-users-get-bounces-with-421-Too-many-concurrent-SMTP-connections-please-try-again-later
- https://kb.vander.host/knowledgebase/control-panels/how-to-turn-fine-tune-or-turn-off-excessive-notifications-in-whm-when-using-csf-firewall/
- https://kb.vander.host/knowledgebase/security/how-to-disable-configserver-firewall-on-whms-lfd-blocked-emails/
- https://kb.vander.host/knowledgebase/control-panels/how-to-get-ftp-on-whm-working-if-youre-using-pure-ftpd-and-configserver-firewall/
- https://kb.vander.host/knowledgebase/control-panels/how-to-change-the-ssh-port-on-a-whm-server/