Synopsis
cPanel uses Dovecot as it’s IMAP storage. Dovecot can be configured for either Maildir
or mdbox
format.
During a large (1 TB) transfer from the one continent to another one, we observed the following behaviour during a botched transfer:
Server A: Reliable
Server B: Mostly reliable but crashed a few times during the transfer due to poor NFS connectivity and the NAS bombing. Bandwidth was also pretty dire.
Network link: Fine, but not super fast. Transfer took almost 5 days due to slow disks and a bad network on the destination side.
Symptoms
Client observed some emails disappearing before their eyes.
Those same clients observed that certain emails were lost, between set dates. Everything else was there and one could see in Track Delivery that all those emails were delivered.
Explanation and Fix
Somewhere along the line the Transfer Tool decides to convert Maildir
to mdbox
. Since the transfer was botched there were batches on the destination side where Maildir messages were still stuck in cur
and new
and all the other traditional Maildir
places. Email works, it’s just messages are missing but found in the file system at these locations.
Once off fix per mailbox
Replacements in the script example below:
- mailbox is username before the @ sign
- cpanel_user = the cPanel user directory
- example.com = the domain name
Check your work before running the script. Check by going into Roundcube at the missing dates, and make sure they are missing. Then when you run the script below, Roundcube should display the older messages first.
doveadm import -u mailbox@example.com maildir:/home/cpanel_user/mail/example.com/mailbox INBOX ALL
Batch script to fix for everyone
I called this recover-maildir.sh
#!/bin/bash # Users to skip skip_users=("user1" "user2" "user3" "user4") for dir in /home/cpanel_user/mail/example.com/*; do user=$(basename "$dir") # Check if user is in the skip list if [[ " ${skip_users[*]} " == *" $user "* ]]; then echo "Skipping $user@example.com (already processed)" continue fi echo "Importing mail for $user@example.com..." doveadm import -u "$user@example.com" maildir:"$dir" INBOX ALL
Errors during batch script
Doing this en-mass for 1000s of users especially legacy ones will produce some or all of the following self explanatory errors:
User doesn’t exist, corrupted indexes, INBOX.spam or INBOX.whatever doesn’t exist
doveadm(user1@example.com): Error: User doesn’t exist
doveadm(user2@example.com): Warning: fscking index file /home/cpanel_user/mail/example.com/user2/storage/dovecot.map.index
doveadm(user2@example.com): Warning: mdbox /home/cpanel_user/mail/example.com/user2/storage: rebuilding indexes
doveadm(user3@example.com): Error: Couldn’t create mailbox INBOX.spam: Mailbox doesn’t exist: INBOX.N.G
There might be more serious indexing, inode, or user mailbox full errors too. Carefully save and then analyse the output.
Remaining tasks
The script doesn’t clean up the old Maildir
s, and doing so it out of the scope of this article. However, as a responsible network administrator you may not leave those old files around. If you can’t script the deletions yourself, maybe ask cPanel via their ticketing system for help, get a specialist, or use AI. Make sure you have backups first.
Good luck!