After Ubuntu 18.04 to 20.04 upgrade, MySQL stops working with “Server upgrade in progress”

Background

After upgrading from Ubuntu 18.04 to Ubuntu 20.04, MySQL isn’t working and systemctl status mysql.service show there are problems. You see this line lurking in the output:

Status: "Server upgrade in progress"

Next you check, and on the surface it appears that the latest MySQL is actually installed:

root@server:~# mysql -V
mysql Ver 8.0.22-0ubuntu0.20.04.3 for Linux on x86_64 ((Ubuntu))

You then reach for google and your old toolbox and try this command

root@ns3:~# mysql_upgrade

The next minute you are completely overwhelmed by Oracle speak. Do not panic. Go and make a cut of tea. Skip over this nonsense and read on after Oracle’s strange message that completely jerk’s the head.

The mysql_upgrade client is now deprecated. The actions executed by the upgrade client are now done by the server.
To upgrade, please start the new MySQL binary with the older data directory. Repairing user tables is done automatically. Restart is not required after upgrade.
The upgrade process automatically starts on running a new MySQL binary with an older data directory. To avoid accidental upgrades, please use the –upgrade=NONE option with the MySQL binary. The option –upgrade=FORCE is also provided to run the server upgrade sequence on demand.
It may be possible that the server upgrade fails due to a number of reasons. In that case, the upgrade sequence will run again during the next MySQL server start. If the server upgrade fails repeatedly, the server can be started with the –upgrade=MINIMAL option to start the server without executing the upgrade sequence, thus allowing users to manually rectify the problem.

-Amen

You find this stack answer:
https://askubuntu.com/questions/292865/upgrade-mysql-server-issue

It’s strikingly similar to your situation, so much so, that you’re getting confidence the problem will be resolved soon. The tea is suddenly tasting better.

You try the approach mentioned in Stack, getting to the purge stuff, and then become a bit nervous and have a mild shiver in your spine thinking about the word purge and your beautiful databases.

Luckily somewhere in the comments you read that purging is fine, no databases will be removed. You relax again, and sip some more tea.

Alas you do the whole thing, but still no luck. Eish.

Solution

On our system we had to take a more radical approach. Instead of doing purge mysql-server-core-5.5 we did a purge mysql*

But here you need to carefully observe output, because although this fixed the MySQL problem, it removed another library which depends on MySQL server (libdbd-mysql-perl). This is specific to our system. We simply added that library back again.

This was the key output to observe:

The following packages will be REMOVED:
libdbd-mysql-perl* libmysqlclient21* mysql-client* mysql-client-8.0* mysql-client-core-8.0* mysql-common*
mysql-server-5.7* mysql-server-8.0* mysql-server-core-8.0*

If I had the opportunity again to break a server, I’d substitute the Stack author’s line

sudo apt-get purge mysql-server-core mysql-server-core-5.5

With

sudo apt-get purge mysql-server-core mysql-server-core-5.7

Alas no can do have to finish this tea and move on to the other 1000 things on the list.

Reference

https://askubuntu.com/questions/292865/upgrade-mysql-server-issue