Clean up incomplete 20.04->22.04 upgrade: repos, packages, restarts

Found the underlying cause of the flaky third-party repos: a
do-release-upgrade from 2026-07-18 stalled mid-way at an interactive
dpkg prompt and was never resumed, leaving mariadb-server-10.3,
docker, rspamd, and php7.4/8.0 all on stale focal-era state. Fixed the
three third-party apt sources (docker, rspamd, ondrej PHP PPA) to
point at jammy - only actually worked once the DNSSEC bug was fixed,
since that was silently blocking repo resolution too. Reviewed
rspamd's 3.11->4.1.3 changelog against the live config before
upgrading (no breaking changes apply here), then ran apt upgrade
(dry-run confirmed zero removals, no mariadb involvement) covering
docker-ce, PHP, rspamd, and 74 other packages. Restarted all
needrestart-flagged services carefully, verifying each (ssh first and
deliberately given it's the only remote access, then DNS/mail/security
services, then containers). Everything confirmed healthy.

MariaDB 10.3->10.6 migration and the actual 22.04->24.04 upgrade are
still pending as deliberately separate next steps.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
2026-07-26 13:38:07 +02:00
co-authored by Claude Sonnet 5
parent c5bedb53b3
commit acec58b7f9
+52
View File
@@ -44,6 +44,58 @@ Nothing here is urgent; all are low-risk, no-downtime changes.
breaking DNSSEC validation for *any* signed external domain something on this host breaking DNSSEC validation for *any* signed external domain something on this host
might need to reach — not just apt/PPA lookups. might need to reach — not just apt/PPA lookups.
- [x] **Cleaned up an incomplete 20.04→22.04 release upgrade** — found and fixed 2026-07-26,
while investigating third-party apt repos ahead of a planned 22.04→24.04 upgrade.
`/var/log/dist-upgrade/` showed a `do-release-upgrade` run from 2026-07-18 that got most
of the way through (core OS, Apache, BIND, SSH — matches everything else fixed this
week) but stalled at an interactive dpkg conffile prompt for `/etc/systemd/resolved.conf`
and was never resumed. Left behind: `mariadb-server-10.3`/`mariadb-client-10.3` (and
`galera-3`, `fuse`, a couple of `libtss2`/`libsemanage` libs) still installed and marked
"pending removal" but never actually replaced by jammy's `mariadb-server-10.6`; and three
third-party apt sources (`docker.list`, `rspamd.list`, `ondrej-ubuntu-php-focal.list`)
still pointed at `focal`, with the PHP PPA disabled entirely (the release-upgrade tool
correctly remapped its `deb` line to `jammy` but left it commented out — `.distUpgrade`
backups of the originals were left alongside, clearly meant to be manually re-enabled
after upgrade completion, which never happened). System itself was stable throughout
(`dpkg --audit`/`apt-get check` both clean) — not an emergency, just deferred work.
Fixed: pointed all three sources at `jammy` (backed up originals to
`/root/removed-configs-backup/apt-repo-fixes-20260726132055/` and
`dyndns-per-host-auth-...`-adjacent dirs), confirmed with `apt update` that all three now
fetch cleanly (only worked once the DNSSEC fix above landed — this Docker repo failure
had looked like "just" a stale codename, but was actually blocked by the SERVFAIL bug
too). Reviewed rspamd's actual 3.11.0→4.1.3 changelog against this host's live
`local.d`/`override.d` config before upgrading — none of the documented breaking changes
(ed25519 DKIM, `mx_check` symbol renames, neural autolearn, worker `ssl=true`, custom Lua
`task:get_urls()`/`orig` address handling) touch anything actually configured here
(stock RSA DKIM/ARC, Redis-backed Bayes, milter proxy, no custom Lua). Ran
`apt upgrade --dry-run` first and confirmed **zero removals** and no `mariadb`/`galera`
involvement (a plain `apt upgrade` correctly refuses the `mariadb-server` metapackage
transition since it requires package replacement, not just a version bump) before
applying: 77 packages upgraded, 13 newly-installed dependency libraries, `docker-ce`
5:28.1.1→5:29.6.2, PHP 7.4.33/8.0.30 rebuilt for jammy (still the same, EOL, PHP
versions — just properly matching this OS release instead of running focal binaries),
rspamd 3.11.0→4.1.3.
`needrestart` flagged ~25 services after the upgrade, including `ssh.service`,
`named.service`, `mariadb.service`, `postfix@-.service`, `dovecot.service`, and
`fail2ban.service`. Restarted `ssh` first and deliberately (`sshd -t` config check, then
restart, then verified with a genuinely fresh SSH connection rather than assuming the
existing one survived) given it's the only remote access to this box; then the other
mail/DNS/security services one at a time with functional checks after each (SMTP/IMAP
banners, `fail2ban-client status` confirming all 19 jails and the custom
`findtime=1d` sshd/dovecot hardening survived, live `dig` against BIND). Confirmed
`docker-ce`'s own postinst had already restarted the daemon automatically (not in
needrestart's list) and all 5 containers (`portainer`, `nextcloud`, `nextcloud-redis`,
`nextcloud-db`, `web`) came back up without disruption; `rspamd 4.1.3` config syntax
checks clean (`rspamadm configtest`) with only a pre-existing, non-blocking
task-timeout-tuning advisory, web UI reachable. Left `user@1001.service` (a per-login
systemd session, not a real service) unrestarted since restarting it would disrupt the
active login for no real benefit.
**Still open, deliberately deferred**: the actual `mariadb-server-10.3``10.6` major
version migration on the live mail-account database — needs its own careful pass
(backup, `mysql_upgrade`, compatibility check) rather than being folded into this
cleanup. The 22.04→24.04 release upgrade itself is also still pending, now that this
release is actually in a clean, consistent state to upgrade *from*.
- [ ] **Mail server IP (`89.58.8.149`) listed on Barracuda's BRBL** — found - [ ] **Mail server IP (`89.58.8.149`) listed on Barracuda's BRBL** — found
2026-07-25 (user report: "jayfield.org is on the Barracuda MX 2026-07-25 (user report: "jayfield.org is on the Barracuda MX
blacklist"). Confirmed via `dig +short 149.8.58.89.b.barracudacentral.org A` blacklist"). Confirmed via `dig +short 149.8.58.89.b.barracudacentral.org A`