root@jayfield.org didn't resolve to anything (missing from both
/etc/aliases and vmail.aliases) - added it as a new alias to
jens@jayfield.org, matching the existing postmaster/hostmaster/
webmaster/wlan pattern, so the weekly report doesn't just bounce.
Also added per-run dated report files under
/var/log/attacker-check/reports/ alongside the existing CSVs, for a
plain-text historical archive browsable by date.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Parses fail2ban.log for the past 7 days into two append-only CSVs
(per-IP and per-jail rollups) for longitudinal analysis, plus a
human-readable summary emailed weekly. Two detection heuristics run on
top of the raw data: distributed single-shot-per-IP scans across a /24
(shape of the apache-noscript campaign), and regular-timing evasion on
repeat hits (shape of the sshd/dovecot low-and-slow campaigns) - both
modeled on real campaigns found earlier this engagement. A live test
run against real logs correctly re-identified the same dovecot
campaign IPs found manually earlier, with matching interval timing.
Runs weekly via cron, Sunday 06:00, clear of logrotate's daily timer.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Codifies every manual verification done throughout today's 22.04->24.04
upgrade work into a single script: package consistency, SSH hardening,
DNS/DNSSEC, full mail stack (including a live rspamd scan test - would
have caught today's phantom-active rspamd immediately), MariaDB/vmail,
fail2ban jails and custom hardening, all public vhosts, and Docker
containers. Runs automatically once per boot via a systemd oneshot
unit, since every significant change so far has ended in a reboot
anyway; also safe to run manually. Emails a pass/warn/fail summary to
jens@jayfield.org via the local mail stack. Tested live: manual run
and an actual reboot both produced identical clean results (44 pass,
1 warn - pending routine package updates, 0 fail).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Ran do-release-upgrade detached and non-interactive after the specific
failure mode that killed the 2026-07-18 attempt (an unattended
interactive prompt), validated with a full test reboot first. Caught
and fixed 12 more leftover packages from the original incomplete
upgrade that an earlier cleanup pass had missed. Completed successfully
into Ubuntu 24.04.4 / kernel 6.8.0-136.
Two real post-upgrade problems found and fixed same day: fail2ban
failed to start due to a genuine gap in the noble package (a
[DEFAULT] value referenced but never defined), fixed with an owned
jail.d drop-in; rspamd was fully removed (not just held back like
PHP), its service masked as a safety measure, its apt source migrated
to the new deb822 format and disabled - reinstalled from a re-enabled
noble-pointed repo and verified it's genuinely scanning again, not
just a phantom "active" service.
Full verification pass confirms mail, DNS/DNSSEC, dyndns, Docker,
and SSH hardening all intact. MariaDB auto-upgraded again to noble's
native 10.11.14 as part of this run; vmail data verified unchanged.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
README.md's credential-audit table and command-injection writeup still
described the per-user auth binding gap as open; it's been closed since
2026-07-26. SETUP.md's TSIG section still taught the old single-shared-
key pattern for a fresh install - replaced with the per-host key +
update-policy pattern actually running in production now, plus a
pointer to the phased-rollout approach documented in TODO.md.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Both were left as "revisit if it escalates" notes from earlier audits.
Checked live logs on alpha via SSH (port 10022): postfix-sasl remains
scattered low-volume noise, not a coordinated low-and-slow campaign like
the dovecot spray; the original three /24s behind the apache-noscript
distributed scan are no longer active, current activity is normal
per-IP jail behavior. No fixes needed for either; also reconfirmed the
dyndns per-user auth binding gap is still present and unfixed.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Real root cause turned out to differ from the original TODO: there was
no broken global CustomLog, mod_vhost_combined was already logging
every vhost without its own CustomLog into other_vhosts_access.log.
Added a dedicated CustomLog per real site anyway for easier incident
review, verified live with curl against all 7 domains after reload.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
README's public-access table and "previously resolved" summary now
reflect the dyndns injection fix and the TSIG-permission regression it
uncovered. Adds two open TODO items found during that audit: the dyndns
app's lack of real per-user auth binding, and the fact that access
logging for every real vhost on this host is currently a no-op.
A stray /var/www/html/.htaccess (dated 2025-02-18, predating everything
else in TODO.md) was silently requiring HTTP Basic Auth for jayfield.org
and www.jayfield.org's shared DocumentRoot, 401ing every real visitor to
the intentionally-public static site. Removed (backed up, not deleted).
Also adds a README table auditing which public hostnames do/don't
require credentials, prompted by checking this.
Moved Portainer's HTTPS UI behind an Apache reverse proxy on its own
subdomain (portainer.jayfield.org) instead of publishing it directly on
:9443 with a self-signed cert, which HSTS's includeSubDomains policy made
unreachable in Firefox with no click-through exception. Same pattern as
the existing web.jayfield.org/cloud.jayfield.org proxied subdomains: DNS
A record, Apache vhost proxying to 127.0.0.1:9443 over HTTPS, real
Let's Encrypt cert via certbot, standard HSTS header. Portainer container
recreated to bind 9443 to loopback only; verified end-to-end (cert chain,
HSTS header, redirect, and that :9443 is no longer reachable externally).
Firefox refuses https://alpha.jayfield.org:9443 with
MOZILLA_PKIX_ERROR_SELF_SIGNED_CERT and no click-through: the
includeSubDomains HSTS policy on the real alpha.jayfield.org vhost pins
the hostname to trusted-cert-only HTTPS on every port, but Portainer
still serves its stock self-signed cert directly on 9443. Records the
root cause and the planned reverse-proxy fix; no server changes made yet.
Records the host's service inventory, setup runbook, mail account list,
sync plan, package diff vs. clean install, and hardening TODOs, including
today's SSH hardening (key-only, no root login) and the fail2ban sshd
findtime fix for a low-and-slow brute-force evasion pattern.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RdTyvEJkfNLDVAWt8WQ6X9
jensandClaude Sonnet 5