jensandClaude Sonnet 5
b926af5fca
Both were left as "revisit if it escalates" notes from earlier audits. Checked live logs on alpha via SSH (port 10022): postfix-sasl remains scattered low-volume noise, not a coordinated low-and-slow campaign like the dovecot spray; the original three /24s behind the apache-noscript distributed scan are no longer active, current activity is normal per-IP jail behavior. No fixes needed for either; also reconfirmed the dyndns per-user auth binding gap is still present and unfixed. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
296 lines
19 KiB
Markdown
296 lines
19 KiB
Markdown
# alpha.jayfield.org — TODO
|
|
|
|
Outstanding hardening items (see `README.md` for the full service overview).
|
|
Nothing here is urgent; all are low-risk, no-downtime changes.
|
|
|
|
- [ ] **Mail server IP (`89.58.8.149`) listed on Barracuda's BRBL** — found
|
|
2026-07-25 (user report: "jayfield.org is on the Barracuda MX
|
|
blacklist"). Confirmed via `dig +short 149.8.58.89.b.barracudacentral.org A`
|
|
→ `127.0.0.2` (listed); clean on Spamhaus ZEN, SpamCop, and SORBS —
|
|
Barracuda-specific. Checked the server for actual abuse before
|
|
requesting removal: mail queue empty, outbound volume ~11 msgs over
|
|
the prior week, only known SASL logins (`jens@`/`otto2022@jayfield.org`
|
|
from the usual home IP), no backscatter, correct PTR
|
|
(`alpha.jayfield.org`), SPF/DKIM/DMARC (`p=reject`) all already live —
|
|
no sign of a compromised account or open relay, so most likely a
|
|
stale/inherited reputation hit on the IP rather than active abuse
|
|
from this host.
|
|
Submitted a removal request via
|
|
`https://www.barracudacentral.org/rbl/removal-request` on 2026-07-25.
|
|
**Confirmation number: `BBR21785009436-19117-9182`.** Per Barracuda's
|
|
response, the IP gets a temporary 48h reputation bump to "normal"
|
|
while they investigate (up to ~1h to propagate), after which it
|
|
either stays clear or reverts to listed.
|
|
**Status 2026-07-26: still clear** (`dig +short
|
|
149.8.58.89.b.barracudacentral.org A` → NXDOMAIN, no `127.0.0.2`) —
|
|
matches the 1h check (routine `trig_01F1RMhEtNHtAbuHb9ffe1oX`, fired
|
|
2026-07-25 21:02 UTC, also clear). This is expected during the 48h
|
|
"normal" bump window, not yet proof of permanent delisting. The real
|
|
test is the 48h check (routine `trig_019GayXSo5TW5eSxiJ99Ghrp`,
|
|
scheduled 2026-07-27 20:03 UTC) — if it reverts to listed, next step
|
|
is a manual follow-up to Barracuda citing the confirmation number
|
|
above.
|
|
|
|
- [ ] **dyndns app has no real per-user auth binding** — found 2026-07-20,
|
|
while fixing the command injection below.
|
|
`index.php`'s `pass == '_NO_PASS_'` check is a hardcoded magic
|
|
string, not a real per-user credential — there's nothing tying the
|
|
single shared Apache Basic Auth identity to *which* hostname label
|
|
gets updated. Any caller with valid dyndns credentials can update
|
|
any of the zone's 5 hostnames (`jens`, `kack`, `test`, `uschi`,
|
|
`vpn`), not just their own. Low severity today since apparently only
|
|
one person (`jens`) has credentials, but worth fixing before adding
|
|
any second dyndns user. Fix: either move to per-host TSIG-style
|
|
tokens, or add a static host-allowlist per `.htpasswd` user in the
|
|
PHP.
|
|
|
|
- [x] **dyndns/default-vhost access logging is a no-op** — done 2026-07-25
|
|
found 2026-07-20 while investigating the command injection below.
|
|
`dyndns.jayfield.org`, `jayfield.org`, `www.jayfield.org`,
|
|
`mail.jayfield.org`, `web.jayfield.org`, and `cloud.jayfield.org`
|
|
had no `CustomLog` directive of their own, and `access.log` (only
|
|
written by `default-ssl.conf`/`000-default.conf`) had 0-8 lines —
|
|
looking like no real vhost was being logged at all.
|
|
Root cause (2026-07-25 re-check): the fallback assumption was wrong.
|
|
There's no global `CustomLog access.log` in `apache2.conf`; instead
|
|
`conf-enabled/other-vhosts-access-log.conf` enables `mod_vhost_combined`,
|
|
which was already writing every vhost-without-its-own-`CustomLog`'s
|
|
traffic to `/var/log/apache2/other_vhosts_access.log` (5,432 lines,
|
|
confirmed real traffic incl. bot scans against `cloud.jayfield.org`).
|
|
So requests were never actually un-logged — just commingled in one
|
|
shared, harder-to-grep file instead of `access.log`.
|
|
Fixed anyway, since a dedicated per-site log remains the better
|
|
end state for incident review: added an explicit
|
|
`CustomLog ${APACHE_LOG_DIR}/<site>_access.log combined` to every
|
|
`<VirtualHost>` block for `dyndns`, `jayfield.org`, `www`, `mail`,
|
|
`web`, `cloud`, and `portainer` (the `:80` and `:443`/`le-ssl` blocks
|
|
of each site share one log file). Configs backed up to
|
|
`/root/removed-configs-backup/vhost-customlog-20260725203959/`.
|
|
`/etc/logrotate.d/apache2` already globs `*.log`, so no logrotate
|
|
change needed. Validated with `apache2ctl configtest`, applied via
|
|
`systemctl reload apache2`, verified live with `curl` against all 7
|
|
domains and confirmed each new `<site>_access.log` recorded the hit.
|
|
|
|
- [x] **OS command injection in the dyndns update scripts** — done 2026-07-20
|
|
`/var/www/dyndns/nsupdate.php`, `nsupdate_ipv4.php`, and
|
|
`nsupdate_ipv6.php` (used by both `index.php` and `nic/update.php`,
|
|
the router/ddclient-facing endpoint) built a shell command string by
|
|
directly interpolating unsanitized `$_GET` input (`hostname`/`myip`/
|
|
`ip`, which become `$host`/`$ip`/`$ipv4`/`$ipv6`) and passed it to
|
|
PHP's `exec()`, which runs it via `/bin/sh -c` — no
|
|
`escapeshellarg()`/`escapeshellcmd()` anywhere. A request like
|
|
`hostname=$(id>/tmp/pwned).dyndns.jayfield.org` would have executed
|
|
arbitrary shell commands as `www-data` (the domain-suffix check
|
|
still passed after the `.`-split). `nic/update.php` had no
|
|
application-level auth of its own at all.
|
|
Mitigating factor: the whole `dyndns.jayfield.org` vhost sits behind
|
|
Apache Basic Auth, so this needed valid (or leaked/brute-forced)
|
|
dyndns credentials to reach — not exploitable by an anonymous
|
|
visitor. Compounding factors found during the audit: PHP is
|
|
`7.4.33`, EOL since Nov 2022, `disable_functions` was empty (`exec`
|
|
fully available); `index.php`'s `pass == '_NO_PASS_'` check is a
|
|
hardcoded magic string, not real per-user auth, so any valid dyndns
|
|
credential can update *any* of the zone's 5 hostnames (`jens`,
|
|
`kack`, `test`, `uschi`, `vpn`), not just its own — left as-is, out
|
|
of scope for this fix; and the vhost's access logging turned out to
|
|
be a no-op (shared `access.log` has 0 lines), so past exploitation
|
|
couldn't be fully ruled out from logs alone — though file mtimes,
|
|
`www-data` processes/crontab, and zone contents all looked
|
|
unremarkable, consistent with unexploited.
|
|
Fixed: all three files backed up to
|
|
`/root/removed-configs-backup/dyndns-preinjectionfix-<timestamp>/`,
|
|
then rewritten to build the nsupdate command list as a plain string
|
|
written to `proc_open()`'s stdin pipe using the **array form** of
|
|
the command (`array("/usr/bin/nsupdate", "-k", ...)`) — this never
|
|
invokes a shell at all, so no value can be interpreted as shell
|
|
syntax regardless of content. Added strict `preg_match` validation
|
|
requiring `$host`/`$domain` to be shaped like a valid hostname
|
|
(letters/digits/hyphens, dot-separated labels) and
|
|
`filter_var(..., FILTER_VALIDATE_IP, FILTER_FLAG_IPV4/IPV6)` on the
|
|
IP before use, as defense in depth on top of removing the shell.
|
|
**Regression found and fixed along the way**: functional testing
|
|
showed legitimate updates were *already silently failing*
|
|
(`update failed: REFUSED`, `permission denied` reading the TSIG
|
|
key) — turned out `www-data` was never granted read access to
|
|
`/etc/bind/dyndns.jayfield.org.key` when its permissions were
|
|
tightened to `640 root:bind` in the item below, so the dyndns web
|
|
update feature had been broken for real users since 2026-07-19,
|
|
independent of this vulnerability. Fixed with a narrowly-scoped
|
|
POSIX ACL (`setfacl -m u:www-data:r
|
|
/etc/bind/dyndns.jayfield.org.key`, `acl` package installed) rather
|
|
than adding `www-data` to the `bind` group — the group also owns
|
|
`rndc.key` (full remote BIND control), which would have been a much
|
|
bigger privilege grant than dyndns needs.
|
|
Verified via `sudo -u www-data php` driving the real functions
|
|
directly (same code path as the web UI, without needing the Basic
|
|
Auth password): two injection payloads (`$(...)` and `;`-chained)
|
|
both rejected (`-1`, no `/tmp` side-effect file created), an
|
|
invalid IP rejected (`-1`), and a legitimate update succeeded
|
|
(`0`), confirmed live in the zone via `dig`, then cleaned up (test
|
|
record deleted via `nsupdate`, temp test script removed). Also
|
|
confirmed unauthenticated requests still get `401` from Apache
|
|
before ever reaching the PHP, and `apache2ctl configtest` stayed
|
|
clean throughout.
|
|
|
|
- [x] **Public static site (`jayfield.org`/`www.jayfield.org`) was 401'ing every visitor** — done 2026-07-20
|
|
Found while auditing which services are reachable without credentials:
|
|
a stray `/var/www/html/.htaccess` (`AuthType Basic`, same
|
|
`AuthName "Restricted Content"`/`AuthUserFile /etc/apache2/.htpasswd`
|
|
as the intentional `dyndns.jayfield.org` protection) was silently
|
|
requiring login for the DocumentRoot both `jayfield.org` and
|
|
`www.jayfield.org` share — `/var/www` has `AllowOverride all`, so it
|
|
took effect with no vhost-level directive needed. File was dated
|
|
2025-02-18 (`index.html`/`apache.html` themselves are from 2022) —
|
|
predates every other change in this doc by well over a year, almost
|
|
certainly a forgotten leftover rather than intentional, and
|
|
contradicts `README.md`'s own description of both as a public static
|
|
site.
|
|
Fixed: moved to `/root/removed-configs-backup/htaccess-var-www-html.<timestamp>`
|
|
(recoverable, not deleted — same convention as this file's other
|
|
removed-config backups). No Apache reload needed (`.htaccess` is
|
|
read per-request). Verified both `jayfield.org` and
|
|
`www.jayfield.org` now return `200` with real page content;
|
|
`dyndns.jayfield.org`'s own (intentional, vhost-level) Basic Auth is
|
|
untouched and still returns `401` without credentials.
|
|
|
|
- [x] **Lock down the dyndns TSIG key file permissions** — done 2026-07-19
|
|
`/etc/bind/dyndns.jayfield.org.key` was `644` (world-readable), holding
|
|
the shared secret that authenticates dynamic DNS updates for
|
|
`dyndns.jayfield.org` with no IP-based `allow-update` backing it up.
|
|
Fixed: `chmod 640` + `chown root:bind` (owner was already correct),
|
|
matching `rndc.key`'s permissions.
|
|
|
|
- [x] **Suppress BIND version disclosure** — done 2026-07-19
|
|
No `version` statement in `named.conf.options`, so
|
|
`dig CH TXT version.bind @ns1.jayfield.org` leaked the exact BIND
|
|
version (`9.18.39`) — a minor fingerprinting aid for attackers.
|
|
Fixed: added `version "unknown";` inside the `options {}` block,
|
|
validated with `named-checkconf`, and applied via `rndc reload`.
|
|
Verified `dig CH TXT version.bind` now returns `"unknown"`.
|
|
|
|
- [x] **Add response rate limiting (RRL)** — done 2026-07-19
|
|
No `rate-limit {}` block was configured. Since `allow-query { any; }`
|
|
is required for a public authoritative zone, RRL is the standard
|
|
mitigation against the server being used for spoofed-source DNS
|
|
reflection/amplification against third parties.
|
|
Fixed: added `rate-limit { responses-per-second 10; window 5; }` to
|
|
`named.conf.options`, validated with `named-checkconf`, and applied
|
|
via `rndc reload`. Started with conservative defaults (10/s, 5s
|
|
window) — revisit if legitimate high-volume resolvers get throttled.
|
|
|
|
- [x] **Add HSTS headers to all vhosts** — done 2026-07-19
|
|
No vhost sent a `Strict-Transport-Security` header, leaving a window
|
|
for on-path SSL-stripping on a user's very first plaintext request to
|
|
any domain.
|
|
Fixed: added `Header always set Strict-Transport-Security
|
|
"max-age=63072000; includeSubDomains"` to all 6 domain `:443` blocks
|
|
(`dyndns`, `jayfield.org`, `www`, `mail`, `web`, `cloud`) — skipped
|
|
`default-ssl.conf` since it's the self-signed catch-all with no real
|
|
trusted hostname for a browser to pin. Validated with
|
|
`apache2ctl configtest`, applied via `systemctl reload apache2`,
|
|
verified live with `curl -sI` against all 6 domains.
|
|
|
|
- [x] **`jayfield.org` had no `:80` vhost of its own** — done 2026-07-19
|
|
Bare `http://jayfield.org` fell through to the `dyndns` default vhost
|
|
and redirected to the wrong domain.
|
|
Fixed: added a `:80` block to `jayfield.org.conf` redirecting to
|
|
`https://jayfield.org/`, matching the `dyndns`/`mail` pattern.
|
|
Verified: `curl -sI http://jayfield.org/` → `302` with
|
|
`Location: https://jayfield.org/`.
|
|
|
|
- [x] **SSH: disable root login + password auth** — done 2026-07-19
|
|
`PermitRootLogin yes` and `PasswordAuthentication yes` were both still
|
|
enabled (the "known deviation" flagged in `SETUP.md` §1) — every
|
|
credential-stuffing bot on the internet got a real login prompt to
|
|
grind against.
|
|
Fixed: `PermitRootLogin no`, `PasswordAuthentication no`,
|
|
`PubkeyAuthentication yes` in `/etc/ssh/sshd_config` (backed up as
|
|
`sshd_config.bak.20260719225249`). The `Match Group sftp` block's own
|
|
`PasswordAuthentication yes` was left as-is — separate, already-chrooted,
|
|
no-shell/no-forwarding accounts, not part of this risk. Validated with
|
|
`sshd -t`, applied via `systemctl reload sshd`, verified live with
|
|
`sshd -T` and a fresh key-only connection before considering it done.
|
|
|
|
- [x] **Portainer's `:9443` UI is unreachable in HSTS-enforcing browsers** — done 2026-07-20
|
|
`https://alpha.jayfield.org:9443` threw
|
|
`MOZILLA_PKIX_ERROR_SELF_SIGNED_CERT` in Firefox with no click-through
|
|
option ("keine Ausnahme kann hinzugefügt werden"). Root cause: the
|
|
`includeSubDomains` HSTS policy sent by the real `alpha.jayfield.org`
|
|
vhost (`Strict-Transport-Security` header, added in the item above)
|
|
pins the *hostname* to HTTPS-with-a-trusted-cert on every port, but
|
|
Portainer (`README.md` §Docker services) was still serving its stock
|
|
self-signed cert directly on `9443`. Not a MITM — self-inflicted by
|
|
combining HSTS with an unproxied self-signed service.
|
|
Fixed: added a `portainer` A record to `db.jayfield.org` (SOA serial
|
|
`2022032216` → `2022032217`); recreated the `portainer` container
|
|
binding the HTTPS UI to `127.0.0.1:9443:9443` instead of publishing it
|
|
(old container kept renamed until the new one was verified, then
|
|
removed, matching the update procedure documented in `README.md`);
|
|
added an Apache vhost for `portainer.jayfield.org`
|
|
(`ProxyPass`/`ProxyPassReverse` to `https://127.0.0.1:9443/`,
|
|
`SSLProxyEngine on`, `SSLProxyVerify none` + `SSLProxyCheckPeerCN off`/
|
|
`SSLProxyCheckPeerName off` for the loopback hop to Portainer's
|
|
self-signed cert) — same pattern as `web.jayfield.org`/
|
|
`cloud.jayfield.org` (`SETUP.md` §4/§5); issued a real cert via
|
|
`certbot --apache -d portainer.jayfield.org --redirect`; added the
|
|
standard HSTS header. Left the `8000` edge-agent tunnel port
|
|
published as-is — no evidence it's in use.
|
|
Verified: `openssl s_client`/`curl` confirm the Let's Encrypt cert
|
|
(not self-signed) and `Strict-Transport-Security` header on
|
|
`https://portainer.jayfield.org`, `http://` redirects 301 to
|
|
`https://`, and `ss -ltnp` shows `9443` bound to `127.0.0.1` only
|
|
(via `docker-proxy`), so it's no longer reachable on the public
|
|
interface at all — access is now exclusively via
|
|
`https://portainer.jayfield.org`.
|
|
One transient wrinkle during rollout: the very first few requests
|
|
through the new proxy vhost returned a stray `401` with dyndns's
|
|
Basic Auth realm ("Restricted Content") — looked alarming, but
|
|
`LogLevel trace8` on the vhost showed Apache correctly SNI-matching
|
|
and proxying to the new container with a clean backend `200`;
|
|
resolved itself within seconds (looked like a stale pooled proxy
|
|
connection surviving the container recreate) and was stable across
|
|
repeated checks before the debug logging was removed again.
|
|
|
|
- [x] **fail2ban: `sshd` jail was evadable via low-and-slow attempts** — done 2026-07-19
|
|
Default `findtime=30m`/`maxretry=3` never triggered for `47.76.192.176`,
|
|
which had been hitting `root`/`jayfield` logins roughly every 33 minutes
|
|
for most of a day — just outside the 30-minute window, so failures kept
|
|
aging out before reaching 3 strikes. Confirmed via `fail2ban.log`
|
|
(`Found`/`Ban` counts) that no other jail showed the same per-IP timing
|
|
pattern; a separate, *distributed* single-shot scan on `apache-noscript`
|
|
was noted but left open (see `README.md`).
|
|
Fixed: `/etc/fail2ban/jail.d/sshd-findtime.local` overrides `sshd` to
|
|
`findtime=1d`, `maxretry=4`, `bantime=1d`, kept in its own drop-in
|
|
rather than editing the Debian-packaged jail file. Applied via
|
|
`fail2ban-client reload sshd`; `47.76.192.176` was banned on its very
|
|
next attempt after the change.
|
|
|
|
- [x] **fail2ban: `dovecot` jail had the same low-and-slow gap as `sshd`** — done 2026-07-25
|
|
found while auditing which IPs were hitting webmail hard enough to
|
|
deserve a ban but weren't in any currently-banned list. Same default
|
|
`findtime=30m`/`maxretry=3` as `sshd`'s pre-fix config. At least 4 IPs
|
|
(`47.95.196.60` 167 hits, `82.157.132.42` 144, `120.48.118.132` 76,
|
|
`188.164.195.116` 57, `157.245.38.26` 34) ran a coordinated IMAP
|
|
password spray against generic mailbox names (`admin@`, `test@`,
|
|
`support@`, `sales@jayfield.org`) starting almost simultaneously around
|
|
2026-07-21 22:40-23:38, each pacing itself at ~1 attempt/22-25min —
|
|
just outside the 30-minute window, so none ever tripped 3 strikes.
|
|
`120.48.118.132` was still active as late as 2026-07-25 13:32, so the
|
|
campaign was ongoing, not historical. `postfix-sasl` shares the same
|
|
default thresholds and mail stack but showed much lower per-IP volume
|
|
in this pass — worth a follow-up check if it starts trending the same
|
|
way.
|
|
**Follow-up 2026-07-26**: checked `postfix-sasl` — still just scattered
|
|
background noise, not a coordinated campaign. Top offender
|
|
(`172.94.9.203`) logged 32 hits but all from late June/early July, none
|
|
since; current activity is single/double attempts from many different
|
|
IPs (no repeated per-IP low-and-slow pattern like the dovecot spray).
|
|
`findtime=30m`/`maxretry=3` defaults left as-is — no fix needed unless
|
|
this changes.
|
|
Fixed: `/etc/fail2ban/jail.d/dovecot-findtime.local`, same values as
|
|
the `sshd` fix (`findtime=1d`, `maxretry=4`, `bantime=1d`). Applied via
|
|
`fail2ban-client reload dovecot`; verified live with
|
|
`fail2ban-client get dovecot findtime/maxretry/bantime` and confirmed
|
|
the jail's existing ban state (`47.88.56.201`, 29 total bans) survived
|
|
the reload.
|