diff --git a/README.md b/README.md index ae28a53..2de1cd7 100644 --- a/README.md +++ b/README.md @@ -94,12 +94,13 @@ one-off `450 4.7.25 Client host rejected` shows up in `mail.log`. | `mail.jayfield.org` | → redirect to :443 | Webmail-adjacent, proxies `/rspamd` → `http://localhost:11334/` | | | `web.jayfield.org` | → redirect to :443 | Reverse-proxies to the **Docker `web` (nginx) container** on `127.0.0.1:8081` | New — see below | | `cloud.jayfield.org` | → redirect to :443 | Reverse-proxies to the **Docker `nextcloud` container** on `127.0.0.1:8082`, sets `X-Forwarded-Proto: https` | Nextcloud — see below | +| `portainer.jayfield.org` | → redirect to :443 | Reverse-proxies to the **Docker `portainer` container** on `127.0.0.1:9443` over HTTPS (`SSLProxyEngine`/`SSLProxyVerify none` for the loopback hop to Portainer's self-signed cert) | Added 2026-07-20 — see "Portainer" below | | default (`_default_:443`, `000-default` on :80) | Apache's stock placeholder page | — | | -All Let's Encrypt certs (7 total, one per hostname/group) auto-renew via +All Let's Encrypt certs (8 total, one per hostname/group) auto-renew via `certbot.timer`; verified with `certbot renew --dry-run`. -All 6 domain `:443` blocks send `Strict-Transport-Security: max-age=63072000; +All 7 domain `:443` blocks send `Strict-Transport-Security: max-age=63072000; includeSubDomains` (`mod_headers`). Deliberately skipped on the `default-ssl.conf` catch-all — it's the self-signed fallback for unmatched SNI/Host headers, not a real trusted hostname a browser could pin. @@ -108,7 +109,7 @@ SNI/Host headers, not a real trusted hostname a browser could pin. | Container | Image | Ports | Purpose | |---|---|---|---| -| `portainer` | `portainer/portainer-ce:2.39.5` | `8000` (edge-agent tunnel, published), `9443` (HTTPS UI, published), `9000` (UI, internal only) | Docker management UI — `https://alpha.jayfield.org:9443` directly on its stock self-signed cert (**broken in HSTS-enforcing browsers**, see "Open items" below; planned fix is a `portainer.jayfield.org` reverse-proxy vhost, not yet applied) | +| `portainer` | `portainer/portainer-ce:latest` | `8000` (edge-agent tunnel, published), `9443` (HTTPS UI, `127.0.0.1`-only), `9000` (UI, internal only) | Docker management UI — `https://portainer.jayfield.org`, reverse-proxied by Apache to `127.0.0.1:9443` (fixed 2026-07-20, see below) | | `web` | `nginx:latest` | `127.0.0.1:8081` (localhost-only, not published externally) | Backing service for `web.jayfield.org`; only reachable through the Apache reverse proxy | | `nextcloud` | `nextcloud:apache` | `127.0.0.1:8082` (localhost-only) | Nextcloud app (PHP+Apache bundled in the image); backing service for `cloud.jayfield.org` | | `nextcloud-db` | `mariadb:lts` | internal only (`nextcloud_net`) | Dedicated DB for Nextcloud — kept separate from the host's own MySQL (which only holds `vmail`) | @@ -139,6 +140,26 @@ This is the pattern to follow for any future container-backed site: publish the container to `127.0.0.1:`, add an Apache vhost for the subdomain that proxies to it, then run `certbot --apache -d ` to get HTTPS. +### Portainer (`portainer.jayfield.org`) + +``` +Internet → Apache :80/:443 (portainer.jayfield.org vhost) + → Let's Encrypt TLS termination + → ProxyPass https://127.0.0.1:9443/ (SSLProxyVerify none — + backend keeps Portainer's own self-signed cert, but that + hop never leaves the host) + → Docker container "portainer" HTTPS UI, published to + 127.0.0.1 only +``` + +Previously Portainer published `9443` directly to the public interface with +its own self-signed cert; fixed 2026-07-20 after that became unreachable in +HSTS-enforcing browsers (see `TODO.md`). The container was recreated with +`-p 127.0.0.1:9443:9443` (old container kept renamed until the new one was +verified healthy, then removed — same caution as the update procedure +below). The `8000` edge-agent tunnel port is still published directly; +nothing currently depends on it, but it wasn't in scope for this fix. + ### Nextcloud (`cloud.jayfield.org`) Minimal, data-hosting-focused deployment — core Files app only. No @@ -217,18 +238,6 @@ endpoint instead of a bespoke Apache-WebDAV + SFTP-upload pipeline: ## Open items / known quirks -- **Portainer `:9443` UI unreachable in HSTS-enforcing browsers** — found - 2026-07-20 when Firefox refused `https://alpha.jayfield.org:9443` with - `MOZILLA_PKIX_ERROR_SELF_SIGNED_CERT` and no click-through exception. - Cause: the `includeSubDomains` HSTS policy sent by the real - `alpha.jayfield.org` vhost pins the hostname to HTTPS-with-a-trusted-cert - on *every* port, but Portainer still answers `9443` directly with its own - self-signed cert — not a MITM, a self-inflicted interaction between two - otherwise-correct pieces of config. Planned fix: move Portainer behind an - Apache reverse proxy on its own subdomain (`portainer.jayfield.org`), - same pattern as `web.jayfield.org`/`cloud.jayfield.org`. See `TODO.md` - for the concrete steps; not yet applied. - - **Intranet migration from `vlda-01` (files, Gitea, Jellyfin, Dogecoin node/wallet)** — planning stage only, not started. See `SYNC-PLAN.md` for the full per-service design, open questions, and build order. @@ -331,4 +340,14 @@ key-only) and the `sshd` fail2ban jail's `findtime` widened from the 33 minutes for most of a day — just outside the old 30-minute window, so it never accumulated enough strikes to get banned; see `TODO.md` and `SETUP.md` §1/§3 for the fix and the same evasion check applied to every -other jail (only one other pattern turned up — see "Open items" above).) +other jail (only one other pattern turned up — see "Open items" above). +Also: Portainer's `:9443` UI, previously published directly to the public +interface on its own self-signed cert, became unreachable in +HSTS-enforcing browsers (`MOZILLA_PKIX_ERROR_SELF_SIGNED_CERT`, no +click-through) once the `includeSubDomains` HSTS policy above started +pinning `alpha.jayfield.org` to trusted-cert-only HTTPS on every port — +not a MITM, a self-inflicted interaction between two otherwise-correct +pieces of config. Fixed 2026-07-20 by moving it behind an Apache reverse +proxy on its own subdomain, `portainer.jayfield.org`, same pattern as +`web.jayfield.org`/`cloud.jayfield.org`; see "Portainer" above and +`TODO.md` for the full details.) diff --git a/TODO.md b/TODO.md index 58ce0c8..2cd8acf 100644 --- a/TODO.md +++ b/TODO.md @@ -61,26 +61,45 @@ Nothing here is urgent; all are low-risk, no-downtime changes. `sshd -t`, applied via `systemctl reload sshd`, verified live with `sshd -T` and a fresh key-only connection before considering it done. -- [ ] **Portainer's `:9443` UI is unreachable in HSTS-enforcing browsers** — found 2026-07-20 - `https://alpha.jayfield.org:9443` throws +- [x] **Portainer's `:9443` UI is unreachable in HSTS-enforcing browsers** — done 2026-07-20 + `https://alpha.jayfield.org:9443` threw `MOZILLA_PKIX_ERROR_SELF_SIGNED_CERT` in Firefox with no click-through option ("keine Ausnahme kann hinzugefügt werden"). Root cause: the `includeSubDomains` HSTS policy sent by the real `alpha.jayfield.org` vhost (`Strict-Transport-Security` header, added in the item above) pins the *hostname* to HTTPS-with-a-trusted-cert on every port, but - Portainer (`README.md` §Docker services) still serves its stock + Portainer (`README.md` §Docker services) was still serving its stock self-signed cert directly on `9443`. Not a MITM — self-inflicted by combining HSTS with an unproxied self-signed service. - Planned fix (not yet applied): rebind Portainer's published port to - `127.0.0.1:9443:9443`, add a `portainer.jayfield.org` Apache vhost + Fixed: added a `portainer` A record to `db.jayfield.org` (SOA serial + `2022032216` → `2022032217`); recreated the `portainer` container + binding the HTTPS UI to `127.0.0.1:9443:9443` instead of publishing it + (old container kept renamed until the new one was verified, then + removed, matching the update procedure documented in `README.md`); + added an Apache vhost for `portainer.jayfield.org` (`ProxyPass`/`ProxyPassReverse` to `https://127.0.0.1:9443/`, - `SSLProxyEngine on`, `SSLProxyVerify none` for the loopback hop), - issue a real cert via `certbot --apache -d portainer.jayfield.org`, - and add the standard HSTS header — same pattern already used for - `web.jayfield.org`/`cloud.jayfield.org` (`SETUP.md` §4/§5). DNS needs - a new `portainer` A record in `db.jayfield.org` first (SOA serial - bump). Leave the `8000` edge-agent tunnel port as-is — no evidence - it's in use. + `SSLProxyEngine on`, `SSLProxyVerify none` + `SSLProxyCheckPeerCN off`/ + `SSLProxyCheckPeerName off` for the loopback hop to Portainer's + self-signed cert) — same pattern as `web.jayfield.org`/ + `cloud.jayfield.org` (`SETUP.md` §4/§5); issued a real cert via + `certbot --apache -d portainer.jayfield.org --redirect`; added the + standard HSTS header. Left the `8000` edge-agent tunnel port + published as-is — no evidence it's in use. + Verified: `openssl s_client`/`curl` confirm the Let's Encrypt cert + (not self-signed) and `Strict-Transport-Security` header on + `https://portainer.jayfield.org`, `http://` redirects 301 to + `https://`, and `ss -ltnp` shows `9443` bound to `127.0.0.1` only + (via `docker-proxy`), so it's no longer reachable on the public + interface at all — access is now exclusively via + `https://portainer.jayfield.org`. + One transient wrinkle during rollout: the very first few requests + through the new proxy vhost returned a stray `401` with dyndns's + Basic Auth realm ("Restricted Content") — looked alarming, but + `LogLevel trace8` on the vhost showed Apache correctly SNI-matching + and proxying to the new container with a clean backend `200`; + resolved itself within seconds (looked like a stale pooled proxy + connection surviving the container recreate) and was stable across + repeated checks before the debug logging was removed again. - [x] **fail2ban: `sshd` jail was evadable via low-and-slow attempts** — done 2026-07-19 Default `findtime=30m`/`maxretry=3` never triggered for `47.76.192.176`,