brewpi/docs/overshoot_hold_windup.md

# HOLD-state overshoot from a disturbance: root cause and fix

Investigation of the overshoot visible in `docs/overshoot.png`: a cold-water
disturbance during a steady HOLD at 30°C caused `theta_ist` to overshoot to
~30.5°C and stay there, rather than settling back at soll. This document
records the diagnosis, the fix that shipped for it, and the test coverage
added alongside it. Numbers in the incident summary below are pulled from
`logs/log_latest.json`; names in that section (`pid_hold`/`pid_heat`) are
the *pre-fix* names — see "Rename for honesty" below for what they became.

## Incident summary

- Steady HOLD at 30.0°C. Cold water added → `theta_ist` (Smith-corrected)
  dipped to ~29.49°C (`diff ≈ 0.51°C`).
- `diff` never reached `Thresholds.HoldHeat` (1.0°C, `temp_controller_fsm.py`),
  so **the FSM never left `HOLD`** — the entire episode happened inside what
  the system still considered "holding".
- The outer loop (`pid_hold`, `Hold: {kp:0.6, ki:0, kd:0, kt:0}` — pure
  proportional) tracked `0.6 * max(0, diff)` exactly, tick for tick, peaking
  at `y ≈ 0.31` — nowhere near its `y_max=1.0` ceiling. Confirmed directly
  against logged samples: `rate_soll` == `0.6*diff` to the last digit
  throughout.
- The inner loop (`pid_heat`, `Heat: {kp:0.08, ki:0.02, kd:0, kt:1.5}`) then
  chased that `heatrate_soll` target, driving commanded power from a
  baseline ~500W burst-cycle up to **~2600W** (`y ≈ 0.74` of the 3500W max,
  see `tasks/heater.py:58-59`: `power = get_power_max() * y`).
- Plant dead time (`Td=17s`) and thermal mass (`M=27.96 kg`,
  `C≈3403 J/(kg·K)`, from `PlantParams` in the log) meant heat from that
  burst kept arriving after `theta_ist` had already crossed back over soll.
  By the time `rate_soll` had fallen back to exactly `0` (outer loop asking
  for zero heat rate), `power_set` was still **1442 W**, decaying only
  gradually over the next ~35s (1193 → 979 → 817 → 632 → 487 → 337 → 142 W)
  rather than snapping to 0.
- No active cooling actuator exists; the passive loss coefficient
  (`L=0.2 W/(kg·K)`) gives a time constant `C/L ≈ 17,000s (~4.7h)`, so the
  resulting ~0.4°C plateau above soll does not visibly decay within any
  reasonable observation window.

**Root cause:** the inner loop's integral term (`yi`) accumulated during
the ~130s the outer loop demanded a real (if modest) heat rate, and nothing
bounded that accumulation — the existing anti-windup in `Pid.process()`
(`components/pid/pid.py:45-48`) is back-calculation that only corrects `yi`
when the combined output `y` is actually clamped at `y_min`/`y_max`. Since
`y` peaked at ~0.74, well under the `1.0` ceiling, `awu` was `0` the entire
time and the anti-windup mechanism never engaged. The integrator just had to
unwind naturally against real (delayed) negative error, which took ~35+
seconds after the outer loop had already zeroed its target.

## Rejected approach: a flat `yi_max` clamp on the inner loop

The first fix considered was bounding `yi` directly, with one constant for
the inner loop regardless of state. Numeric check against the real plant
model kills this as a standalone fix:

| scenario | P needed | y needed |
|---|---|---|
| 1.5 K/min ramp @ 30°C | 2435 W | 0.696 |
| 1.5 K/min ramp @ 50°C | 2547 W | 0.728 |
| 1.5 K/min ramp @ 66°C | 2636 W | 0.753 |
| the actual HOLD disturbance (peak) | 2600 W | 0.74 |
| steady HOLD loss compensation @ 66°C | ~257 W | ~0.073 |

At steady state (`err=0`), `kp*err` contributes nothing — the entire
`y≈0.7-0.75` needed to sustain a genuine 1.5 K/min ramp has to come from
`yi` alone, for as long as the ramp lasts. That's the *same* range the
disturbance transient itself peaked at, but far above what real steady-HOLD
loss compensation ever needs (~0.07-0.15 at realistic brew temperatures). A
single constant can't serve both: tight enough to matter for the disturbance
(well under ~0.74) permanently starves a real ramp of the ~0.7 it needs;
loose enough not to interfere with ramps (~0.75+) never engages during the
disturbance at all. **Rejected as a single global constant.**

An FSM-gating alternative (freeze the inner loop's output to 0 while in
`HOLD` unless explicitly engaged by command or a new threshold) was also
sketched, but left an open question about how a new engage threshold should
relate to the existing `HoldHeat` FSM threshold, and needed extra
engage/disengage hysteresis to avoid chatter. **Superseded** by the plan
below, which reaches the same effect with less new machinery.

## Refined plan: split the clamp by *which state is driving the same loop*,
## not by freezing the loop

The insight above — real HOLD-time demand (~0.07-0.15) and real ramp demand
(~0.7-0.75) occupy clearly different ranges — means a **per-state parameter
set** on the *same* PID instance solves this more cleanly than gating the
loop on/off: same accumulated `yi`/`d` state carried across `HOLD↔HEAT`
transitions (bumpless transfer for free, no explicit reset, no engage/
disengage hysteresis to tune), just a different `yi_max` ceiling depending
on which state is currently active.

### Rename for honesty

The pre-fix names didn't match what actually ran when:
- `pid_hold` already ran unconditionally *every* tick regardless of state
  (`process_pid()`'s first line, `temp_controller_base.py:137`) — it's the
  outer loop, not "the HOLD-state PID". Renamed to **`pid_outer`**.
- `pid_heat` already ran in *both* `HOLD` and `HEAT` (only `IDLE`/`COOL`
  skip it, `temp_controller_base.py:154-163`) — it's the inner loop for
  the heating direction. Renamed to **`pid_inner`**.
- `pid_cool` only ever ran in `COOL` — no `HOLD`-time ambiguity, since a
  disturbance that pushes temp *above* soll during `HOLD` is still handled
  by `pid_inner` (the outer loop's `max(0.0, pid_outer_y)` floor sends
  `heatrate_soll` to 0, and `pid_inner` reacts to the resulting negative
  `heatrate_err` — the state only escalates to real `COOL` past
  `Thresholds.HoldCool`). Renamed to **`pid_inner_cool`** for symmetry, kept
  as its own instance — the explicit `reset()` calls when crossing
  between heat-direction and cool-direction states
  (`temp_controller_fsm.py:104,108,115,123`) stay exactly as they were;
  there is no reason to share integrator state across a heater/chiller
  boundary.

### Config: `Hold`/`Heat`/`Cool` → `Outer` / `Inner.{Heat,Hold,Cool}`

```json
"TempCtrl": {
  "pid_type": "Smith",
  "beta": 0.9,
  "Outer": { "kp": 0.6, "ki": 0.0, "kd": 0.0, "kt": 0.0 },
  "Inner": {
    "Heat": { "kp": 0.08, "ki": 0.02, "kd": 0.0, "kt": 1.5 },
    "Hold": { "kp": 0.08, "ki": 0.02, "kd": 0.0, "kt": 1.5, "yi_max": 0.3 },
    "Cool": { "kp": 0.08, "ki": 0.02, "kd": 0.0, "kt": 1.5 }
  },
  "Thresholds": { "...": "unchanged" }
}
```
- `Inner.Heat` keeps today's `Heat` gains, no `yi_max` (or a very loose one)
  — a real ramp must be able to reach `y≈0.75`.
- `Inner.Hold` starts as a copy of the same gains, with `yi_max≈0.2-0.3`
  added — comfortably above realistic steady-loss compensation (~0.07-0.15)
  but well below what turned a 0.5°C dip into a 2600W burst.
- `Inner.Cool` is `Cool`'s existing gains, moved under `Inner` purely for
  structural consistency — introduced now, not because we've observed a
  cooling-side incident. `pid_inner_cool` never runs during `HOLD`, so it
  doesn't need its own `Hold` variant the way `Heat` does; one params block
  is enough.
- **This was a breaking config change** — no backward-compat shim for the
  old flat `Hold`/`Heat`/`Cool` keys (per the "no compat hacks" convention).
  Every deployed `config.json` needed migrating, not just the repo's
  `config-real.json.tpl`/`config-sim.json.tpl` templates.

### Code changes (implemented)

1. **`components/pid/pid.py`** — the symmetric `yi_max` clamp lives inside
   `process()` (line 40): `self.yi = max(-yi_max, min(yi_max, self.yi))`
   when `self.params.get('yi_max')` is set, applied right after
   accumulating `yi` and before it's summed into `y`.
2. **`components/pid/temp_controller_fsm.py`** — `self.pid_hold` →
   `self.pid_outer`, `self.pid_heat` → `self.pid_inner`, `self.pid_cool` →
   `self.pid_inner_cool` (constructor at lines 33/34/40, all `reset()` call
   sites and comments at lines 11-12, 83, 87-89, 99, 104, 108, 111, 115,
   119, 123).
3. **`components/pid/temp_controller_base.py`**:
   - `set_params()` (lines 30-37): `self.pid_outer.set_params(params['Outer'])`;
     stores `self._inner_heat_params = params['Inner']['Heat']` and
     `self._inner_hold_params = params['Inner']['Hold']` for the per-tick
     lookup below; `self.pid_inner_cool.set_params(params['Inner']['Cool'])`.
   - `process_pid()` (lines 136-163): `pid_hold_y` → `pid_outer_y`, and the
     `self.pid_hold.process(...)` call. In the combined `HOLD`/`HEAT`
     branch (lines 159-163), the active param set is selected before
     processing:
     ```python
     else:
         inner_params = self._inner_heat_params if self.state == States.HEAT else self._inner_hold_params
         self.pid_inner.set_params(inner_params)
         self.pid_inner.process(heatrate_err, -self.heatrate_ist)
         self.y = self.pid_inner.get_y()
     ```
     `set_params()` is a cheap dict-reference assignment (`pid.py:22-23`),
     so calling it every tick has no meaningful cost. Because `kp`/`ki`/`kd`/
     `kt` are identical between `Inner.Heat` and `Inner.Hold` in the
     shipped config, switching the active set at a `HOLD↔HEAT` transition
     changes no term of `y` at that instant — only the `yi_max` ceiling
     going forward, with one caveat noted in Status below.
4. **Config files** — `config.json`, `config-real.json.tpl`,
   `config-sim.json.tpl` restructured into `Outer`/`Inner.{Heat,Hold,Cool}`
   as above. The inline `"Cool": {...}` dicts in `scripts/demos/pid/
   demo_temp_controller_smith.py`, `demo_temp_controller.py`, and
   `scripts/demos/sud/demo_sud.py` got the same restructure.
5. **`utils/replay_sim.py`** — `_apply_gain_overrides()` and the CLI flag
   loop now iterate the shared `GAIN_SECTIONS = (('Outer','outer'),
   ('Inner.Heat','inner-heat'), ('Inner.Hold','inner-hold'),
   ('Inner.Cool','inner-cool'))`, with nested dict access for the
   `Inner.*` entries. The params print loop and
   `_infer_heatrate_soll_set()`'s docstring were updated to match
   (`pid_outer.get_y()` instead of "the hold PID").
6. **`components/pid/TODO.md`** — the windup entry is marked `[x]` and
   points at this section.

## Tests

Stdlib `unittest`, no pytest — `tests/components/pid/test_pid.py` and
`test_temp_controller_closed_loop.py`, discoverable via
`python -m unittest discover -t . -s tests/components/pid` (or `-s tests`
for the whole repo suite). Simpler than the FSM-gating plan's test plan
would have needed: no engage/disengage hysteresis or threshold-relationship
behavior to cover, since the loop is never turned off — only its `yi_max`
ceiling changes with state.

**A. Unit-level, isolated `Pid`** (`test_pid.py`) — no plant involved. Feeds
a synthetic `err` sequence shaped like the incident (positive
`heatrate_err = 0.3` held for 130 ticks, matching the outer loop's real
demand during the disturbance, then a flat `-0.3` tail for 200 ticks,
matching the real `rate_soll - rate_ist` gap once the outer loop had
zeroed its target) into two `Pid` instances with identical gains, one with
`yi_max` set (the `Inner.Hold` case) and one without (`Inner.Heat`).
Asserts: recovery time (ticks after the error goes negative until `y`
drops back under a small threshold) is measurably shorter when clamped;
`yi` never exceeds the configured bound; the unclamped instance's `yi`
does exceed it (sanity-checks the test itself isn't vacuous).

**B. Closed-loop** (`test_temp_controller_closed_loop.py`) — real `Pot(dt)`
plant with `M=27.96, C=3403.43, L=0.2, Td=17`, ambient `20`°C, driven by a
real `TempController(Smith)` with the shipped `Outer`/`Inner.*` gains.
Controller's own `y` feeds back into the plant each tick (unlike
`utils/replay_sim.py`'s open-loop observe-only mode):
- **Disturbance case**: hold at 30°C until settled, knock `plant.temp` down
  0.5°C to emulate the cold-water event, keep ticking for 600 more ticks —
  confirms the state stays in `HOLD` throughout, matching the incident.
  Asserts peak overshoot above 30.0°C is measurably smaller with
  `Inner.Hold`'s `yi_max` set than with an unclamped copy of `Inner.Heat`.
- **Ramp case**: commands a genuine 1.5 K/min ramp to 40°C (state reaches
  `HEAT`) and asserts the sustained heat rate actually exceeds 1.4 K/min —
  guards against reintroducing the flat-clamp regression from the rejected
  approach above.
- **Transition case**: drives a `HOLD→HEAT→HOLD` sequence and asserts the
  `y` step at either transition stays under `0.1` — see the retroactive-clamp
  caveat in Status below for why this isn't a stricter "no discontinuity"
  assertion.

## Status

Implemented: `pid.py`'s `yi_max` clamp, the `pid_outer`/`pid_inner`/
`pid_inner_cool` rename, the `Outer`/`Inner.{Heat,Hold,Cool}` config
restructure (`config.json`, both `.tpl` templates, and the three demo
scripts), and `utils/replay_sim.py`'s matching CLI-flag/print-loop rename.
Tests added under `tests/components/pid/` (`test_pid.py` for the isolated
`Pid` clamp behavior, `test_temp_controller_closed_loop.py` for the
closed-loop disturbance/ramp/transition cases) — all passing.

One subtlety found while writing the closed-loop transition test that
this plan didn't anticipate: `Inner.Hold`'s `yi_max` clamp applies
retroactively. If a sustained `HEAT` ramp pushes `yi` above the `Hold`
ceiling before the `HeatHold` threshold fires, the very next tick after
the `HEAT→HOLD` transition clamps `yi` back down immediately, producing a
small (~0.07 in testing, well below the pre-fix disturbance's ~0.74 peak)
step in `y` rather than the fully bumpless transfer described above. Not
addressed here — flagged for awareness, not a blocker.

## Follow-up: steady-state overshoot in HOLD (`Outer.y_hold_min`)

Found while testing the rework against a real Sud run (`sude/sud_0030.json`,
`logs/log_20260706T074658_Sud-0030.json`). Visible in `docs/overshoot2.png`
(`theta_ist` vs `theta_soll` across the run): at both the 55°C and 63°C
rests, `theta_ist` overshoots the step and then plateaus above `theta_soll`
for the rest of the hold instead of converging back down — most clearly at
the first rest, where it settles at ~55.5-55.6°C against a 55.0°C target.

A grain-fill-in disturbance during "1. Rast" (mash-in rest, `HOLD` at 55°C)
pushed `temp_ist` to a ~0.4°C overshoot — well under `HoldCool`'s 1.0
threshold, so the FSM stayed in `HOLD` throughout, same as the transient
windup case above. But this overshoot never decayed: `temp_ist` sat in a
55.30-55.44 band for the rest of the 20-minute hold instead of converging
back to 55.0. Distinct failure mode from the transient windup fixed above
(that one unwound over ~35s; this one was flat/permanent for as long as the
hold lasted).

Root cause: `process_pid()`'s HOLD-state floor on `pid_outer_y` (added in
`bb5af3c` to break a limit cycle - see the History section) clamped to
exactly `0.0`. Once `temp_ist > temp_soll`, that floor forces
`heatrate_soll = 0`, i.e. "hold flat" - and `pid_inner` then actively fights
the pot's own ambient heat loss to keep the *overshot* temperature flat,
rather than being allowed to request a genuine decline back toward
setpoint. With `Outer.ki = 0`, there's no integral action to null the
resulting steady-state error any other way, so the offset persists for the
rest of the hold.

Fix: replaced the hardcoded `0.0` floor with a configurable
`Outer.y_hold_min` (default `0.0`, so configs that don't set it keep the old
behavior), set to `-0.1` in `config.json`, both `.tpl` templates, and the
three demo scripts. A small negative floor lets HOLD ask for a gentle
decline that roughly matches
passive ambient cooling, rather than the fully unclamped `[-1, 1]` range
that caused the original limit cycle (a large negative `heatrate_soll` asks
for a decline steeper than passive loss can deliver, pinning power at 0 for
an extended stretch and producing a hard undershoot/rebound). Test coverage
added: `TestHoldOvershootRecoversToSetpoint` in
`tests/components/pid/test_temp_controller_closed_loop.py` reproduces the
sud_0030 disturbance, asserts the old flat-clamp behavior still fails to
recover (regression guard) and the new floor converges close to setpoint,
plus a guard that the recovery doesn't undershoot by more than the injected
disturbance itself (i.e. doesn't reintroduce the `bb5af3c` limit cycle).

`-0.1` was chosen from the real Sud's plant params (`Pot.mass=5.96` +
`water_mass=22` ≈ the test harness's `M=27.96`, `L=0.2`): passive ambient
loss at a ~35°C delta works out to roughly 0.1-0.15 K/min, so
`heatrate_soll_set * -0.1` lands in that same ballpark for a typical
`heatrate_soll_set` of ~1.0-1.5 K/min. Not derived from first-principles
tuning - may need adjustment per installation, same as the other PID gains.

Confirmed against a live re-run of `sud_0030` with the fix applied, not just
the unit test above.

## Architecture diagram

A full signal-flow diagram of the cascade (`Outer`/`Inner.*` PIDs, FSM
state gating, Smith-predictor feedback) exists as a Claude Artifact:
https://claude.ai/code/artifact/a32e4752-b4a6-4153-b344-eb2423eb6512 — this
is a session-scoped link, not a durable one, so it may not resolve for
everyone with repo access. `docs/fsm_states.png` is a static screenshot of
just the diagram's FSM-states panel, checked in as the durable copy.