JaySynth/tools/testhost/TODO.md

# testhost TODO

Tracks known gaps and future work for `tools/testhost/`, the generic headless
plugin test host (see `README.md` for what it does and how to use it, and
`/home/jens/.claude/plans/sharded-finding-tower.md` for the original design
plan). Nothing here blocks current use — Phases 1-4 of the plan are done and
verified (PR https://git.jayfield.org/jens/JaySynth/pulls/1).

## Deferred by design (Phase 5 of the plan)

- [ ] **LV2 plugin format support.** JUCE 3.1.1 (the version currently
  vendored at `sdk/juce/JUCE-3.1.1`) has no LV2 hosting backend. Once
  JaySynth's own JUCE dependency is upgraded (or a custom `AudioPluginFormat`
  subclass is written against the current version), register it in
  `PluginHost`'s constructor alongside `VSTPluginFormat` - no other host code
  should need to change, since everything downstream talks to the
  format-agnostic `AudioPluginInstance`/`AudioProcessor` API already.
- [ ] **VST3/LADSPA format support.** JUCE already ships
  `VST3PluginFormat`/`LADSPAPluginFormat` in this same vendored version -
  registering them is a one-line addition to `PluginHost`'s constructor,
  just not done yet since nothing has needed it.
- [ ] **Windows build (`Makefile.win`).** The plan calls for mirroring
  `src/plug`'s Linux/Windows Makefile pattern; only the Linux side has been
  built and tested so far.

## Test coverage gaps

- [ ] **Only mono (1 in/1 out) and asymmetric (2 in/1 out) effect channel
  configs have been smoke-tested** (`ZamDelay-vst.so`, `ZamComp-vst.so`,
  `ZamEQ2-vst.so`). A genuinely stereo-in/stereo-out effect hasn't been run
  through yet - worth doing before relying on this tool for a stereo effect
  plugin specifically.
- [ ] **No automated regression baseline comparison.** The plan's stretch
  goal: batch-render every `.fxp`/`.xmp` under `extras/sounds/` through a
  fixed note pattern and diff against stored baseline WAVs/checksums, to
  catch DSP or save-load regressions automatically instead of via one-off
  manual runs. Not built.
- [ ] **No CI wiring.** Nothing currently runs `testhost` automatically on
  push/PR; it's a manual, opt-in tool for now.
- [ ] **No unit tests for `TestScenario`'s JSON parsing itself** (malformed
  JSON, missing required fields, wrong types, out-of-range values) - only
  exercised indirectly via a couple of hand-written scenario files during
  development. Edge-case handling (e.g. a negative `sample`, an unterminated
  `noteOn` with no matching `noteOff`) is untested.
- [ ] **Polyphony/multiple simultaneous notes untested** - only ever a single
  held note in all smoke tests so far.
- [ ] **`tests/run_tests.sh` never asserts specific expected values for
  `PluginHost::load()`/`getNumInputChannels()`/`getNumOutputChannels()`/
  `getNumParameters()`** - every test exercises them (they're printed in
  the load banner), but nothing pins e.g. "JaySynth must report exactly
  0 in / 2 out / 140 parameters," so a regression there would only be
  caught if it also broke something else. Same underlying gap as "no
  automated regression baseline comparison" above, just called out
  separately since it's about the load-time metadata rather than the
  rendered audio. See `tests/README.md`'s `PluginHost` coverage table.

## Investigate

- [ ] **Bundled sample `.fxp` files under `extras/sounds/` silently have no
  effect when loaded via `--patch`** (confirmed by diffing `--printParams`
  output with and without `--patch` - zero parameters differ), even though
  loading doesn't crash. Resolved: this is *not* a host/plugin capability
  mismatch - `usesChunks()` is genuinely true and `test_chunk_roundtrip` in
  `tests/run_tests.sh` proves the real `bankEncodeXml`/`bankDecodeXml` chunk
  path round-trips correctly for files this build produces (see
  `tests/README.md`'s "corrected finding" note for the full disproof). What
  remains open is narrower: why these specific (10+ year old) *external*
  sample files fail to apply - most likely a compression or XML-schema
  shape that `JaySynthAudioProcessor::setCurrentProgramStateInformation`/
  `getXmlFromBinary` no longer parses. Not one of the tracked repo-root
  `TODO.md` findings; worth a closer look at some point (hex-diff one of
  these files' decompressed XML against what this build produces), but not
  chased further here.
- [ ] **DPF-based plugins print a non-fatal assertion on load.** All three
  Zam*-vst.so plugins tested (built with the DISTRHO Plugin Framework) print
  `assertion failure: "fIsActive" in .../DistrhoPluginInternal.hpp` during
  `PluginHost::load()`, despite JUCE's `VSTPluginInstance::prepareToPlay`
  already sending `effMainsChanged`/`effStartProcess`. Rendering still
  completes correctly (confirmed via WAV output), so this looks like a
  DPF/JUCE VST2 lifecycle-ordering quirk rather than a testhost bug, but it
  hasn't been root-caused - worth a closer look if it turns out to affect
  correctness (not just log noise) for some other plugin.
- [ ] **`WavAudioFormat`'s base-class doc comment says channel count "must
  be either 1 or 2"** (`AudioFormat::createWriterFor`'s Doxygen comment).
  `WavRecorder` passes through whatever `PluginHost::getNumOutputChannels()`
  reports with no clamping, and this has worked fine for the 1- and 2-channel
  plugins tested so far - but a plugin with more than 2 output channels
  hasn't been tried, so it's unconfirmed whether JUCE's WAV writer actually
  enforces that limit or the doc comment is just generic/stale boilerplate
  shared across format subclasses.

## Nice-to-have (not planned, just noted)

- [x] A `--list-params` or similar introspection mode (print every
  parameter's index/name/current value) would help writing new scenario
  JSON files without cross-referencing the plugin's own GUI or source.
  Done: `--printParams` (prints `PARAM <index> <value>` for every
  parameter, after any initial `--patch`/`--bank` load) - added to make
  the patch/bank round-trip regression tests in `tests/` possible; only
  prints values, not names, so cross-referencing the plugin's own
  parameter-name list is still occasionally needed.