jens/JaySynth
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
Files
2a0ea726f3b8d7e2ffc454f13b224a78e2f81c85
JaySynth/tools/testhost/TODO.md
T
jensandClaude Sonnet 5 f54689f071 Add regression suite testing JaySynth's TODO.md findings via testhost 
tools/testhost/tests/run_tests.sh dynamically exercises the tracked
hardening findings using testhost itself - no GUI, sub-second wall
time:

- oversized_blocksize (Critical #5): --blockSize 16384 under a
  timeout, catching either the overflow or the deadlock the original
  bug could cause.
- nrpn_out_of_range/nrpn_in_range (Critical #1): JSON scenarios
  sending the 5-message MIDI NRPN CC sequence, once targeting the
  max 14-bit ID (16383) and once a legitimate in-range ID (500), so
  a regression that made the bounds check too aggressive would also
  show up.
- patch_save_load_roundtrip / bank_save_load_roundtrip: save a
  distinctive continuous parameter value, reload in a fresh process,
  check it round-trips within tolerance - covers the
  setCurrentProgramStateInformation no-op bug and the
  patchDecodeXml/patchImportXml fixes.
- stability_sweep: loads every bundled .fxp under extras/sounds/ and
  renders a held note through each, as a broad crash/NaN net.

Added --printParams to main.cpp (prints every parameter's index and
value) to make the round-trip tests possible - this was also already
a noted nice-to-have in testhost/TODO.md.

While building the oversized_blocksize test, found and fixed (on the
hardening branch, commit d713ca5) a second, previously-unknown bug:
VCF_CalcCoeff_LPF/_HPF/_BPF advance the coefficient buffer pointer
once per (sample, filter-section) pair, but the buffer was only ever
allocated for bufsize samples, not bufsize*sections - a 4th-order
filter overflowed it for any block between 4097 and 8192 samples.
Found via bisecting the crashing block size then confirming with an
ASan build; see tests/README.md's debugging-notes section for exactly
how, including the ASan symbolizer hang encountered along the way and
the workaround.

Also discovered (documented in tools/testhost/TODO.md's Investigate
section, not fixed - out of scope): JaySynth's compiled VST2 wrapper
never advertises chunk support to the host, so real .fxp/.fxb sample
files have no effect when loaded through the generic AudioProcessor
state API. The round-trip tests work around this by using testhost's
own save output as the fixture rather than an external sample file.

All 10 checks pass against the current hardening-branch build.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011dhtwRLARk4eiPngcQykLJ
2026-07-27 20:06:44 +02:00

5.4 KiB
Raw Blame History

testhost TODO

Tracks known gaps and future work for tools/testhost/, the generic headless plugin test host (see README.md for what it does and how to use it, and /home/jens/.claude/plans/sharded-finding-tower.md for the original design plan). Nothing here blocks current use — Phases 1-4 of the plan are done and verified (PR #1).

Deferred by design (Phase 5 of the plan)

  • LV2 plugin format support. JUCE 3.1.1 (the version currently vendored at sdk/juce/JUCE-3.1.1) has no LV2 hosting backend. Once JaySynth's own JUCE dependency is upgraded (or a custom AudioPluginFormat subclass is written against the current version), register it in PluginHost's constructor alongside VSTPluginFormat - no other host code should need to change, since everything downstream talks to the format-agnostic AudioPluginInstance/AudioProcessor API already.
  • VST3/LADSPA format support. JUCE already ships VST3PluginFormat/LADSPAPluginFormat in this same vendored version - registering them is a one-line addition to PluginHost's constructor, just not done yet since nothing has needed it.
  • Windows build (Makefile.win). The plan calls for mirroring src/plug's Linux/Windows Makefile pattern; only the Linux side has been built and tested so far.

Test coverage gaps

  • Only mono (1 in/1 out) and asymmetric (2 in/1 out) effect channel configs have been smoke-tested (ZamDelay-vst.so, ZamComp-vst.so, ZamEQ2-vst.so). A genuinely stereo-in/stereo-out effect hasn't been run through yet - worth doing before relying on this tool for a stereo effect plugin specifically.
  • No automated regression baseline comparison. The plan's stretch goal: batch-render every .fxp/.xmp under extras/sounds/ through a fixed note pattern and diff against stored baseline WAVs/checksums, to catch DSP or save-load regressions automatically instead of via one-off manual runs. Not built.
  • No CI wiring. Nothing currently runs testhost automatically on push/PR; it's a manual, opt-in tool for now.
  • No unit tests for TestScenario's JSON parsing itself (malformed JSON, missing required fields, wrong types, out-of-range values) - only exercised indirectly via a couple of hand-written scenario files during development. Edge-case handling (e.g. a negative sample, an unterminated noteOn with no matching noteOff) is untested.
  • Polyphony/multiple simultaneous notes untested - only ever a single held note in all smoke tests so far.

Investigate

  • JaySynth's compiled VST2 wrapper never advertises chunk support to the host, so real .fxp/.fxb sample files silently have no effect when loaded generically. Found while building tests/run_tests.sh: loading a bundled sample under extras/sounds/ via --patch produces byte- identical output to not loading anything (confirmed by diffing --printParams). The files are legitimately in VST2's opaque-chunk ("FPCh") format, but JUCE 3.1.1's plugin-side VST wrapper (juce_VST_Wrapper.cpp, compiled into JaySynth.so) never sets effFlagsProgramChunks on the AEffect struct, so JUCE's own host-side VSTPluginInstance::usesChunks() returns false and silently no-ops both load and save. Not one of the tracked TODO.md (repo root) findings, and not attempted here - fixing it means patching JUCE's own VST wrapper, a bigger and riskier change than this pass's scope. See tests/README.md for the full writeup; tests/run_tests.sh's round-trip tests work around it by using testhost's own save output as the fixture instead of an external sample file.
  • DPF-based plugins print a non-fatal assertion on load. All three Zam*-vst.so plugins tested (built with the DISTRHO Plugin Framework) print assertion failure: "fIsActive" in .../DistrhoPluginInternal.hpp during PluginHost::load(), despite JUCE's VSTPluginInstance::prepareToPlay already sending effMainsChanged/effStartProcess. Rendering still completes correctly (confirmed via WAV output), so this looks like a DPF/JUCE VST2 lifecycle-ordering quirk rather than a testhost bug, but it hasn't been root-caused - worth a closer look if it turns out to affect correctness (not just log noise) for some other plugin.
  • WavAudioFormat's base-class doc comment says channel count "must be either 1 or 2" (AudioFormat::createWriterFor's Doxygen comment). WavRecorder passes through whatever PluginHost::getNumOutputChannels() reports with no clamping, and this has worked fine for the 1- and 2-channel plugins tested so far - but a plugin with more than 2 output channels hasn't been tried, so it's unconfirmed whether JUCE's WAV writer actually enforces that limit or the doc comment is just generic/stale boilerplate shared across format subclasses.

Nice-to-have (not planned, just noted)

  • A --list-params or similar introspection mode (print every parameter's index/name/current value) would help writing new scenario JSON files without cross-referencing the plugin's own GUI or source. Done: --printParams (prints PARAM <index> <value> for every parameter, after any initial --patch/--bank load) - added to make the patch/bank round-trip regression tests in tests/ possible; only prints values, not names, so cross-referencing the plugin's own parameter-name list is still occasionally needed.
Reference in New Issue View Git Blame Copy Permalink