jensandClaude Sonnet 5
f54689f071
tools/testhost/tests/run_tests.sh dynamically exercises the tracked
hardening findings using testhost itself - no GUI, sub-second wall
time:
- oversized_blocksize (Critical #5): --blockSize 16384 under a
timeout, catching either the overflow or the deadlock the original
bug could cause.
- nrpn_out_of_range/nrpn_in_range (Critical #1): JSON scenarios
sending the 5-message MIDI NRPN CC sequence, once targeting the
max 14-bit ID (16383) and once a legitimate in-range ID (500), so
a regression that made the bounds check too aggressive would also
show up.
- patch_save_load_roundtrip / bank_save_load_roundtrip: save a
distinctive continuous parameter value, reload in a fresh process,
check it round-trips within tolerance - covers the
setCurrentProgramStateInformation no-op bug and the
patchDecodeXml/patchImportXml fixes.
- stability_sweep: loads every bundled .fxp under extras/sounds/ and
renders a held note through each, as a broad crash/NaN net.
Added --printParams to main.cpp (prints every parameter's index and
value) to make the round-trip tests possible - this was also already
a noted nice-to-have in testhost/TODO.md.
While building the oversized_blocksize test, found and fixed (on the
hardening branch, commit d713ca5) a second, previously-unknown bug:
VCF_CalcCoeff_LPF/_HPF/_BPF advance the coefficient buffer pointer
once per (sample, filter-section) pair, but the buffer was only ever
allocated for bufsize samples, not bufsize*sections - a 4th-order
filter overflowed it for any block between 4097 and 8192 samples.
Found via bisecting the crashing block size then confirming with an
ASan build; see tests/README.md's debugging-notes section for exactly
how, including the ASan symbolizer hang encountered along the way and
the workaround.
Also discovered (documented in tools/testhost/TODO.md's Investigate
section, not fixed - out of scope): JaySynth's compiled VST2 wrapper
never advertises chunk support to the host, so real .fxp/.fxb sample
files have no effect when loaded through the generic AudioProcessor
state API. The round-trip tests work around this by using testhost's
own save output as the fixture rather than an external sample file.
All 10 checks pass against the current hardening-branch build.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011dhtwRLARk4eiPngcQykLJ
155 lines
7.1 KiB
Markdown
155 lines
7.1 KiB
Markdown
# testhost
|
|
|
|
A generic, headless command-line host for testing audio plugins. It loads a
|
|
plugin out-of-process and drives it entirely offline — no audio device, no
|
|
GUI event loop, no real-time pacing — so a test run is bounded by CPU speed
|
|
rather than wall-clock playback time (60-1000x+ real-time observed on this
|
|
machine, depending on the plugin).
|
|
|
|
**This tool is not JaySynth-specific.** It was built to test JaySynth, but it
|
|
talks to plugins purely through JUCE's own `AudioPluginFormatManager` /
|
|
`AudioPluginInstance` hosting API, and it never assumes a fixed channel
|
|
layout — it always asks the loaded plugin how many input/output channels it
|
|
actually has. That means it drives an instrument (0 inputs, MIDI-triggered,
|
|
e.g. JaySynth) and an audio effect (real input required, e.g. a reverb,
|
|
EQ, or compressor) through the exact same code path. See
|
|
`/home/jens/.claude/plans/sharded-finding-tower.md` for the original design
|
|
plan and rationale.
|
|
|
|
## Building
|
|
|
|
```sh
|
|
MAKE_HOME=$(realpath ../../submodule/make) make -C tools/testhost
|
|
```
|
|
|
|
This produces `tools/testhost/build/linux/release/testhost`. It has its own
|
|
`JuceLibraryCode/` package (compiling only the module amalgams a host needs -
|
|
not `juce_audio_plugin_client`, which is the plugin-*side* VST entry point
|
|
and would collide with this executable's own `main()`), but reuses the
|
|
already-vendored `sdk/juce/JUCE-3.1.1` rather than a second JUCE checkout.
|
|
The host and any plugin it loads are decoupled at the VST2 ABI, not the JUCE
|
|
source level, so this doesn't need to track JaySynth's own JUCE version.
|
|
|
|
`make CONFIG=debug -C tools/testhost` builds a debug variant the same way
|
|
the main plugin build does.
|
|
|
|
## Quick start
|
|
|
|
```sh
|
|
# Render 2s of a held note through JaySynth and record it:
|
|
./build/linux/release/testhost \
|
|
--plugin ../../build/linux/release/JaySynth.so \
|
|
--note 60 100 --duration 2 --wav out.wav
|
|
|
|
# Feed pink noise through an audio effect (no MIDI needed):
|
|
./build/linux/release/testhost \
|
|
--plugin /usr/lib/vst/ZamDelay-vst.so \
|
|
--input pink --duration 2 --wav out.wav
|
|
```
|
|
|
|
Run `./build/linux/release/testhost --help` for the full flag reference
|
|
(patch/bank load & save, parameters, sample rate/block size, performance CSV
|
|
export, etc).
|
|
|
|
## JSON test scenarios
|
|
|
|
For anything beyond a one-off check, `--scenario file.json` drives a full
|
|
timed sequence of actions instead of a handful of CLI flags - this is what
|
|
makes tests repeatable and batchable (e.g. one file per regression case, run
|
|
unattended in CI).
|
|
|
|
```json
|
|
{
|
|
"sampleRate": 44100,
|
|
"blockSize": 512,
|
|
"durationSeconds": 2.0,
|
|
"input": { "type": "noise", "kind": "pink", "amplitude": 0.5 },
|
|
"loadBank": "some_bank.fxb",
|
|
"events": [
|
|
{ "sample": 0, "type": "noteOn", "channel": 1, "note": 60, "velocity": 100 },
|
|
{ "sample": 22050, "type": "param", "index": 5, "value": 0.75 },
|
|
{ "sample": 44100, "type": "noteOff", "channel": 1, "note": 60 },
|
|
{ "sample": 88199, "type": "savePatch", "file": "result_patch.bin" }
|
|
],
|
|
"recordWav": "out.wav"
|
|
}
|
|
```
|
|
|
|
Relative file paths inside the JSON (`loadBank`, `loadPatch`, `recordWav`,
|
|
and any event's `file`) resolve against **the scenario file's own
|
|
directory**, not the process's working directory, so a scenario file stays
|
|
portable no matter where it's run from.
|
|
|
|
### Top-level fields
|
|
|
|
| Field | Type | Notes |
|
|
|---|---|---|
|
|
| `sampleRate` | number | default 44100 |
|
|
| `blockSize` | number | default 512 |
|
|
| `durationSamples` **or** `durationSeconds` | number | one of these is required |
|
|
| `input` | object | `{"type": "silence"}` (default) / `"sine"` (+ `frequencyHz`) / `"noise"` (+ `kind`: `"white"`/`"pink"`) / `"impulse"`, all with optional `amplitude` (default 0.5) |
|
|
| `loadPatch` / `loadBank` | string (file path) | loaded once, before rendering starts |
|
|
| `recordWav` | string (file path) | omit to skip recording (e.g. pure performance runs) |
|
|
| `events` | array | see below; need not be pre-sorted, `TestScenario` sorts by `sample` |
|
|
|
|
### Event types
|
|
|
|
All events have `sample` (absolute sample index) and `type`:
|
|
|
|
| `type` | Extra fields | Effect |
|
|
|---|---|---|
|
|
| `noteOn` | `channel`, `note`, `velocity` | `MidiMessage::noteOn`, timed to the exact sample within its block |
|
|
| `noteOff` | `channel`, `note` | `MidiMessage::noteOff`, same timing precision |
|
|
| `controller` | `channel`, `controllerNumber`, `controllerValue` | `MidiMessage::controllerEvent` |
|
|
| `param` | `index`, `value` (0.0-1.0) | `AudioProcessor::setParameter`, applied at the start of the block containing `sample` (see caveat below) |
|
|
| `loadPatch` / `loadBank` | `file` | reads the file's raw bytes and calls `setCurrentProgramStateInformation`/`setStateInformation` |
|
|
| `savePatch` / `saveBank` | `file` | calls `getCurrentProgramStateInformation`/`getStateInformation` and writes the raw bytes out |
|
|
|
|
**Caveat:** MIDI events keep full per-sample timing (JUCE's `MidiBuffer`
|
|
carries a sample offset natively). Non-MIDI events (`param`/`loadPatch`/
|
|
`loadBank`/`savePatch`/`saveBank`) scheduled for a sample in the middle of a
|
|
block are applied at the *start* of that block, not their exact offset -
|
|
fine for test purposes, but not sample-accurate. Use a smaller `blockSize`
|
|
if you need tighter timing on those.
|
|
|
|
### Patch/bank format note
|
|
|
|
`loadPatch`/`savePatch`/`loadBank`/`saveBank` read/write exactly the bytes
|
|
that JUCE's host-side `AudioPluginInstance::get/setCurrentProgramState
|
|
Information` and `get/setStateInformation` produce/consume for a hosted
|
|
VST2 plugin - i.e. real `.fxp`/`.fxb` file bytes (JUCE's own VST2 host
|
|
wrapper handles the fxb/fxp chunk header itself; this tool never parses
|
|
that format directly). This is a different code path from JaySynth's own
|
|
GUI-triggered "Load Patch from disk" button (`JaySynthAudioProcessor::
|
|
loadPatchFromFile`/`patchImportXml` in `src/plug/PluginProcessor.cpp`) -
|
|
both are legitimate ways to get a patch into the plugin, but they aren't
|
|
the same call chain, so a fix to one doesn't automatically test the other.
|
|
|
|
## Architecture
|
|
|
|
- `PluginHost` — owns the `AudioPluginFormatManager`, loads/prepares/drives
|
|
one plugin instance. The only class that talks to `AudioPluginInstance`
|
|
directly; everything else in this tool is format-agnostic by construction.
|
|
- `AudioInputSource` — fills the plugin's input channels each block
|
|
(silence/sine/noise/impulse).
|
|
- `WavRecorder` — thin wrapper over `WavAudioFormat`/`AudioFormatWriter`.
|
|
- `TestScenario` — parses the JSON format above into a sorted event list
|
|
plus render/I-O config.
|
|
- `PerformanceStats` — per-block timing, summary + optional CSV.
|
|
- `main.cpp` — CLI parsing, and a single render loop shared by both the
|
|
ad-hoc-flags path and the `--scenario` path (ad-hoc flags are converted
|
|
into an equivalent in-memory `TestScenario` before rendering starts, so
|
|
there's only one loop to maintain).
|
|
|
|
## Regression suite
|
|
|
|
`tests/run_tests.sh` uses this tool to dynamically test the findings
|
|
tracked in the repo-root `TODO.md` - see `tests/README.md` for what each
|
|
test covers (and its limits). It's also how a real, previously-unknown VCF
|
|
buffer-sizing bug got found and fixed (see that file and the repo-root
|
|
`TODO.md`'s Critical section).
|
|
|
|
## Known limitations
|
|
|
|
See `TODO.md` in this directory.
|