Get game audio into the browser via PulseAudio + a WebSocket bridge
VNC/noVNC only ever streams video, so game sound needed a completely separate path. Adds pulseaudio/pulseaudio-utils/libasound2-plugins and routes ALSA's default device through Pulse (/etc/asound.conf), so dosbox/scummvm need zero special config. server.sh starts one PulseAudio daemon and one pcm_ws_bridge.py (from docker-common) for the container's whole lifetime, capturing Pulse's single default sink via parec. Audio is a single shared mix, not per-game - considered and dropped a per-slot-isolated design (mirroring the video architecture) as unnecessary complexity per direction. Every running game's audio just mixes into the one default sink; every /screen/<name> page connects to the same AUDIO_PORT. setup_server.py gains GET /screen/<name> (an iframe onto the game's noVNC screen plus an Enable Sound button - browsers require a user gesture before audio can start) and GET /pcm-worklet.js. The "Open Screen" link simplifies from a client-side-JS-built cross-port link to a plain same-origin relative link, since /screen/<name> now reads the real host server-side from the request's own Host header. Found and fixed a real bug along the way: parec --device=@DEFAULT_SINK@.monitor looks correct but fails with "Stream error: Invalid argument" - the actual PulseAudio macro is the single token @DEFAULT_MONITOR@. Verified end-to-end with real audio, not just plumbing: confirmed via `pactl list sink-inputs` that dosbox connects to Pulse correctly (unmuted, uncorked), then used xdotool to advance stuntcar past its silent title screen and captured real audible game audio (RMS ~9292) through the WebSocket bridge with a raw Python client. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NiNnj78HGx1KWyCCo39HSz
This commit is contained in:
@@ -69,27 +69,32 @@
|
||||
with two different games running concurrently (separate processes, separate displays,
|
||||
separate noVNC ports, independent stop/teardown) and the 10-slot cap logic.
|
||||
|
||||
- Get game sound actually audible during play. `--device /dev/snd` is passed through and
|
||||
`alsa-utils` is installed, so DOSBox can write to an ALSA device inside the container, but
|
||||
VNC/noVNC only ever streams video, not audio — nothing currently carries that sound out to
|
||||
the browser.
|
||||
- ~~Get game sound actually audible during play~~ Done: PulseAudio + a hand-rolled stdlib
|
||||
WebSocket bridge (`pcm_ws_bridge.py`, in `docker-common` — reusable, not dosbox-specific,
|
||||
per the earlier plan for this item) carries raw PCM to the browser, played back via a
|
||||
Web Audio `AudioWorkletNode` (`pcm-worklet.js`, also in `docker-common`). Verified real
|
||||
end-to-end audio: DOSBox → ALSA (default device, routed through Pulse via
|
||||
`/etc/asound.conf`) → PulseAudio's one default sink → `parec` → the bridge → a raw
|
||||
WebSocket client, measured RMS ≈9292 while `stuntcar` was actually playing (vs. silence
|
||||
before advancing past its title screen) — not just a plumbing check, actual game audio.
|
||||
|
||||
Target latency is a few tens of ms, not seconds — that rules out the "obvious" approach of
|
||||
a PulseAudio null sink piped through `ffmpeg` into a compressed (mp3/ogg) HTTP/Icecast-style
|
||||
stream consumed by an `<audio>` tag: codec frame buffering plus the `<audio>` element's own
|
||||
jitter buffer realistically puts that in the 1-3s range, no matter how it's tuned.
|
||||
**Audio is a single shared mix, not per-game** — deliberately simplified from an
|
||||
earlier per-slot-isolated design (considered and dropped as unnecessary complexity):
|
||||
every running game's audio mixes into Pulse's one default sink, and every `/screen/<name>`
|
||||
page connects to the same single `AUDIO_PORT` (`71${DISPLAY_NUM}`, `7199` by default).
|
||||
Trade-off: with more than one game running, you can't tell their audio apart.
|
||||
|
||||
Concrete approach instead: PulseAudio null sink → capture raw/lightly-buffered PCM (small
|
||||
frames, e.g. `parec` with a short `--latency`) → push those frames to the browser over a
|
||||
plain WebSocket (new endpoint alongside `setup_server.py`, or a small dedicated process) →
|
||||
browser side, feed them straight into the Web Audio API via an `AudioWorkletNode` (not
|
||||
`<audio>`, not `ScriptProcessorNode` — that's deprecated and has worse latency) for
|
||||
near-real-time scheduled playback. No container/codec framing in the path at all.
|
||||
**Real bug found and fixed along the way**: `parec --device=@DEFAULT_SINK@.monitor`
|
||||
(the seemingly-obvious macro syntax) fails with "Stream error: Invalid argument" —
|
||||
PulseAudio has a *separate* single-token macro, `@DEFAULT_MONITOR@`, for exactly this;
|
||||
concatenating `@DEFAULT_SINK@` with a literal `.monitor` suffix isn't valid.
|
||||
|
||||
Build this to be reusable across the other noVNC-family projects (`docker-xserver-novnc`,
|
||||
`docker-sdr-novnc`), not dosbox-specific — it's a "browser audio + noVNC" concern, nothing
|
||||
about it is really about DOSBox. Follow the `docker-common/scripts/signals.sh` precedent
|
||||
(see the `docker-common` repo): keep the generic PulseAudio-sink/WebSocket/AudioWorklet
|
||||
piece project-agnostic there, then copy it (not symlink) into each project's own `scripts/`
|
||||
the same way signal handling is shared, so a fix in one place gets propagated by hand to
|
||||
the others.
|
||||
`server.sh` now starts `pulseaudio --start --exit-idle-time=-1` and the bridge
|
||||
(`pcm_ws_bridge.py 7199 parec --device=@DEFAULT_MONITOR@ ...`) once, before `exec`-ing
|
||||
`setup_server.py` — both live for the container's whole lifetime, independent of any
|
||||
game's start/stop. `setup_server.py` gained `GET /screen/<name>` (an iframe onto the
|
||||
game's noVNC screen plus an "Enable Sound" button — browsers require a user gesture
|
||||
before audio can start) and `GET /pcm-worklet.js`; the "Open Screen" link is now a plain
|
||||
same-origin relative link (`/screen/<name>`) instead of the old client-side-JS-built
|
||||
cross-port link, since the wrapper page itself now handles connecting to the right ports
|
||||
(reading the actual host from the request's own `Host:` header server-side).
|
||||
|
||||
Reference in New Issue
Block a user