From 7bd3bbad5772626497e92c4dcb500a1e5ba5ce18 Mon Sep 17 00:00:00 2001 From: Jens Ahrensfeld Date: Mon, 27 Jul 2026 16:24:34 +0200 Subject: [PATCH] Add optional NVBLAS GPU acceleration; fix stale CLAUDE.md sections The RTX 4070 Ti + CUDA 12 toolkit + libnvblas.so are already installed on this machine (from pyRBM's cupy setup); Bandicoot (the alternative, much bigger Armadillo-to-GPU rewrite) is not. NVBLAS is a drop-in BLAS shim -- zero source changes -- that intercepts Armadillo's large matrix-multiply calls (v_to_h, h_to_v, CD gradients) and offloads them to the GPU via cuBLAS, falling back to the system's OpenBLAS otherwise. Add nvblas.conf (CPU fallback pointed at openblas-pthread, GPU_LIST ALL) and run_gpu.sh, an opt-in LD_PRELOAD wrapper: ./run_gpu.sh [args...]. Verified for real: poet.elf f produced byte-identical output through the wrapper, and a real training run against moby_ch1.txt showed GPU utilization rise from ~0-1% idle to 7-28% and memory usage grow from 893MiB to 1.4GB, while training progressed correctly (error decreasing, coherent generated text). Added nvblas.log to .gitignore. Also fixed two stale CLAUDE.md sections found along the way: the TEST target description still described the old broken manual smoke-test main.cpp instead of the minimal test suite that replaced it, and the "Project files on disk" section still described the flat repo-root layout from before the prj// restructure. Co-Authored-By: Claude Sonnet 5 Claude-Session: https://claude.ai/code/session_016K8Gu7Qejd11JbdiHZqYAs --- .gitignore | 1 + CLAUDE.md | 35 ++++++++++++++++++++++++++++------- nvblas.conf | 5 +++++ run_gpu.sh | 12 ++++++++++++ 4 files changed, 46 insertions(+), 7 deletions(-) create mode 100644 nvblas.conf create mode 100755 run_gpu.sh diff --git a/.gitignore b/.gitignore index 31d2d86..a812aac 100644 --- a/.gitignore +++ b/.gitignore @@ -1,2 +1,3 @@ build/ juce/ +nvblas.log diff --git a/CLAUDE.md b/CLAUDE.md index 9081f01..70ac663 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -26,14 +26,32 @@ The `GUI` target additionally depends on `juce/build/${CONFIG}/libjuce.a`, built `install` copies `build/release/rbm.elf` to `${HOME}/bin`. Object/dependency files land in `build/${CONFIG}/`; `.d` files are auto-included for header dependency -tracking. There is no separate lint or test-runner command — `TEST_CXX_SRCS` builds `main.cpp` into -`test.elf`, which is a manual smoke-test program (load a project, train, print matrices), not a unit test -suite. Run it directly to sanity-check core behavior: +tracking. There is no separate lint command. `TEST_CXX_SRCS` builds `main.cpp` into `test.elf`, a +minimal dependency-free test suite: for each named project under `prj//`, it loads the project, +loads weights and training batch, runs a full up-pass/down-pass reconstruction (`DeepStack::upDownPass`), +and checks the reconstruction error is finite and beats a trivial per-feature-mean baseline. Prints +PASS/FAIL (or SKIP for known issues) per project and a summary line; exit code reflects only genuine +failures. ```sh make PRJ=TEST CONFIG=debug && ./build/debug/test.elf ``` +### GPU acceleration (optional) + +`run_gpu.sh` runs any built binary with NVBLAS (`nvblas.conf`), which transparently intercepts Armadillo's +large matrix-multiply BLAS calls (`v_to_h`, `h_to_v`, CD gradients) and offloads them to an NVIDIA GPU via +cuBLAS, falling back to the system's OpenBLAS otherwise. No source changes — it's a runtime `LD_PRELOAD` +shim, opt-in only: + +```sh +./run_gpu.sh ./build/release/poet.elf t some_text.txt +``` + +Requires an NVIDIA GPU + the CUDA runtime's `libnvblas.so` (already present on this machine, alongside +pyRBM's cupy setup). Confirmed via nvidia-smi utilization/memory during a real `poet` training run — no +behavior change, same output, just optionally GPU-accelerated matrix multiplies. + ### External dependencies (system-installed, not vendored) - **Armadillo** (``, linked `-larmadillo`) — all matrix/vector math goes through `arma::mat`. @@ -87,11 +105,14 @@ make PRJ=TEST CONFIG=debug && ./build/debug/test.elf ### Project files on disk -Each named project `` (e.g. `mnist`, `poet_2v_5s`, `context99` — see the many `*.prj` files at repo -root, mostly saved experiment artifacts) consists of: +Each named project `` (e.g. `mnist`, `poet_2v_5s`, `context99` — see the ~40 project folders under +`prj/`, mostly saved experiment artifacts) lives under `prj//` and consists of: - `.prj` — JSON describing the stack type and layer geometry (via `StackCreator`). - `.training.dat` — the training batch matrix. - `.Layer..w.dat` / `.bh.dat` / `.bv.dat` — per-layer weights/biases (Armadillo binary/text format). -These root-level `.dat`/`.prj`/`.txt` files are experiment data and generated artifacts, not something to -edit by hand; treat them as fixtures unless a task specifically concerns training data or saved models. +`StackCreator`/`AStack` resolve these paths themselves (`{dir}/prj/{name}/{name}.*`) from whatever base +`dir` the caller passes (`poet.cpp`/the GUI's startup auto-load both pass `"."`, the repo root) — callers +never construct the `prj//` part themselves. These are experiment data and generated artifacts, not +something to edit by hand; treat them as fixtures unless a task specifically concerns training data or +saved models. diff --git a/nvblas.conf b/nvblas.conf new file mode 100644 index 0000000..709e1fa --- /dev/null +++ b/nvblas.conf @@ -0,0 +1,5 @@ +NVBLAS_LOGFILE nvblas.log +NVBLAS_CPU_BLAS_LIB /usr/lib/x86_64-linux-gnu/openblas-pthread/libblas.so.3 +NVBLAS_GPU_LIST ALL +NVBLAS_TILE_DIM 2048 +NVBLAS_AUTOPIN_MEM_ENABLED diff --git a/run_gpu.sh b/run_gpu.sh new file mode 100755 index 0000000..8db10c4 --- /dev/null +++ b/run_gpu.sh @@ -0,0 +1,12 @@ +#!/bin/bash +# Run a built binary (poet.elf, test.elf, rbm.elf) with NVBLAS: large +# Armadillo matrix multiplies (v_to_h, h_to_v, CD gradients, ...) get +# transparently offloaded to the GPU via cuBLAS, falling back to the +# system's CPU BLAS (nvblas.conf) for anything NVBLAS doesn't handle. +# No source changes -- this only intercepts BLAS symbol calls at runtime. +# +# Usage: ./run_gpu.sh build/release/poet.elf f "the " +set -euo pipefail +here="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +export NVBLAS_CONFIG_FILE="$here/nvblas.conf" +exec env LD_PRELOAD=libnvblas.so "$@"