From 7bbacc729b279b5a0db0e8637d6d1b7870f1e4e5 Mon Sep 17 00:00:00 2001 From: Jens Ahrensfeld Date: Sat, 30 May 2026 14:47:23 +0200 Subject: [PATCH] [README] - document all bug fixes in implementation notes Co-Authored-By: Claude Sonnet 4.6 --- src/tests/README.md | 25 ++++++++++++++++++++----- 1 file changed, 20 insertions(+), 5 deletions(-) diff --git a/src/tests/README.md b/src/tests/README.md index c8a632f..6920626 100644 --- a/src/tests/README.md +++ b/src/tests/README.md @@ -51,15 +51,30 @@ These expect a trained model to already exist in `results/`. ## Implementation notes -### `cd_gaussian_binary` — two fixes applied (see `rbm/train.py`) +### Bug fixes applied (see `rbm/train.py`, `rbm/matrix.py`, `rbm/label.py`) -Surfaced by `test_gaussian_autoencoder.py`. The original negative phase had two bugs: +#### `train.py` — momentum reset every epoch (critical) -1. **Spurious Gaussian noise** — `h_probs_neg` was computed from `data_neg + N(0,1)` instead of `data_neg` directly. Noise in the negative hidden activations injects variance into every gradient step. +`grad_zero()` was called inside the epoch loop, zeroing the momentum accumulator before every gradient update. For the default full-batch case (one mini-batch per epoch), momentum had zero effect — every step was just `lr * dw`. Moved `grad_zero()` outside the loop so momentum accumulates correctly across epochs. **GB-RBM reconstruction MSE dropped from ~0.22 to ~0.014 on the Gaussian blob test.** -2. **Soft hidden states for reconstruction** — `data_neg` was computed from `h_probs_pos` (soft probabilities) instead of sampled binary states. Using probabilities produces a reconstruction that averages over all hidden configurations, giving an overly smooth fantasy particle that biases the negative phase toward the mean. +#### `train.py` — `cd_gaussian_binary` negative phase (two fixes) -Both are now fixed. Reconstruction MSE on the Gaussian blob test improved from ~0.225 to ~0.217. The remaining error floor (~0.22) is a model capacity limit (32 binary hidden units for 64-dimensional continuous input), not a training algorithm issue. +1. **Spurious Gaussian noise** — `h_probs_neg` was computed from `data_neg + N(0,1)` instead of `data_neg`. Noise injected variance into every gradient step. +2. **Soft hidden states for reconstruction** — `data_neg` was computed from `h_probs_pos` (probabilities) instead of sampled binary states, biasing the fantasy particle toward the mean. + +#### `train.py` — `cd_gaussian_gaussian` negative phase (same pattern) + +Same two bugs as `cd_gaussian_binary`: noise was added to `data_neg` before computing `h_probs_neg`, and `h_probs_pos` (mean) was used to drive the negative reconstruction instead of a Gaussian sample. Fixed to use sampled `h` for reconstruction and clean means for weight updates. + +#### `matrix.py` — `rms_error` wrong denominator + +`d_err_squared[1]` (row 1 of the matrix) was used instead of `d_err_squared.shape[1]` (column count). Dead code but now correct. + +#### `label.py` — CuPy compatibility and incomplete implementation + +- `_dec_binary` used Python's `reversed()` on a CuPy array (unsupported) — replaced with `np.flip()`. +- `label2vec_onehot` returned a Python list instead of a stacked array — now uses `np.stack()`. +- `vec2label_onehot` was unimplemented (returned `None`) — now uses `np.argmax()`. ---