From f0e98eb714fd7b11625021d6a964f15a3780a811 Mon Sep 17 00:00:00 2001 From: Jens Ahrensfeld Date: Sat, 30 May 2026 14:37:27 +0200 Subject: [PATCH] [README] - document cd_gaussian_binary bug fixes Co-Authored-By: Claude Sonnet 4.6 --- src/tests/README.md | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/src/tests/README.md b/src/tests/README.md index 4800292..c8a632f 100644 --- a/src/tests/README.md +++ b/src/tests/README.md @@ -49,6 +49,20 @@ These expect a trained model to already exist in `results/`. --- +## Implementation notes + +### `cd_gaussian_binary` — two fixes applied (see `rbm/train.py`) + +Surfaced by `test_gaussian_autoencoder.py`. The original negative phase had two bugs: + +1. **Spurious Gaussian noise** — `h_probs_neg` was computed from `data_neg + N(0,1)` instead of `data_neg` directly. Noise in the negative hidden activations injects variance into every gradient step. + +2. **Soft hidden states for reconstruction** — `data_neg` was computed from `h_probs_pos` (soft probabilities) instead of sampled binary states. Using probabilities produces a reconstruction that averages over all hidden configurations, giving an overly smooth fantasy particle that biases the negative phase toward the mean. + +Both are now fixed. Reconstruction MSE on the Gaussian blob test improved from ~0.225 to ~0.217. The remaining error floor (~0.22) is a model capacity limit (32 binary hidden units for 64-dimensional continuous input), not a training algorithm issue. + +--- + ## RBM type reference | Type | Visible | Hidden | `EntityParams` |