[README] - document cd_jens removal, dbv simplification, Gibbs sampling fix

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
2026-05-30 20:55:00 +02:00
co-authored by Claude Sonnet 4.6
parent b5104ebc97
commit 9911caae06
+12
View File
@@ -98,6 +98,18 @@ The combined `(l1+l2)·W` term also incorrectly mixed both penalties into one. F
The `repeat`/`reshape` approach was replaced with `keepdims=True` broadcasting and a `std < 1e-8` guard to prevent NaN on constant patches. The duplicate normalization logic in `test_faces_sub_image.py:load_image_patches` was removed in favour of calling `normalize()` directly.
#### `train.py` — dead `cd_jens` function removed
`cd_jens` was never called by `train()` or anywhere else. It contained multiple bugs (`prob()` applied to raw data, `entity.forward()` called twice). Removed entirely.
#### `train.py` — `cd_gaussian_binary` redundant visible bias subtraction
`dbv = sum(data_pos - b_v) - sum(data_neg - b_v)` — the `b_v` terms cancel algebraically, making the subtraction redundant and misleading. Simplified to `sum(data_pos) - sum(data_neg)`.
#### `train.py` — `cd_binary_gaussian` Gibbs loop used means instead of samples
For CD-k > 1, the Gibbs loop must sample both visible and hidden units to continue the Markov chain. Previously the loop used the visible mean (`prob(...)` instead of `sample(prob(...))`) and never sampled the Gaussian hidden units. Fixed to respect `do_rao_blackwell`: when False, visible is sampled and Gaussian noise is added to hidden at each step. CD-1 (default) is unaffected since the loop runs 0 times.
#### `train.py` — no data shuffling between epochs
Mini-batches were always sliced in the same order every epoch, introducing systematic gradient bias for mini-batch training. A random permutation is now applied to the full batch at the start of each epoch.