- Remove cd_jens: dead code never called, contained multiple bugs.
- cd_gaussian_binary: dbv formula simplified — b_v terms cancelled between
positive and negative phases, making the subtraction redundant.
- cd_binary_gaussian: Gibbs loop (CD-k > 1) now properly samples both visible
(sample(prob(...))) and hidden (+ sample_gaussian) states when
do_rao_blackwell=False. Previously used means throughout, giving biased
gradient estimates for k > 1.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Shuffling eliminates systematic gradient bias from fixed mini-batch ordering.
elif/else raises ValueError for unrecognised entity types instead of silently
calling None and crashing with a cryptic TypeError.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Regularisation terms (L1, L2, weight_decay) were being divided by mini_batch_size
in state_adjust along with the CD gradient. The CD gradient is a batch sum so
1/N normalisation is correct; regularisation penalties are per-weight and
batch-size independent. Separating them makes l1_lambda/l2_lambda directly
interpretable regardless of batch size.
Also adds --l1_lambda CLI arg to test_faces_sub_image.py and sets num_epochs=1000.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Previous code computed scalar norms and multiplied by W, giving gradients
that scaled with total weight magnitude rather than the correct per-element
derivatives:
L1: gradient is λ·sign(W), not λ·‖W‖₁·W
L2: gradient is 2λ·W, not λ·‖W‖₂²·W
Also removed the combined (l1+l2)*W term which incorrectly mixed the two.
weight_decay (correct L2 form λW) is kept as a separate term.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
image.py: replace repeat/reshape with keepdims=True; guard std < 1e-8 to
prevent NaN on constant patches (previously relied on call-site pre-filtering).
test_faces_sub_image.py: load_image_patches now calls normalize() directly
instead of duplicating the per-sample normalization logic.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
GRAYSCALE flag drives N_CH (1 or 3) and N_VIS (1024 or 3072) at runtime.
_img_to_chw() handles BGR→gray or BGR→RGB conversion.
All visualisation functions use cmap='gray' when GRAYSCALE is set.
Separate model name (faces_sub_image_gray) keeps weights isolated.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Replace non-overlapping stride=32 with stride=16 (50% overlap) throughout.
Add assemble_overlapping() which accumulates reconstructed patches into a
float buffer and averages contributions per pixel, eliminating blocking artefacts.
Update load_image_patches to return nx_steps/ny_steps for correct reassembly.
Add start parameter to load_patches for held-out test image selection.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Extracts non-overlapping 32x32 RGB patches from face images using SubImage,
trains a GB-RBM (3072 visible, 128 hidden), and visualises learned filters,
patch reconstructions, and full image reconstruction.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- train.py: grad_zero() was called inside the epoch loop, resetting momentum
to zero before every update — momentum had no effect in the default full-batch
case. Moved outside the loop so momentum accumulates across epochs.
- train.py: cd_gaussian_gaussian: use mean h for weight updates and sampled h
to drive the negative visible reconstruction (same pattern as cd_gaussian_binary fix).
Remove spurious Gaussian noise added to data_neg before computing h_probs_neg.
- matrix.py: rms_error divided by d_err_squared[1] (row 1) instead of
d_err_squared.shape[1] (column count).
- label.py: _dec_binary used reversed() on a CuPy array — replaced with np.flip().
label2vec_onehot now returns np.stack() array instead of a Python list.
vec2label_onehot implemented via np.argmax (was returning None).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
In cd_gaussian_binary, two CD formulation bugs:
1. Gaussian noise was incorrectly added to data_neg before computing h_probs_neg
2. Negative visible reconstruction used h_probs_pos (soft probabilities) instead
of sampled binary hidden states, biasing the fantasy particle toward the mean
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
jensandClaude Sonnet 4.6