pyRBM/tests

Tests

Run any test from the src/tests/ directory:

cd src/tests
python test_<name>.py

---

Self-contained tests

These generate their own data and run without external files.

File	RBM type	What it tests
test_binary_autoencoder.py	BB-RBM	Auto-encoder on synthetic bars-and-stripes binary images (8×8). Trains, reconstructs, and plots original vs. reconstruction.
test_model.py	BB-RBM (4 layers)	Deep auto-encoder on random binary data (1024-dim). Prints per-pattern reconstruction error.
test_xor.py	BB-RBM	Auto-encoder on XOR-like 3-bit patterns using the older Layer API.
test_gaussian_autoencoder.py	GB-RBM	Auto-encoder on synthetic Gaussian blob images (8×8). Trains, reconstructs, and plots original vs. reconstruction.
test_linear.py	GB-RBM or GG-RBM	GB/GG-RBM on random continuous data (3000-dim). Toggle do_gaussian_hidden at the top of the file.

---

Tests requiring external image data

These load images from a fixed path on disk.

File	RBM type	What it tests
test_faces_sub_image.py	GB-RBM	Auto-encoder on 32×32 patches from Caltech WebFaces (/media/jens/cifs/bilder/MachineVision/Caltech_WebFaces/). Uses SubImage with 50% overlap (stride=16); reconstruction blends overlapping patches by pixel-averaging. Supports RGB and grayscale modes. CLI args: --grayscale, --load_model, --do_train, --l1_lambda, --num_epochs, --mini_batch_size. After training: shows learned weight filters, patch-level reconstructions, and a full overlap-blended image reconstruction. Results summary (grayscale, 256 hidden units, 1000 epochs): L1=0.01 → MSE 0.168, MAE 0.214, structured edge/texture filters; L1=0.05 → MSE 0.195, MAE 0.228, sparse blob detectors. L1=0.01 is the recommended sweet spot. Mini-batch size has little effect on final quality (mini_batch=200 vs 1000 give equivalent MSE) but smaller batches amplify the effective L1 penalty per epoch due to more frequent updates.
test_sub_image.py	—	Unit test for SubImage (patch extraction). No RBM involved.
test_conv2d.py	—	Unit test for conv2d. No RBM involved.

---

Tests requiring external data

These load data from /home/jens/work/repos/Rbm/ (Armadillo .dat files).

File	RBM type	What it tests
test_norbs.py	GB-RBM or GG-RBM	Single-layer auto-encoder on NORB small (96×96 images). Toggle DO_HIDDEN_GAUSSIAN at the top of the file.
test_deep_norbs.py	GB-RBM + BB-RBM	Two-layer deep auto-encoder: loads pre-trained GB-RBM weights, trains a BB-RBM on top.
test_learn_norbs_labels.py	GB-RBM + BB-RBM	Multimodal model combining NORB image features with one-hot label embeddings in a joint hidden layer.
test_rbm.py	Any (from .prj file)	Full pipeline via StackFactory. Takes a project name as CLI argument (python test_rbm.py <name>). Uses OpenCV for interactive reconstruction display.

---

Tests requiring saved state

These expect a trained model to already exist in results/.

File	Prerequisite
test_label.py	Requires a saved Label fitter. Trains a binary label encoder (vocabulary of 100 integers → 5×5 binary codes) and visualises encoded labels. Set do_train = True on first run.
test_learn_encoded_labels.py	Requires a saved one-hot label fitter. Trains a BB-RBM on one-hot encoded integer labels and plots hidden representation, reconstruction, and original. Set do_train = True on first run.

---

Implementation notes

Bug fixes applied (see rbm/train.py, rbm/matrix.py, rbm/label.py)

train.py — momentum reset every epoch (critical)

grad_zero() was called inside the epoch loop, zeroing the momentum accumulator before every gradient update. For the default full-batch case (one mini-batch per epoch), momentum had zero effect — every step was just lr * dw. Moved grad_zero() outside the loop so momentum accumulates correctly across epochs. GB-RBM reconstruction MSE dropped from ~0.22 to ~0.014 on the Gaussian blob test.

train.py — cd_gaussian_binary negative phase (two fixes)

1. Spurious Gaussian noise — h_probs_neg was computed from data_neg + N(0,1) instead of data_neg. Noise injected variance into every gradient step.
2. Soft hidden states for reconstruction — data_neg was computed from h_probs_pos (probabilities) instead of sampled binary states, biasing the fantasy particle toward the mean.

train.py — cd_gaussian_gaussian negative phase (same pattern)

Same two bugs as cd_gaussian_binary: noise was added to data_neg before computing h_probs_neg, and h_probs_pos (mean) was used to drive the negative reconstruction instead of a Gaussian sample. Fixed to use sampled h for reconstruction and clean means for weight updates.

matrix.py — rms_error wrong denominator

d_err_squared[1] (row 1 of the matrix) was used instead of d_err_squared.shape[1] (column count). Dead code but now correct.

label.py — CuPy compatibility and incomplete implementation

• _dec_binary used Python's reversed() on a CuPy array (unsupported) — replaced with np.flip().
• label2vec_onehot returned a Python list instead of a stacked array — now uses np.stack().
• vec2label_onehot was unimplemented (returned None) — now uses np.argmax().

entity.py — L1 and L2 regularisation gradients wrong

grad_compute computed scalar norms and multiplied by W, giving gradients that scaled with the total weight magnitude rather than the correct per-element derivatives:

• L1: gradient should be λ·sign(W), not λ·‖W‖₁·W
• L2: gradient should be 2λ·W, not λ·‖W‖₂²·W

The combined (l1+l2)·W term also incorrectly mixed both penalties into one. Fixed to apply each gradient independently. weight_decay (the correct λW form of L2) is kept as a separate term.

image.py — normalize() fragile and verbose

The repeat/reshape approach was replaced with keepdims=True broadcasting and a std < 1e-8 guard to prevent NaN on constant patches. The duplicate normalization logic in test_faces_sub_image.py:load_image_patches was removed in favour of calling normalize() directly.

train.py — dead cd_jens function removed

cd_jens was never called by train() or anywhere else. It contained multiple bugs (prob() applied to raw data, entity.forward() called twice). Removed entirely.

train.py — cd_gaussian_binary redundant visible bias subtraction

dbv = sum(data_pos - b_v) - sum(data_neg - b_v) — the b_v terms cancel algebraically, making the subtraction redundant and misleading. Simplified to sum(data_pos) - sum(data_neg).

train.py — cd_binary_gaussian Gibbs loop used means instead of samples

For CD-k > 1, the Gibbs loop must sample both visible and hidden units to continue the Markov chain. Previously the loop used the visible mean (prob(...) instead of sample(prob(...))) and never sampled the Gaussian hidden units. Fixed to respect do_rao_blackwell: when False, visible is sampled and Gaussian noise is added to hidden at each step. CD-1 (default) is unaffected since the loop runs 0 times.

train.py — no data shuffling between epochs

Mini-batches were always sliced in the same order every epoch, introducing systematic gradient bias for mini-batch training. A random permutation is now applied to the full batch at the start of each epoch.

train.py — if chains instead of elif/else for CD function selection

All four if statements were evaluated even after a match, and an unrecognised entity type would leave cd_func = None, crashing later with a cryptic TypeError. Replaced with elif/else that raises ValueError immediately.

entity.py — L1/L2 regularisation divided by mini-batch size

state_adjust(grad, k=1/N) applied the 1/N scale to the entire gradient, including the regularisation terms. The CD gradient is a batch sum so 1/N normalisation is correct; L1/L2/weight-decay penalties are per-weight and batch-size independent. Dividing them by N made the effective regularisation strength lambda / N, so e.g. l1_lambda=0.5 with mini_batch_size=1000 was equivalent to an effective L1 of 0.0005 — too small to have any visible effect on filters. Fixed by moving the regularisation update into state_adjust where it is applied at full strength before the 1/N CD update.

---

RBM type reference

Type	Visible	Hidden	EntityParams
BB-RBM	Binary	Binary	EntityParams()
GB-RBM	Gaussian	Binary	EntityParams(do_gaussian_visible=True)
BG-RBM	Binary	Gaussian	EntityParams(do_gaussian_hidden=True)
GG-RBM	Gaussian	Gaussian	EntityParams(do_gaussian_visible=True, do_gaussian_hidden=True)