# Tests

Run any test from the `src/tests/` directory:

```bash
cd src/tests
python test_<name>.py
```

---

## Self-contained tests

These generate their own data and run without external files.

| File | RBM type | What it tests |
|---|---|---|
| `test_binary_autoencoder.py` | BB-RBM | Auto-encoder on synthetic bars-and-stripes binary images (8×8). Trains, reconstructs, and plots original vs. reconstruction. |
| `test_model.py` | BB-RBM (4 layers) | Deep auto-encoder on random binary data (1024-dim). Prints per-pattern reconstruction error. |
| `test_xor.py` | BB-RBM | Auto-encoder on XOR-like 3-bit patterns using the older `Layer` API. |
| `test_gaussian_autoencoder.py` | GB-RBM | Auto-encoder on synthetic Gaussian blob images (8×8). Trains, reconstructs, and plots original vs. reconstruction. |
| `test_linear.py` | GB-RBM or GG-RBM | GB/GG-RBM on random continuous data (3000-dim). Toggle `do_gaussian_hidden` at the top of the file. |

---

## Tests requiring external image data

These load images from a fixed path on disk.

| File | RBM type | What it tests |
|---|---|---|
| `test_faces_sub_image.py` | GB-RBM | Auto-encoder on 32×32 patches from Caltech WebFaces (`/media/jens/cifs/bilder/MachineVision/Caltech_WebFaces/`). Uses `SubImage` with 50% overlap (stride=16); reconstruction blends overlapping patches by pixel-averaging. Supports RGB and grayscale modes. CLI args: `--grayscale`, `--load_model`, `--do_train`, `--l1_lambda`, `--num_epochs`, `--mini_batch_size`. After training: shows learned weight filters, patch-level reconstructions, and a full overlap-blended image reconstruction. Results summary (grayscale, 256 hidden units, 1000 epochs): L1=0.01 → MSE 0.168, MAE 0.214, structured edge/texture filters; L1=0.05 → MSE 0.195, MAE 0.228, sparse blob detectors. **L1=0.01 is the recommended sweet spot.** Mini-batch size has little effect on final quality (mini_batch=200 vs 1000 give equivalent MSE) but smaller batches amplify the effective L1 penalty per epoch due to more frequent updates. |
| `test_sub_image.py` | — | Unit test for `SubImage` (patch extraction). No RBM involved. |
| `test_conv2d.py` | — | Unit test for `conv2d`. No RBM involved. |

---

## Tests requiring external data

These load data from `/home/jens/work/repos/Rbm/` (Armadillo `.dat` files).

| File | RBM type | What it tests |
|---|---|---|
| `test_norbs.py` | GB-RBM or GG-RBM | Single-layer auto-encoder on NORB small (96×96 images). Toggle `DO_HIDDEN_GAUSSIAN` at the top of the file. |
| `test_deep_norbs.py` | GB-RBM + BB-RBM | Two-layer deep auto-encoder: loads pre-trained GB-RBM weights, trains a BB-RBM on top. |
| `test_learn_norbs_labels.py` | GB-RBM + BB-RBM | Multimodal model combining NORB image features with one-hot label embeddings in a joint hidden layer. |
| `test_rbm.py` | Any (from `.prj` file) | Full pipeline via `StackFactory`. Takes a project name as CLI argument (`python test_rbm.py <name>`). Uses OpenCV for interactive reconstruction display. |

---

## Tests requiring saved state

These expect a trained model to already exist in `results/`.

| File | Prerequisite |
|---|---|
| `test_label.py` | Requires a saved `Label` fitter. Trains a binary label encoder (vocabulary of 100 integers → 5×5 binary codes) and visualises encoded labels. Set `do_train = True` on first run. |
| `test_learn_encoded_labels.py` | Requires a saved one-hot label fitter. Trains a BB-RBM on one-hot encoded integer labels and plots hidden representation, reconstruction, and original. Set `do_train = True` on first run. |

---

## Implementation notes

### Bug fixes applied (see `rbm/train.py`, `rbm/matrix.py`, `rbm/label.py`)

#### `train.py` — momentum reset every epoch (critical)

`grad_zero()` was called inside the epoch loop, zeroing the momentum accumulator before every gradient update. For the default full-batch case (one mini-batch per epoch), momentum had zero effect — every step was just `lr * dw`. Moved `grad_zero()` outside the loop so momentum accumulates correctly across epochs. **GB-RBM reconstruction MSE dropped from ~0.22 to ~0.014 on the Gaussian blob test.**

#### `train.py` — `cd_gaussian_binary` negative phase (two fixes)

1. **Spurious Gaussian noise** — `h_probs_neg` was computed from `data_neg + N(0,1)` instead of `data_neg`. Noise injected variance into every gradient step.
2. **Soft hidden states for reconstruction** — `data_neg` was computed from `h_probs_pos` (probabilities) instead of sampled binary states, biasing the fantasy particle toward the mean.

#### `train.py` — `cd_gaussian_gaussian` negative phase (same pattern)

Same two bugs as `cd_gaussian_binary`: noise was added to `data_neg` before computing `h_probs_neg`, and `h_probs_pos` (mean) was used to drive the negative reconstruction instead of a Gaussian sample. Fixed to use sampled `h` for reconstruction and clean means for weight updates.

#### `matrix.py` — `rms_error` wrong denominator

`d_err_squared[1]` (row 1 of the matrix) was used instead of `d_err_squared.shape[1]` (column count). Dead code but now correct.

#### `label.py` — CuPy compatibility and incomplete implementation

- `_dec_binary` used Python's `reversed()` on a CuPy array (unsupported) — replaced with `np.flip()`.
- `label2vec_onehot` returned a Python list instead of a stacked array — now uses `np.stack()`.
- `vec2label_onehot` was unimplemented (returned `None`) — now uses `np.argmax()`.

#### `entity.py` — L1 and L2 regularisation gradients wrong

`grad_compute` computed scalar norms and multiplied by `W`, giving gradients that scaled with the total weight magnitude rather than the correct per-element derivatives:
- **L1**: gradient should be `λ·sign(W)`, not `λ·‖W‖₁·W`
- **L2**: gradient should be `2λ·W`, not `λ·‖W‖₂²·W`

The combined `(l1+l2)·W` term also incorrectly mixed both penalties into one. Fixed to apply each gradient independently. `weight_decay` (the correct `λW` form of L2) is kept as a separate term.

#### `image.py` — `normalize()` fragile and verbose

The `repeat`/`reshape` approach was replaced with `keepdims=True` broadcasting and a `std < 1e-8` guard to prevent NaN on constant patches. The duplicate normalization logic in `test_faces_sub_image.py:load_image_patches` was removed in favour of calling `normalize()` directly.

#### `train.py` — dead `cd_jens` function removed

`cd_jens` was never called by `train()` or anywhere else. It contained multiple bugs (`prob()` applied to raw data, `entity.forward()` called twice). Removed entirely.

#### `train.py` — `cd_gaussian_binary` redundant visible bias subtraction

`dbv = sum(data_pos - b_v) - sum(data_neg - b_v)` — the `b_v` terms cancel algebraically, making the subtraction redundant and misleading. Simplified to `sum(data_pos) - sum(data_neg)`.

#### `train.py` — `cd_binary_gaussian` Gibbs loop used means instead of samples

For CD-k > 1, the Gibbs loop must sample both visible and hidden units to continue the Markov chain. Previously the loop used the visible mean (`prob(...)` instead of `sample(prob(...))`) and never sampled the Gaussian hidden units. Fixed to respect `do_rao_blackwell`: when False, visible is sampled and Gaussian noise is added to hidden at each step. CD-1 (default) is unaffected since the loop runs 0 times.

#### `train.py` — no data shuffling between epochs

Mini-batches were always sliced in the same order every epoch, introducing systematic gradient bias for mini-batch training. A random permutation is now applied to the full batch at the start of each epoch.

#### `train.py` — `if` chains instead of `elif/else` for CD function selection

All four `if` statements were evaluated even after a match, and an unrecognised entity type would leave `cd_func = None`, crashing later with a cryptic `TypeError`. Replaced with `elif/else` that raises `ValueError` immediately.

#### `entity.py` — L1/L2 regularisation divided by mini-batch size

`state_adjust(grad, k=1/N)` applied the `1/N` scale to the entire gradient, including the regularisation terms. The CD gradient is a batch sum so `1/N` normalisation is correct; L1/L2/weight-decay penalties are per-weight and batch-size independent. Dividing them by `N` made the effective regularisation strength `lambda / N`, so e.g. `l1_lambda=0.5` with `mini_batch_size=1000` was equivalent to an effective L1 of `0.0005` — too small to have any visible effect on filters. Fixed by moving the regularisation update into `state_adjust` where it is applied at full strength before the `1/N` CD update.

---

## RBM type reference

| Type | Visible | Hidden | `EntityParams` |
|---|---|---|---|
| BB-RBM | Binary | Binary | `EntityParams()` |
| GB-RBM | Gaussian | Binary | `EntityParams(do_gaussian_visible=True)` |
| BG-RBM | Binary | Gaussian | `EntityParams(do_gaussian_hidden=True)` |
| GG-RBM | Gaussian | Gaussian | `EntityParams(do_gaussian_visible=True, do_gaussian_hidden=True)` |