[README] - document train() shuffling and elif fixes
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
@@ -98,6 +98,14 @@ The combined `(l1+l2)·W` term also incorrectly mixed both penalties into one. F
|
||||
|
||||
The `repeat`/`reshape` approach was replaced with `keepdims=True` broadcasting and a `std < 1e-8` guard to prevent NaN on constant patches. The duplicate normalization logic in `test_faces_sub_image.py:load_image_patches` was removed in favour of calling `normalize()` directly.
|
||||
|
||||
#### `train.py` — no data shuffling between epochs
|
||||
|
||||
Mini-batches were always sliced in the same order every epoch, introducing systematic gradient bias for mini-batch training. A random permutation is now applied to the full batch at the start of each epoch.
|
||||
|
||||
#### `train.py` — `if` chains instead of `elif/else` for CD function selection
|
||||
|
||||
All four `if` statements were evaluated even after a match, and an unrecognised entity type would leave `cd_func = None`, crashing later with a cryptic `TypeError`. Replaced with `elif/else` that raises `ValueError` immediately.
|
||||
|
||||
#### `entity.py` — L1/L2 regularisation divided by mini-batch size
|
||||
|
||||
`state_adjust(grad, k=1/N)` applied the `1/N` scale to the entire gradient, including the regularisation terms. The CD gradient is a batch sum so `1/N` normalisation is correct; L1/L2/weight-decay penalties are per-weight and batch-size independent. Dividing them by `N` made the effective regularisation strength `lambda / N`, so e.g. `l1_lambda=0.5` with `mini_batch_size=1000` was equivalent to an effective L1 of `0.0005` — too small to have any visible effect on filters. Fixed by moving the regularisation update into `state_adjust` where it is applied at full strength before the `1/N` CD update.
|
||||
|
||||
Reference in New Issue
Block a user