[README] - document entity.py L1/L2 and image.py normalize fixes
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
@@ -86,6 +86,18 @@ Same two bugs as `cd_gaussian_binary`: noise was added to `data_neg` before comp
|
||||
- `label2vec_onehot` returned a Python list instead of a stacked array — now uses `np.stack()`.
|
||||
- `vec2label_onehot` was unimplemented (returned `None`) — now uses `np.argmax()`.
|
||||
|
||||
#### `entity.py` — L1 and L2 regularisation gradients wrong
|
||||
|
||||
`grad_compute` computed scalar norms and multiplied by `W`, giving gradients that scaled with the total weight magnitude rather than the correct per-element derivatives:
|
||||
- **L1**: gradient should be `λ·sign(W)`, not `λ·‖W‖₁·W`
|
||||
- **L2**: gradient should be `2λ·W`, not `λ·‖W‖₂²·W`
|
||||
|
||||
The combined `(l1+l2)·W` term also incorrectly mixed both penalties into one. Fixed to apply each gradient independently. `weight_decay` (the correct `λW` form of L2) is kept as a separate term.
|
||||
|
||||
#### `image.py` — `normalize()` fragile and verbose
|
||||
|
||||
The `repeat`/`reshape` approach was replaced with `keepdims=True` broadcasting and a `std < 1e-8` guard to prevent NaN on constant patches. The duplicate normalization logic in `test_faces_sub_image.py:load_image_patches` was removed in favour of calling `normalize()` directly.
|
||||
|
||||
---
|
||||
|
||||
## RBM type reference
|
||||
|
||||
Reference in New Issue
Block a user