jens/pyRBM

T

jensandClaude Sonnet 4.6 abee870ada [bugfix] - fix L1 and L2 regularisation gradients in grad_compute

Previous code computed scalar norms and multiplied by W, giving gradients
that scaled with total weight magnitude rather than the correct per-element
derivatives:
  L1: gradient is λ·sign(W), not λ·‖W‖₁·W
  L2: gradient is 2λ·W, not λ·‖W‖₂²·W
Also removed the combined (l1+l2)*W term which incorrectly mixed the two.
weight_decay (correct L2 form λW) is kept as a separate term.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

2026-05-30 18:26:06 +02:00

[bugfix] - fix L1 and L2 regularisation gradients in grad_compute

2026-05-30 18:26:06 +02:00

.gitignore

updated gitignore

2025-12-19 11:11:00 +01:00

cifar_test_sub_image.ipynb

updated notebooks

2026-01-11 11:23:35 +01:00

cifar_test.ipynb

[cifar-test]

2026-01-10 21:44:03 +01:00

layer_test.ipynb

refactored

2025-12-19 15:20:04 +01:00

mnist_test_sub_image.ipynb

updated notebooks

2026-01-11 11:23:35 +01:00

mnist_test.ipynb

[mnist-test]

2026-01-10 19:58:39 +01:00

pyproject.toml

added matplotlib

2025-12-18 07:56:45 +01:00

README.md

added L2-Regulation

2026-01-06 18:00:26 +01:00

test_params.json

added Test passed message

2025-12-16 21:46:00 +01:00

README.md

GRBM

Paper

https://medium.com/@rtdcunha/gaussian-bernoulli-restricted-boltzmann-machines-4a68b8765485 https://journals.plos.org/plosone/article/file?id=10.1371/journal.pone.0171015&type=printable

Code

https://github.com/DSL-Lab/GRBM/tree/main

CRBM

Paper

https://www.ee.nthu.edu.tw/hchen/pubs/iee2003.pdf

Regularization

https://benihime91.github.io/blog/machinelearning/deeplearning/python3.x/tensorflow2.x/2020/10/08/adamW.html https://medium.com/analytics-vidhya/l1-vs-l2-regularization-which-is-better-d01068e6658c https://jamesmccaffreyblog.com/2019/05/09/the-difference-between-neural-network-l2-regularization-and-weight-decay