jensandClaude Sonnet 4.6
2a6b00db88
Notebook reconstructs the C++ context33.prj configuration exactly:
- shared-weights StackRnn (1 entity, reused every time step)
- visible = [context(128) | x_t(40)] = 168 units, hidden = 128
- all hyperparams from .prj: lr=0.05, momentum=0.5, epochs=1000,
mini_batch=100, gibbs=3, rao_blackwell=True, weight_decay=0
Training data loaded directly from context33.training.dat (Armadillo format,
12 × 168): sensory one-hot chars decoded as 'XURQLIFB9651'.
Key design: training pairs are (h_t, x_{t+1}) not (h_{t-1}, x_t) —
context is first advanced by seeing x_t, then the model is trained to
predict the next character x_{t+1} from that context. Evaluation using
clamped Gibbs on h_t achieves 100% next-step prediction accuracy.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
GRBM
Paper
https://medium.com/@rtdcunha/gaussian-bernoulli-restricted-boltzmann-machines-4a68b8765485 https://journals.plos.org/plosone/article/file?id=10.1371/journal.pone.0171015&type=printable
Code
https://github.com/DSL-Lab/GRBM/tree/main
CRBM
Paper
https://www.ee.nthu.edu.tw/hchen/pubs/iee2003.pdf
Regularization
https://benihime91.github.io/blog/machinelearning/deeplearning/python3.x/tensorflow2.x/2020/10/08/adamW.html https://medium.com/analytics-vidhya/l1-vs-l2-regularization-which-is-better-d01068e6658c https://jamesmccaffreyblog.com/2019/05/09/the-difference-between-neural-network-l2-regularization-and-weight-decay