jensandClaude Sonnet 4.6
952045b9f1
stack_rnn.py:
- _train_unrolled: C++-style temporal-shift padding — flatten (num_seq, T, s)
→ (N, s), append T-1 zero rows, layer t trains on batch[t:N+t]; joint
training replaces greedy sequential (all layers update each epoch)
- Reverted sequential/greedy training path (poor next-step prediction)
moby_rnn.ipynb:
- Vocabulary reduced 85 → 40 chars (A-Z, 0-9, space, .!?)
- T renamed to TEMPORAL_DEPTH throughout
- TEMPORAL_DEPTH increased 5 → 64; CONTEXT_SIZE 128
- SEED "Call me Ishmael." now falls within training data coverage
state.py:
- Remove per-file load/save print messages (too noisy with 64-layer checkpoints)
README_moby_rnn.md:
- Update vocab, constants, parameter count, checkpoint listing
- Add temporal-shift padding and joint training sections
- Clarify reconstruction accuracy vs next-step prediction as honest metric
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
GRBM
Paper
https://medium.com/@rtdcunha/gaussian-bernoulli-restricted-boltzmann-machines-4a68b8765485 https://journals.plos.org/plosone/article/file?id=10.1371/journal.pone.0171015&type=printable
Code
https://github.com/DSL-Lab/GRBM/tree/main
CRBM
Paper
https://www.ee.nthu.edu.tw/hchen/pubs/iee2003.pdf
Regularization
https://benihime91.github.io/blog/machinelearning/deeplearning/python3.x/tensorflow2.x/2020/10/08/adamW.html https://medium.com/analytics-vidhya/l1-vs-l2-regularization-which-is-better-d01068e6658c https://jamesmccaffreyblog.com/2019/05/09/the-difference-between-neural-network-l2-regularization-and-weight-decay