[add] README.rbm.md: Q&A on RBMs, VAEs, Transformers, and autodiff

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

README.rbm.md

@@ -0,0 +1,190 @@
+# RBM Discussion
+
+A Q&A covering the design of `Rnn.py` and the broader context of RBMs.
+
+---
+
+**Q: look at the Rnn.png in docs. Implement it as ./Rnn.py**
+
+The diagram shows an unrolled RNN-RBM with M time steps:
+
+- `v[0]` = recurrent context (previous hidden state `h`, or zeros at `t=0`)
+- `v[1:N]` = sliding window of N one-hot-encoded characters
+- `W[t]` = separate RBM weight matrix per time step (unrolled mode)
+
+Implemented as `Rnn.py` using `StackRnn` in unrolled mode. For `TEXT="  JAY"`, `N_WIN=3`:
+
+| Step | v[0] | v[1:N] | Weight |
+|---|---|---|---|
+| t=0 | {0} | [' ',' ','J'] | W[0] |
+| t=1 | h₀ | [' ','J','A'] | W[1] |
+| t=M-1 | h_{M-2} | ['J','A','Y'] | W[M-1] |
+
+---
+
+**Q: What are the benefits of RBMs compared to other networks?**
+
+A few key advantages:
+
+**Unsupervised / generative** — RBMs learn the joint distribution P(v, h) over data, so they can generate new samples and work without labels. Most discriminative networks (CNNs, MLPs) only learn P(label | input).
+
+**Energy-based probabilistic foundation** — the energy function gives a principled way to reason about uncertainty and to compare how "likely" different inputs are under the model.
+
+**Efficient training with CD** — Contrastive Divergence avoids backpropagating through time or deep chains. Each layer trains locally, which is biologically more plausible and avoids vanishing gradients.
+
+**Good feature initializers for deep nets** — historically, stacking RBMs (DBNs) gave much better weight initializations than random init, especially before BatchNorm and ReLU were widespread. Less relevant today.
+
+**Compact hidden representations** — the binary hidden units act as a learned feature detector; sparsity regularization (L1) produces interpretable, localized filters.
+
+**The main weaknesses** in comparison: training is slower and less stable than backprop, scaling to high-dimensional data (large images, long sequences) is hard, and modern VAEs/diffusion models have largely replaced them for generative tasks while Transformers dominate sequence modeling. RBMs remain interesting as a research tool and for understanding energy-based learning.
+
+---
+
+**Q: What are VAEs/diffusion models?**
+
+**VAEs (Variational Autoencoders)**
+
+An encoder compresses input → latent vector z, a decoder reconstructs input from z. The trick is forcing z to follow a known distribution (usually Gaussian) via a KL-divergence penalty. You can then sample z ~ N(0,1) and decode to generate new data. Training is end-to-end backprop, which makes them faster and easier to scale than RBMs.
+
+**Diffusion Models**
+
+Learn to reverse a noise process. Forward: gradually add Gaussian noise to data over T steps until it's pure noise. Reverse: train a neural net to predict and remove the noise step by step. At generation time, start from random noise and denoise T times. Current state of the art for image/audio/video generation (Stable Diffusion, DALL-E 3, Sora).
+
+**Compared to RBMs**
+
+| | RBM | VAE | Diffusion |
+|---|---|---|---|
+| Training | CD (local, approximate) | Backprop + KL loss | Backprop + noise prediction |
+| Generation | Gibbs sampling (slow) | Single decoder pass (fast) | T denoising steps (slow-ish) |
+| Quality | Moderate | Tends to blur | State of the art |
+| Theory | Energy-based | Variational Bayes | Score matching / SDE |
+
+---
+
+**Q: How does pyRBM compare to those?**
+
+Honestly, pyRBM is not competitive with VAEs or diffusion models for any practical generative task. It's a research/learning codebase, not a production tool.
+
+**What pyRBM does well**
+- Clean, readable implementation of the RBM family (BB, GB, GG, BG variants)
+- Stacking into deep nets and the RTRBM sequence model
+- CuPy support for GPU acceleration
+- Good for understanding energy-based learning from first principles
+
+**Where it falls short**
+
+| Capability | pyRBM | VAE/Diffusion |
+|---|---|---|
+| Image quality | Low — blurry reconstructions | High — sharp, diverse samples |
+| Sequence modeling | Short sequences only (RTRBM) | Long-range (Transformers) |
+| Scalability | Struggles beyond ~1024 visible units | Scales to megapixel images |
+| Training stability | CD is noisy, sensitive to hyperparams | Backprop is well-understood |
+| Ecosystem | Standalone, no autodiff | PyTorch/JAX — huge community |
+
+**The honest summary**
+
+pyRBM is best thought of as an educational implementation and a testbed for RBM research ideas. If the goal were to build a competitive character-level language model, a small Transformer (even a 1-layer one) would outperform the RTRBM in `Rnn.py` by a wide margin with far less tuning effort. The value here is in understanding the mechanics, not beating benchmarks.
+
+---
+
+**Q: Is the Transformer architecture possible to implement with pyRBM? Or is it too dependent on the basic learning algorithm?**
+
+**Short answer: no, not meaningfully.** The incompatibility is fundamental, not incidental.
+
+**Why Transformers don't fit**
+
+The core Transformer operations — `softmax(QK^T/√d)V`, layer norm, residuals — are deterministic and differentiable. They are trained end-to-end with backpropagation, which pyRBM has no support for at all. You'd need to add autodiff, and at that point you're rebuilding PyTorch inside pyRBM.
+
+CD (Contrastive Divergence) has no equivalent for these operations. It only works for the specific energy-based forward/backward pass of an RBM.
+
+**The interesting theoretical bridge**
+
+The closest legitimate connection is **modern Hopfield networks** (Ramsauer et al., 2020 — "Hopfield Networks is All You Need"). They showed that the Transformer attention mechanism is mathematically equivalent to the update rule of a continuous Hopfield network, which is itself an energy-based model closely related to RBMs. So the *theory* connects, but the *implementation* path still goes through backprop.
+
+**What you could realistically add to pyRBM**
+
+If the goal is better sequence modeling, more tractable extensions would be:
+
+| Extension | Effort | Benefit |
+|---|---|---|
+| Deeper RTRBM stacking | Low — fits existing `Stack` | More expressive hidden states |
+| Larger hidden layer + sparse regularization | Trivial | Better feature quality |
+| Hybrid: RBM features → linear classifier | Medium | Supervised tasks |
+
+Adding a full Transformer would mean replacing the learning algorithm entirely — at which point you no longer have an RBM codebase.
+
+---
+
+**Q: What is autodiff?**
+
+**Autodiff (automatic differentiation)** is a technique for computing exact derivatives of any computation expressed as code, automatically.
+
+**The core idea**
+
+Every computation is a chain of primitive operations (add, multiply, exp, etc.). Each primitive has a known derivative. Autodiff tracks these operations and applies the chain rule mechanically to compute the gradient of any output with respect to any input — exactly, not approximately.
+
+**Two modes**
+
+- **Forward mode** — propagates derivatives forward through the computation alongside the values. Efficient when inputs are few.
+- **Reverse mode** — propagates derivatives backward from the output. This is what neural networks use. You may know it as **backpropagation** — backprop is just reverse-mode autodiff applied to a loss function.
+
+**What it looks like in practice**
+
+```python
+# PyTorch — autodiff is implicit
+x = torch.tensor(2.0, requires_grad=True)
+y = x ** 3 + 2 * x        # forward pass builds a computation graph
+y.backward()               # reverse pass computes dy/dx automatically
+print(x.grad)              # → 14.0  (3x² + 2 at x=2)
+```
+
+pyRBM has none of this. Every gradient in pyRBM is derived analytically by hand (the CD update rules in `train.py`) and hard-coded for the specific RBM energy function. That's why you can't just plug in an attention layer — there's no machinery to differentiate through it.
+
+**Why it matters for Transformers**
+
+A Transformer has millions of parameters spread across many non-trivial operations. Deriving and coding all those gradients by hand would be essentially impossible. Autodiff frameworks (PyTorch, JAX, TensorFlow) make it trivial — you just define the forward pass and get gradients for free.
+
+---
+
+**Q: Can pyRBM be extended with autodiff support?**
+
+Yes, and it's actually a reasonable extension. The cleanest approach would be to replace the numpy/cupy arrays with PyTorch tensors, since PyTorch tensors carry gradient information transparently.
+
+**What would change**
+
+`rbm/matrix.py` is the central switching point — it currently picks numpy or cupy:
+
+```python
+# current
+import cupy as np
+
+# with autodiff
+import torch
+```
+
+Then `RbmState` (the weight matrices `w_hv`, `b_v`, `b_h`) would become `nn.Parameter` tensors, and the forward/backward passes in `entity.py` would be differentiable automatically.
+
+**What you'd keep**
+
+The CD training in `train.py` could stay as-is — CD is still a valid training signal, just now computed through differentiable ops. You'd gain the *option* to use backprop in addition, not a forced replacement.
+
+**What you'd gain**
+
+- Attention layers, layer norm, residuals — anything PyTorch supports — could be added and trained
+- Gradient flow through stacked layers (currently each layer trains independently)
+- Access to PyTorch optimizers (Adam, etc.) instead of the hand-coded momentum in `entity.py`
+
+**The compatibility shim already exists**
+
+`src/compat/torch.py` — pyRBM already has a PyTorch bridge. It's worth reading what's already there before designing anything new.
+
+**Rough effort estimate**
+
+| Task | Effort |
+|---|---|
+| Replace matrix backend with torch tensors | Medium |
+| Refactor `RbmState` → `nn.Module` | Medium |
+| Keep CD working on top of torch | Low |
+| Add new differentiable components (attention etc.) | Low once the above is done |
+
+It's a meaningful but tractable refactor — not a rewrite from scratch.