[update] README.rbm.md: add Q&A on image/video and code generation architectures

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

README.rbm.md

@@ -188,3 +188,70 @@ The CD training in `train.py` could stay as-is — CD is still a valid training
 | Add new differentiable components (attention etc.) | Low once the above is done |
 
 It's a meaningful but tractable refactor — not a rewrite from scratch.
+
+---
+
+**Q: Which network architecture is commonly used for high-res image/video generation?**
+
+**Diffusion models with a U-Net or Transformer backbone** are the current standard.
+
+**U-Net (older, still widely used)**
+
+The original diffusion backbone. An encoder downsamples the image through convolutional blocks, a bottleneck processes the compressed representation, then a decoder upsamples back — with skip connections between matching encoder/decoder levels. Attention layers are injected at lower resolutions. Used in the original Stable Diffusion (v1, v2).
+
+**DiT — Diffusion Transformer (newer, becoming dominant)**
+
+Replaces the U-Net with a pure Transformer operating on image patches. Each patch is a token; the full attention mechanism can model long-range dependencies across the whole image. Sora (video), Stable Diffusion 3, and FLUX all use this architecture. Scales better with compute than U-Net.
+
+**For video specifically**
+
+The same DiT idea extended to space + time: treat each patch at each frame as a token, apply 3D attention. Sora's key insight was training a single DiT on videos of varying resolution and duration with no architectural changes — just more tokens.
+
+**Why not GANs?**
+
+GANs (Generator + Discriminator) dominated image generation around 2018–2021 (StyleGAN, BigGAN). They produce sharp images fast but suffer from training instability and mode collapse. Diffusion models surpassed them in quality and diversity around 2022 and have largely taken over.
+
+**Summary**
+
+| Architecture | Example models | Status |
+|---|---|---|
+| U-Net diffusion | SD 1.x, DALL-E 2 | Mature, widely deployed |
+| DiT diffusion | SD3, FLUX, Sora | Current frontier |
+| GAN | StyleGAN3 | Largely superseded |
+
+---
+
+**Q: Which network architecture is used for code generators, like Claude Code?**
+
+**Decoder-only Transformer** — the same architecture behind GPT, Claude, Gemini, and essentially every large language model used for code generation today.
+
+**How it works**
+
+Tokens (subword pieces of text/code) are fed in left to right. Each token attends to all previous tokens via **causal self-attention** (masked so it can't peek ahead). The model predicts the next token at each position. That's it — code generation is just next-token prediction applied to source code.
+
+**Why decoder-only wins for generation**
+
+- Encoder-decoder (like the original Transformer, T5) was designed for translation: encode a full input, then decode an output. Good for fixed input→output tasks.
+- Decoder-only generates autoregressively with no separate encoding step, which is simpler and scales better to very long contexts (entire files, repos).
+
+**What makes code models specifically good at code**
+
+Not the architecture — the training data. Models trained heavily on GitHub, Stack Overflow, and documentation learn syntax, APIs, idioms, and common patterns. The architecture is identical to a text model.
+
+**At scale**
+
+Claude, GPT-4, Gemini, and similar models are decoder-only Transformers with:
+- Billions to trillions of parameters
+- Context windows of 100k–1M+ tokens
+- Trained on mixed text + code corpora
+- Fine-tuned with RLHF / constitutional AI for instruction following
+
+**Summary relative to this conversation**
+
+| Architecture | Used for |
+|---|---|
+| Decoder-only Transformer | LLMs, code generation (Claude, GPT) |
+| DiT (Diffusion Transformer) | Image/video generation (Sora, FLUX) |
+| U-Net Diffusion | Image generation (SD 1.x) |
+| RBM / RTRBM | Energy-based learning, research |
+