[context33] - pyRBM port of context33.prj; predict next char from current context

Browse Source

Notebook reconstructs the C++ context33.prj configuration exactly:
  - shared-weights StackRnn (1 entity, reused every time step)
  - visible = [context(128) | x_t(40)] = 168 units, hidden = 128
  - all hyperparams from .prj: lr=0.05, momentum=0.5, epochs=1000,
    mini_batch=100, gibbs=3, rao_blackwell=True, weight_decay=0

Training data loaded directly from context33.training.dat (Armadillo format,
12 × 168): sensory one-hot chars decoded as 'XURQLIFB9651'.

Key design: training pairs are (h_t, x_{t+1}) not (h_{t-1}, x_t) —
context is first advanced by seeing x_t, then the model is trained to
predict the next character x_{t+1} from that context. Evaluation using
clamped Gibbs on h_t achieves 100% next-step prediction accuracy.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

...

This commit is contained in:

jens

2026-05-31 15:12:14 +02:00

co-authored by Claude Sonnet 4.6

parent 952045b9f1

commit 2a6b00db88

1 changed files with 572 additions and 0 deletions

Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL

Download Patch File Download Diff File Expand all files Collapse all files

context33.ipynb

+572

File diff suppressed because one or more lines are too long