Replace space ' ' with explicit SPACE='_' vocab token to disambiguate
padding from real spaces, and rewrite forward_step to take a single
character plus rolling state string instead of a reconstructed vc matrix,
producing predictions one character at a time per unit.
- Increase H_SIZE to 128, lr to 0.2, disable Rao-Blackwell
- Expand VOCAB to include lowercase and hyphen/period
- Fix batch_delay to always use delay=1 per unit
- forward_step now returns (vc, text) with predicted char per unit step
- Collect and print generated text in main inference loop
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Switch TEXT to "HALLO MAUSI! SUPER HASE!", H_SIZE to 64, lr to 0.1
- Extract v2char() helper from vc2char()
- Fix shift_left to use vocab_size() step instead of 1
- Re-enable model.train() and model.save() in main
- Clean up dead prediction loop code
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Add batch_delay() to shift visible input per unit index
- Unify forward_step() to work with combined vc matrix
- Fix split() to always slice on axis=1
- Add index param to Entity for readable naming
- Rename test_xor.py to xor.py, replace Mat with np.array
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- reduce num_epochs to 5000 and add momentum=0.9
- switch to OneHot encoding in Label constructor
- call fitter.init(0.1) before load for weight initialisation
- use train_labels as test set, lower hard-decode threshold to 0.5
- improve print output
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- reduce WIN to 3 and H_SIZE to 32 for faster iteration
- increase NUM_EPOCHS to 1000 and collapse NUM_ITERATIONS to a single pass
- add README_JayRnn.md with algorithm description and ASCII architecture diagrams
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Moved src/tests/ → tests/
- Added test_* functions with assertions to script-style test files
- Added main() to each so IDEs offer it as a separate run target from pytest
- Fixed cupy_test.py: remove spurious x_gpu += x_cpu, fix duplicate xlabel
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Add PATCH/STRIDE params; STRIDE=4 for 50% overlap (49 patches per image)
- Reconstruct 32x32 by averaging overlapping patch contributions (assemble_overlapping pattern from test_faces_sub_image)
- All cells updated to use PATCH/STRIDE throughout
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Tile the single sentence 500× to form a (500, T, 40) batch so matrix ops
are (500, 168) instead of (1, 168). Note: with identical rows the gradient
is mathematically equivalent to batch=1; real speedup requires diverse data.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
SENTENCE now accepts mixed case and strips non-vocab chars automatically.
Extended to ~200 chars using the Moby Dick opening passage.
Visualisation cells use adaptive tick spacing (max 40 labels) to handle
longer sequences without crowding.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Remove context33.training.dat loading (and read_armadillo import).
Training data is now a hardcoded 20-character sentence encoded with the
same 40-char vocabulary as moby_rnn.ipynb (space .!? A-Z 0-9).
SENTENCE = "MOBY DICK IS A WHALE"
Results: 95% reconstruction, 95% next-step prediction (one miss each
on the first character where context is zero).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The training.dat file stores samples LIFO (last added in the C++ GUI = row 0).
Reversing the rows restores chronological order, matching the C++ sequence.
Next-step prediction remains 100%: model predicts x_{t+1} from h_t correctly
in the C++ insertion order 1 5 6 9 B F I L Q R U X.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Notebook reconstructs the C++ context33.prj configuration exactly:
- shared-weights StackRnn (1 entity, reused every time step)
- visible = [context(128) | x_t(40)] = 168 units, hidden = 128
- all hyperparams from .prj: lr=0.05, momentum=0.5, epochs=1000,
mini_batch=100, gibbs=3, rao_blackwell=True, weight_decay=0
Training data loaded directly from context33.training.dat (Armadillo format,
12 × 168): sensory one-hot chars decoded as 'XURQLIFB9651'.
Key design: training pairs are (h_t, x_{t+1}) not (h_{t-1}, x_t) —
context is first advanced by seeing x_t, then the model is trained to
predict the next character x_{t+1} from that context. Evaluation using
clamped Gibbs on h_t achieves 100% next-step prediction accuracy.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Previously a single shared-weight Entity processed every time step. Now:
- Shared mode (1 layer via make_layer): original behaviour unchanged
- Unrolled mode (N layers via make_unrolled): layers[t] owns W_t, b_v_t, b_h_t
New API:
make_unrolled(T, sensory_size, h_size, ...) → list[Layer]
next_entity() → Entity for the upcoming step() call
current_entity() → Entity from the most recent step() call
is_shared → bool
moby_rnn.ipynb: switch Build model cell to make_unrolled(T=100); update
predict_next() to use rnn.next_entity() for position-correct Gibbs sampling
README_moby_rnn.md: redraw temporal-unrolling ASCII art showing per-position
weights W_t; update parameter count and checkpoint file listing
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- README_moby_rnn.md: single-step and temporally-unrolled ASCII diagrams,
configuration table, notebook cell guide, generation API docs, references
- moby_rnn.ipynb: move all constants (T, NUM_SEQ, EVAL_CHARS, PRED_CHARS,
N_GIBBS, SEED, SEQ_IDX) into the Build model cell under a Configuration
header; rename H_SIZE → CONTEXT_SIZE throughout
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Implements StackRnn: a cascaded/recurrent RBM where the visible layer at
each time step is the concatenation of the previous hidden state (context)
and the current sensory input — V[t] = [context | x_t]. Weights are
shared across time steps (RTRBM-style concatenation variant).
- stack_rnn.py: StackRnn with step(), reconstruct(), reset(), train(),
make_layer() factory; supports greedy layer-wise training over sequences
of shape (num_seq, T, sensory_size)
- test_rnn.py: single-layer, two-layer, and save/load tests
- moby_rnn.ipynb: character-level language model on Moby Dick; one-hot
encoding, clamped-Gibbs next-char prediction, free text generation,
hidden-state trace and character-distribution visualisations
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
jens
