Single-layer Deep stack (28x28->256), has saved weights and a larger
training batch than the other mnist variants (test run took noticeably
longer). Passes: reconstruction error 0.005 vs. mean-baseline 0.067.
9/12 pass.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016K8Gu7Qejd11JbdiHZqYAs
Single-layer Deep stack (28x28->32), weights load fine but genuinely
fail the reconstruction check: error 0.231 vs. mean-baseline 0.067,
worse than the 1.5x tolerance. Unlike norb_small_16h_v2/many (which fail
because no current-format weights exist at all), this one has weights
that load successfully but just don't reconstruct well -- training
params look ordinary (RaoBlackwell, 1000 epochs) and the training data
looks like normal-scale MNIST pixels, so this looks like a genuinely
under-trained or abandoned saved snapshot rather than a test bug.
7/11 pass.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016K8Gu7Qejd11JbdiHZqYAs
101-layer Deep stack (64->64 x100). Fails the same way as
norb_small_16h_v2: only legacy *.weights.dat files exist (the pre-split
combined-weights format current code never reads), no .w/.bh/.bv.dat for
any layer -- a pre-existing data gap, not a bug. 6/8 pass.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016K8Gu7Qejd11JbdiHZqYAs
2-layer DeepStack (9x9->24, 24->4), has saved weights and training data.
Passes: reconstruction error 0.0007 vs. mean-baseline 0.14 -- exercises
upDownPass through more than one layer for the first time in the suite.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016K8Gu7Qejd11JbdiHZqYAs
The old main.cpp (TEST target) called APIs that no longer exist
(DeepStack::load()/loadWeights()/loadTrainingBatch() with no args,
Layer::upDownPass()) -- it hadn't compiled against the current AStack/
Layer API in a long time and was a manual smoke-test script, not an
actual test suite.
Replace it with a small dependency-free runner matching this repo's
existing no-framework style: for each named project, load it, load
weights and training batch, run a full up-pass/down-pass reconstruction
via DeepStack::upDownPass, and check the reconstruction error is finite
and beats the trivial per-feature-mean baseline (a project-agnostic sanity
bound, rather than a hardcoded threshold). Prints PASS/FAIL per project.
Starting coverage: 1-hot, prims, norb_small_16h_v2, mnist_2. 3/4 pass;
norb_small_16h_v2 fails because it has no saved weight files at all
(only .prj/.training.dat/.test.dat) -- a pre-existing gap in that
fixture's data, not a bug introduced here.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016K8Gu7Qejd11JbdiHZqYAs
Repo root had ~35 groups of .prj/.dat files loose alongside the build
system and source, all sharing one flat namespace. Centralize the new
prj/<name>/ convention inside AStack::loadWeights/saveWeights/
loadTrainingBatch/saveTrainingBatch and StackCreator::fromFile/toFile
(via a new AStack::projectDir() helper) -- these already have access to
the project name, so callers (poet.cpp, the GUI) need no changes at all;
they keep passing the same base directory they always did.
Filenames inside each project folder are unchanged (e.g.
prj/mnist_2/mnist_2.prj, prj/mnist_2/mnist_2.Layer.0.w.dat) -- only the
directory moves. Added Matutils::ensureDir() (mkdir -p equivalent; no
std::filesystem in C++11) since ofstream won't create directories, used
wherever a project is saved for the first time.
File migration for existing projects is a separate commit.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016K8Gu7Qejd11JbdiHZqYAs
doGaussianHidden took priority over doRaoBlackwell when deciding h_states
for the positive-phase gradient, so a Gaussian-hidden RBM always got the
noisy sample (h_probs + randn) for dw/dbh even with doRaoBlackwell set --
ignoring the flag and adding avoidable gradient variance. Standard CD
practice (and pyRBM's cd_gaussian_gaussian) uses the mean for the
weight/bias gradient when Rao-Blackwellizing, regardless of unit type.
Reordered so doRaoBlackwell is checked first (mean, any unit type) and
only samples otherwise, dispatching via the sampleHidden() helper added
in the previous commit so Gaussian/binary hidden units are each sampled
correctly. Behavior-preserving for poet (doRaoBlackwell=true,
doGaussianHidden=false already took the mean branch); only changes
behavior for a Gaussian-hidden RBM trained with doRaoBlackwell enabled.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016K8Gu7Qejd11JbdiHZqYAs
Matutils::sample() always binary-thresholds (src > uniform(src)), but it
was the only sampler in the codebase and was called unconditionally on
h_probs/v_probs/miniBatch in several CD Gibbs-loop branches regardless
of doGaussianVisible/doGaussianHidden. Binary-thresholding a Gaussian
unit's continuous activation is meaningless -- it would corrupt any
Gaussian-visible/hidden RBM (image-domain experiments via the GUI or
TEST target); doesn't affect poet's plain BB-RBM path since both flags
are false there.
Add sample_gaussian() (mean + N(0,1) noise) alongside the existing
Bernoulli sample() in matutils.hpp, plus Rbm::sampleVisible/sampleHidden
helpers that dispatch to the right one per the RBM's configured type.
Replace every visible/hidden Gibbs-step sample() call in cd_jens (the
active path) and cd_hinton (compiled but currently unused, behind
USE_CD_HINTON) with the appropriate dispatch helper, and fix the same
issue in Rbm::train's doSampleBatch path.
Behavior is unchanged for any RBM with doGaussianVisible/doGaussianHidden
both false (confirmed: poet.elf f output identical before/after).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016K8Gu7Qejd11JbdiHZqYAs
Rbm::train had mini-batch chunks as the outer loop and epochs as the
inner loop: each fixed, never-reshuffled slice of the data got all
numEpochs gradient steps back-to-back before ever being revisited, so
"numEpochs" didn't mean "passes over the whole dataset" and training
was biased toward whatever data came last. Invert the nesting (epochs
outer, mini-batches inner, batch reshuffled via arma::shuffle at the
start of each epoch) so every epoch is an actual full pass over the
data in a fresh random order.
Also drop a redundant toHiddenProbs(v_states) call in cd_jens: the
positive-phase hidden probabilities were computed once unconditionally
and then discarded, recomputed a second time with identical input in
two of the three sampling branches. Same result, half the cost, on
every mini-batch of every epoch of every layer.
Both only affect training; inference (step_forward/generation) is
unchanged, confirmed by identical poet.elf f output before and after.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016K8Gu7Qejd11JbdiHZqYAs
Greedy argmax decoding in RbmListener::forward always produces the
exact same character sequence and quickly falls into short repeating
loops once the 5-character lookback state revisits a prior cycle.
Add an optional temperature argument (poet f <seed> [temperature]) that
switches decoding to RnnStack::sample_one_hot, which now does proper
categorical sampling (temperature-scaled, renormalized draw) instead of
the old per-code Bernoulli approach that could leave the result as a
non-one-hot probability vector. Also seed Armadillo's RNG in main(),
since it otherwise defaults to a fixed seed and every run would sample
identically.
Add docs/RNN_ARCHITECTURE.md documenting how the RnnStack/Layer stack
implements the RNN (context chaining across layers, training/generation
data flow, and the decoding behavior above).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016K8Gu7Qejd11JbdiHZqYAs
- enable linear hidden : instead of prob(v_to_h()): use toHiddenProbs()
- enable linear visible: instead of prob(h_to_v()): use toVisibleProbs()
- make v_to_h() and h_to_v() private and force to use toHiddenProbs() and toVisibleProbs()
- use cd_jens or cd_hinton. cd_hinton_hid_lineaer is not of use anymre, since it is not capable of CDn
jensandClaude Sonnet 5