jens/Rbm
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Fix cd_jens positive phase using noisy sample instead of mean

Browse Source 
doGaussianHidden took priority over doRaoBlackwell when deciding h_states
for the positive-phase gradient, so a Gaussian-hidden RBM always got the
noisy sample (h_probs + randn) for dw/dbh even with doRaoBlackwell set --
ignoring the flag and adding avoidable gradient variance. Standard CD
practice (and pyRBM's cd_gaussian_gaussian) uses the mean for the
weight/bias gradient when Rao-Blackwellizing, regardless of unit type.

Reordered so doRaoBlackwell is checked first (mean, any unit type) and
only samples otherwise, dispatching via the sampleHidden() helper added
in the previous commit so Gaussian/binary hidden units are each sampled
correctly. Behavior-preserving for poet (doRaoBlackwell=true,
doGaussianHidden=false already took the mean branch); only changes
behavior for a Gaussian-hidden RBM trained with doRaoBlackwell enabled.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016K8Gu7Qejd11JbdiHZqYAs
This commit is contained in:
jens
2026-07-27 13:57:46 +02:00
co-authored by Claude Sonnet 5
parent eb29e33b81
commit 4cf5f15555
1 changed files with 6 additions and 7 deletions
Download Patch File Download Diff File Expand all files Collapse all files
source/Rbm.cpp
+6 -7
View File
@@ -265,18 +265,17 @@ void Rbm::cd_jens(arma::mat const &v_states, arma::mat &dw, arma::mat &dbh, arma
arma::mat h_probs = toHiddenProbs(v_states);
arma::mat h_states;
// Sample hidden
if (m_params.doGaussianHidden)
{
h_states = h_probs + arma::randn(h_probs.n_rows, h_probs.n_cols);
}
else if (m_params.doRaoBlackwell)
// Sample hidden. Rao-Blackwellization uses the mean for the positive-phase
// gradient (lower variance) regardless of unit type; previously a Gaussian
// hidden unit always got the noisy sample here even with doRaoBlackwell
// set, ignoring the flag and adding avoidable gradient variance.
if (m_params.doRaoBlackwell)
{
h_states = h_probs;
}
else
{
h_states = sample(h_probs);
h_states = sampleHidden(h_probs);
}
// Update weights (positive phase)