jens/Rbm
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Rename weightDecay to l2Lambda

Browse Source 
Rbm::Params::weightDecay is now applied directly to the weights outside
the momentum recursion (previous commit), exactly like the newly-added
l1Lambda and matching pyRBM's l2_lambda -- same lambda*W gradient form,
same decoupled-from-momentum treatment. The two are functionally the
same mechanism in this codebase, so name it accordingly. Renamed the
Params field, its JSON key, and the two GUI references in
MainComponent.cpp that read/write it (the "weightDecayLabel" widget
identifier and its JUCE-generated marker comment are left as-is --
cosmetic, not part of the actual API).

Existing .prj files still have a "weightDecay" JSON key; fromJson()
no longer reads it, so it's silently ignored on load. Harmless today
since every current project has it set to 0.0 (confirmed: unchanged
poet.elf output, unchanged test-suite results). Not migrating the .prj
files themselves in this change.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016K8Gu7Qejd11JbdiHZqYAs
This commit is contained in:
jens
2026-07-27 15:41:01 +02:00
co-authored by Claude Sonnet 5
parent cb20060732
commit 0149cdf7ce
3 changed files with 11 additions and 11 deletions
Download Patch File Download Diff File Expand all files Collapse all files
source/MainComponent.cpp
+2 -2
View File
@@ -784,7 +784,7 @@ void MainComponent::labelTextChanged (Label* labelThatHasChanged)
else if (labelThatHasChanged == weightDecayLabel)
{
//[UserLabelCode_weightDecayLabel] -- add your label text handling code here..
m_pLayer->params().weightDecay = labelThatHasChanged->getText().getFloatValue();
m_pLayer->params().l2Lambda = labelThatHasChanged->getText().getFloatValue();
//[/UserLabelCode_weightDecayLabel]
}
else if (labelThatHasChanged == momentumLabel)
@@ -947,7 +947,7 @@ void MainComponent::updateControls()
rbmDoSampleVisibleToggleButton->setToggleState(m_pLayer->params().gibbsDoSampleVisible, dontSendNotification);
rbmDoSampleHidden->setToggleState(m_pLayer->params().gibbsDoSampleHidden, dontSendNotification);
weightDecayLabel->setText(String(m_pLayer->params().weightDecay), dontSendNotification);
weightDecayLabel->setText(String(m_pLayer->params().l2Lambda), dontSendNotification);
learningRateLabel->setText(String(m_pLayer->params().learningRate), dontSendNotification);
momentumLabel->setText(String(m_pLayer->params().momentum), dontSendNotification);
numVisibleLabel->setText(String(m_pLayer->numVisibleX()), dontSendNotification );
source/Rbm.cpp
+5 -5
View File
@@ -367,14 +367,14 @@ void Rbm::train(arma::mat const &batch, IListener* pListener)
m_bh += inc_bh;
m_whv += inc_whv;
// Weight decay and L1 applied directly to the weights, outside the
// momentum recursion above -- folding either into inc_whv would let
// momentum geometrically amplify its effective strength to roughly
// L2 and L1 applied directly to the weights, outside the momentum
// recursion above -- folding either into inc_whv would let momentum
// geometrically amplify its effective strength to roughly
// lambda/(1-momentum), well past the configured value. Neither
// touches the biases, matching pyRBM's state_adjust().
if (m_params.weightDecay != 0.0)
if (m_params.l2Lambda != 0.0)
{
m_whv -= m_params.learningRate*m_params.weightDecay*m_whv;
m_whv -= m_params.learningRate*m_params.l2Lambda*m_whv;
}
if (m_params.l1Lambda != 0.0)
{
source/Rbm.hpp
+4 -4
View File
@@ -27,7 +27,7 @@ public:
{
Params()
: learningRate(0.1)
, weightDecay(0.0)
, l2Lambda(0.0)
, l1Lambda(0.0)
, momentum(0.9)
, doGaussianHidden(false)
@@ -46,7 +46,7 @@ public:
{
std::cout << "Exporting Rbm::Params" << std::endl;
Json::Value params;
params["weightDecay"] = weightDecay;
params["l2Lambda"] = l2Lambda;
params["l1Lambda"] = l1Lambda;
params["learningRate"] = learningRate;
params["momentum"] = momentum;
@@ -66,7 +66,7 @@ public:
void fromJson(Json::Value params)
{
std::cout << "Importing Rbm::Params" << std::endl;
weightDecay = params.get("weightDecay", weightDecay).asDouble();
l2Lambda = params.get("l2Lambda", l2Lambda).asDouble();
l1Lambda = params.get("l1Lambda", l1Lambda).asDouble();
learningRate = params.get("learningRate", learningRate).asDouble();
momentum = params.get("momentum", momentum).asDouble();
@@ -81,7 +81,7 @@ public:
numEpochs = params.get("numEpochs", numEpochs).asUInt();
}
double weightDecay;
double l2Lambda;
double l1Lambda;
double learningRate;
double momentum;