jensandClaude Sonnet 5
656a0252d3
Greedy argmax decoding in RbmListener::forward always produces the exact same character sequence and quickly falls into short repeating loops once the 5-character lookback state revisits a prior cycle. Add an optional temperature argument (poet f <seed> [temperature]) that switches decoding to RnnStack::sample_one_hot, which now does proper categorical sampling (temperature-scaled, renormalized draw) instead of the old per-code Bernoulli approach that could leave the result as a non-one-hot probability vector. Also seed Armadillo's RNG in main(), since it otherwise defaults to a fixed seed and every run would sample identically. Add docs/RNN_ARCHITECTURE.md documenting how the RnnStack/Layer stack implements the RNN (context chaining across layers, training/generation data flow, and the decoding behavior above). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016K8Gu7Qejd11JbdiHZqYAs
49 lines
1.1 KiB
C++
49 lines
1.1 KiB
C++
/*
|
|
* To change this license header, choose License Headers in Project Properties.
|
|
* To change this template file, choose Tools | Templates
|
|
* and open the template in the editor.
|
|
*/
|
|
|
|
/*
|
|
* File: RnnStack.hpp
|
|
* Author: jens
|
|
*
|
|
* Created on 17. Januar 2022, 12:43
|
|
*/
|
|
|
|
#ifndef RNNSTACK_HPP
|
|
#define RNNSTACK_HPP
|
|
|
|
#include "AStack.hpp"
|
|
#include "matutils.hpp"
|
|
|
|
class RnnStack : public AStack
|
|
{
|
|
const size_t NUM_CODES = 37;
|
|
|
|
public:
|
|
RnnStack(const std::string &name);
|
|
RnnStack(const RnnStack& orig) = delete;
|
|
virtual ~RnnStack();
|
|
|
|
size_t getSeqLen();
|
|
|
|
void train(const arma::mat& batch, Rbm::IListener* pListener) override;
|
|
|
|
arma::mat v_to_vc(const arma::mat &v, const arma::mat &c) const;
|
|
arma::mat vc_to_v(const arma::mat &vc, size_t numContext) const;
|
|
arma::mat step_forward(arma::mat &state, const arma::mat &v);
|
|
|
|
void sample_one_hot(arma::mat &srcDst, double temperature=1.0);
|
|
void clamp_one_hot(arma::mat &srcDst);
|
|
arma::mat to_next(const arma::mat &v);
|
|
arma::mat to_curr(const arma::mat &v);
|
|
|
|
void setParams(Rbm::Params const ¶m);
|
|
|
|
private:
|
|
};
|
|
|
|
#endif /* RNNSTACK_HPP */
|
|
|