<dd><p>LSTM Step Layer. It used in recurrent_group. The lstm equations are shown
<dd><p>LSTM Step Layer. This function is used only in recurrent_group.
as follow.</p>
The lstm equations are shown as follows.</p>
<divclass="math">
<divclass="math">
\[ \begin{align}\begin{aligned}i_t & = \sigma(W_{xi}x_{t} + W_{hi}h_{t-1} + W_{ci}c_{t-1} + b_i)\\f_t & = \sigma(W_{xf}x_{t} + W_{hf}h_{t-1} + W_{cf}c_{t-1} + b_f)\\c_t & = f_tc_{t-1} + i_t tanh (W_{xc}x_t+W_{hc}h_{t-1} + b_c)\\o_t & = \sigma(W_{xo}x_{t} + W_{ho}h_{t-1} + W_{co}c_t + b_o)\\h_t & = o_t tanh(c_t)\end{aligned}\end{align} \]</div>
\[ \begin{align}\begin{aligned}i_t & = \sigma(W_{x_i}x_{t} + W_{h_i}h_{t-1} + W_{c_i}c_{t-1} + b_i)\\f_t & = \sigma(W_{x_f}x_{t} + W_{h_f}h_{t-1} + W_{c_f}c_{t-1} + b_f)\\c_t & = f_tc_{t-1} + i_t tanh (W_{x_c}x_t+W_{h_c}h_{t-1} + b_c)\\o_t & = \sigma(W_{x_o}x_{t} + W_{h_o}h_{t-1} + W_{c_o}c_t + b_o)\\h_t & = o_t tanh(c_t)\end{aligned}\end{align} \]</div>
<p>The input of lstm step is <spanclass="math">\(Wx_t + Wh_{t-1}\)</span>, and user should use
<p>The input of lstm step is <spanclass="math">\(Wx_t + Wh_{t-1}\)</span>, and user should use
<codeclass="code docutils literal"><spanclass="pre">mixed</span></code> and <codeclass="code docutils literal"><spanclass="pre">full_matrix_projection</span></code> to calculate these
<codeclass="code docutils literal"><spanclass="pre">mixed</span></code> and <codeclass="code docutils literal"><spanclass="pre">full_matrix_projection</span></code> to calculate these
input vector.</p>
input vectors.</p>
<p>The state of lstm step is <spanclass="math">\(c_{t-1}\)</span>. And lstm step layer will do</p>
<p>The state of lstm step is <spanclass="math">\(c_{t-1}\)</span>. And lstm step layer will do</p>
<dd><p>LSTM Step Layer. It used in recurrent_group. The lstm equations are shown
<dd><p>LSTM Step Layer. This function is used only in recurrent_group.
as follow.</p>
The lstm equations are shown as follows.</p>
<divclass="math">
<divclass="math">
\[ \begin{align}\begin{aligned}i_t & = \sigma(W_{xi}x_{t} + W_{hi}h_{t-1} + W_{ci}c_{t-1} + b_i)\\f_t & = \sigma(W_{xf}x_{t} + W_{hf}h_{t-1} + W_{cf}c_{t-1} + b_f)\\c_t & = f_tc_{t-1} + i_t tanh (W_{xc}x_t+W_{hc}h_{t-1} + b_c)\\o_t & = \sigma(W_{xo}x_{t} + W_{ho}h_{t-1} + W_{co}c_t + b_o)\\h_t & = o_t tanh(c_t)\end{aligned}\end{align} \]</div>
\[ \begin{align}\begin{aligned}i_t & = \sigma(W_{x_i}x_{t} + W_{h_i}h_{t-1} + W_{c_i}c_{t-1} + b_i)\\f_t & = \sigma(W_{x_f}x_{t} + W_{h_f}h_{t-1} + W_{c_f}c_{t-1} + b_f)\\c_t & = f_tc_{t-1} + i_t tanh (W_{x_c}x_t+W_{h_c}h_{t-1} + b_c)\\o_t & = \sigma(W_{x_o}x_{t} + W_{h_o}h_{t-1} + W_{c_o}c_t + b_o)\\h_t & = o_t tanh(c_t)\end{aligned}\end{align} \]</div>
<p>The input of lstm step is <spanclass="math">\(Wx_t + Wh_{t-1}\)</span>, and user should use
<p>The input of lstm step is <spanclass="math">\(Wx_t + Wh_{t-1}\)</span>, and user should use
<codeclass="code docutils literal"><spanclass="pre">mixed</span></code> and <codeclass="code docutils literal"><spanclass="pre">full_matrix_projection</span></code> to calculate these
<codeclass="code docutils literal"><spanclass="pre">mixed</span></code> and <codeclass="code docutils literal"><spanclass="pre">full_matrix_projection</span></code> to calculate these
input vector.</p>
input vectors.</p>
<p>The state of lstm step is <spanclass="math">\(c_{t-1}\)</span>. And lstm step layer will do</p>
<p>The state of lstm step is <spanclass="math">\(c_{t-1}\)</span>. And lstm step layer will do</p>