diff --git a/python/paddle/v2/fluid/layers/nn.py b/python/paddle/v2/fluid/layers/nn.py index d9fe1ca0aca0a15f2b9742cc7749bb03268d7371..ebcc914f6049ce2093bbd630d77773b86a9d779a 100644 --- a/python/paddle/v2/fluid/layers/nn.py +++ b/python/paddle/v2/fluid/layers/nn.py @@ -249,22 +249,23 @@ def dynamic_lstm(input, h_t & = o_t \odot act_h(c_t) - where the :math:`W` terms denote weight matrices (e.g. :math:`W_{xi}` is the matrix - of weights from the input gate to the input), :math:`W_{ic}, W_{fc}, W_{oc}` - are diagonal weight matrices for peephole connections. In our implementation, - we use vectors to reprenset these diagonal weight matrices. The :math:`b` terms - denote bias vectors (:math:`b_i` is the input gate bias vector), :math:`\sigma` - is the non-line activations, such as logistic sigmoid function, and - :math:`i, f, o` and :math:`c` are the input gate, forget gate, output gate, - and cell activation vectors, respectively, all of which have the same size as - the cell output activation vector :math:`h`. - - The :math:`\odot` is the element-wise product of the vectors. :math:`act_g` and :math:`act_h` - are the cell input and cell output activation functions and `tanh` is usually - used for them. :math:`\\tilde{c_t}` is also called candidate hidden state, - which is computed based on the current input and the previous hidden state. - - Set `use_peepholes` False to disable peephole connection. The formula + where the :math:`W` terms denote weight matrices (e.g. :math:`W_{xi}` is + the matrix of weights from the input gate to the input), :math:`W_{ic}, \ + W_{fc}, W_{oc}` are diagonal weight matrices for peephole connections. In + our implementation, we use vectors to reprenset these diagonal weight + matrices. The :math:`b` terms denote bias vectors (:math:`b_i` is the input + gate bias vector), :math:`\sigma` is the non-line activations, such as + logistic sigmoid function, and :math:`i, f, o` and :math:`c` are the input + gate, forget gate, output gate, and cell activation vectors, respectively, + all of which have the same size as the cell output activation vector :math:`h`. + + The :math:`\odot` is the element-wise product of the vectors. :math:`act_g` + and :math:`act_h` are the cell input and cell output activation functions + and `tanh` is usually used for them. :math:`\\tilde{c_t}` is also called + candidate hidden state, which is computed based on the current input and + the previous hidden state. + + Set `use_peepholes` to `False` to disable peephole connection. The formula is omitted here, please refer to the paper http://www.bioinf.jku.at/publications/older/2604.pdf for details.