From 695651c78296ec1838d46ba9c3caf99e71320563 Mon Sep 17 00:00:00 2001 From: Travis CI Date: Wed, 12 Jul 2017 02:36:54 +0000 Subject: [PATCH] Deploy to GitHub Pages: c1ae339633a559abbac7b260c2dba0b281d9b6e2 --- develop/doc/_sources/howto/dev/new_layer_en.rst.txt | 8 ++++---- develop/doc/howto/dev/new_layer_en.html | 2 +- develop/doc_cn/_sources/howto/dev/new_layer_cn.rst.txt | 2 +- develop/doc_cn/howto/dev/new_layer_cn.html | 2 +- 4 files changed, 7 insertions(+), 7 deletions(-) diff --git a/develop/doc/_sources/howto/dev/new_layer_en.rst.txt b/develop/doc/_sources/howto/dev/new_layer_en.rst.txt index 46481f5ead3..110a9fb38f8 100644 --- a/develop/doc/_sources/howto/dev/new_layer_en.rst.txt +++ b/develop/doc/_sources/howto/dev/new_layer_en.rst.txt @@ -29,7 +29,7 @@ Fully connected layer takes a dense input vector with dimension :math:`D_i`. It where :math:`f(.)` is an nonlinear *activation* function, such as sigmoid, tanh, and Relu. -The transformation matrix :math:`W` and bias vector :math:`b` are the *parameters* of the layer. The *parameters* of a layer are learned during training in the *backward pass*. The backward pass computes the gradients of the output function with respect to all parameters and inputs. The optimizer can use chain rule to compute the gradients of the loss function with respect to each parameter. +The transformation matrix :math:`W` and bias vector :math:`b` are the *parameters* of the layer. The *parameters* of a layer are learned during training in the *backward pass*. The backward pass computes the gradients of the output function with respect to all parameters and inputs. The optimizer can use chain rule to compute the gradients of the loss function with respect to each parameter. Suppose our loss function is :math:`c(y)`, then @@ -37,7 +37,7 @@ Suppose our loss function is :math:`c(y)`, then \frac{\partial c(y)}{\partial x} = \frac{\partial c(y)}{\partial y} \frac{\partial y}{\partial x} -Suppose :math:`z = f(W^T x + b)`, then +Suppose :math:`z = W^T x + b`, then .. math:: @@ -48,7 +48,7 @@ This derivative can be automatically computed by our base layer class. Then, for fully connected layer, we need to compute: .. math:: - + \frac{\partial z}{\partial x} = W, \frac{\partial z_j}{\partial W_{ij}} = x_i, \frac{\partial z}{\partial b} = \mathbf 1 where :math:`\mathbf 1` is an all one vector, :math:`W_{ij}` is the number at the i-th row and j-th column of the matrix :math:`W`, :math:`z_j` is the j-th component of the vector :math:`z`, and :math:`x_i` is the i-th component of the vector :math:`x`. @@ -322,7 +322,7 @@ All the gradient check unit tests are located in :code:`paddle/gserver/tests/tes /* weight */ true); } } - + If you are creating a new file for the test, such as :code:`paddle/gserver/tests/testFCGrad.cpp`, you need to add the file to :code:`paddle/gserver/tests/CMakeLists.txt`. An example is given below. All the unit tests will run when you execute the command :code:`make tests`. Notice that some layers might need high accuracy for the gradient check unit tests to work well. You need to configure :code:`WITH_DOUBLE` to `ON` when configuring cmake. .. code-block:: bash diff --git a/develop/doc/howto/dev/new_layer_en.html b/develop/doc/howto/dev/new_layer_en.html index 12b91ebdd27..e715d6fa795 100644 --- a/develop/doc/howto/dev/new_layer_en.html +++ b/develop/doc/howto/dev/new_layer_en.html @@ -207,7 +207,7 @@ Fully connected layer takes a dense input vector with dimension \(c(y)\), then

\[\frac{\partial c(y)}{\partial x} = \frac{\partial c(y)}{\partial y} \frac{\partial y}{\partial x}\]
-

Suppose \(z = f(W^T x + b)\), then

+

Suppose \(z = W^T x + b\), then

\[\frac{\partial y}{\partial z} = \frac{\partial f(z)}{\partial z}\]

This derivative can be automatically computed by our base layer class.

diff --git a/develop/doc_cn/_sources/howto/dev/new_layer_cn.rst.txt b/develop/doc_cn/_sources/howto/dev/new_layer_cn.rst.txt index 9489a921c70..75037e693b3 100644 --- a/develop/doc_cn/_sources/howto/dev/new_layer_cn.rst.txt +++ b/develop/doc_cn/_sources/howto/dev/new_layer_cn.rst.txt @@ -37,7 +37,7 @@ \frac{\partial c(y)}{\partial x} = \frac{\partial c(y)}{\partial y} \frac{\partial y}{\partial x} -假设 :math:`z = f(W^T x + b)` ，那么 +假设 :math:`z = W^T x + b` ，那么 .. math:: diff --git a/develop/doc_cn/howto/dev/new_layer_cn.html b/develop/doc_cn/howto/dev/new_layer_cn.html index e49c69d3d0c..cdacb6f4a8d 100644 --- a/develop/doc_cn/howto/dev/new_layer_cn.html +++ b/develop/doc_cn/howto/dev/new_layer_cn.html @@ -207,7 +207,7 @@

假设损失函数是 \(c(y)\) ，那么

\[\frac{\partial c(y)}{\partial x} = \frac{\partial c(y)}{\partial y} \frac{\partial y}{\partial x}\]
-

假设 \(z = f(W^T x + b)\) ，那么

+

假设 \(z = W^T x + b\) ，那么

\[\frac{\partial y}{\partial z} = \frac{\partial f(z)}{\partial z}\]

PaddlePaddle的base layer类可以自动计算上面的导数。

-- GitLab