ONNX is the acronym of Open Neural Network Exchange. The project is aimed to make a full communication and usage among different neural network development frameworks.
You will learn how to develop programs in local environment under the guidelines of this document.
## Requirements of coding
- Please refer to the coding comment format of [Doxygen](http://www.stack.nl/~dimitri/doxygen/)
- Please refer to the coding comment format of [Doxygen](http://www.stack.nl/~dimitri/doxygen/)
- Make sure that option of builder `WITH_STYLE_CHECK` is on and the build could pass through the code style check.
- Unit test is needed for all codes.
- Pass through all unit tests.
...
...
@@ -26,7 +26,7 @@ Clone remote git to local:
## Create local branch
At present [Git stream branch model](http://nvie.com/posts/a-successful-git-branching-model/) is applied to Paddle to undergo task of development,test,release and maintenance.Please refer to [branch regulation of Paddle](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/design/releasing_process.md#paddle-分支规范) about details。
At present [Git stream branch model](http://nvie.com/posts/a-successful-git-branching-model/) is applied to Paddle to undergo task of development,test,release and maintenance.Please refer to [branch regulation of Paddle](https://github.com/PaddlePaddle/FluidDoc/blob/develop/doc/fluid/design/others/releasing_process.md) about details。
All development tasks of feature and bug fix should be finished in a new branch which is extended from `develop` branch.
...
...
@@ -80,7 +80,7 @@ no changes added to commit (use "git add" and/or "git commit -a")
It needs a variety of development tools to build PaddlePaddle source code and generate documentation. For convenience, our standard development procedure is to put these tools together into a Docker image,called *development mirror* , usually named as `paddle:latest-dev` or `paddle:[version tag]-dev`,such as `paddle:0.11.0-dev` . Then all that need `cmake && make` ,such as IDE configuration,are replaced by `docker run paddle:latest-dev` .
You need to bulid this development mirror under the root directory of source code directory tree
You need to bulid this development mirror under the root directory of source code directory tree
```bash
➜ docker build -t paddle:latest-dev .
...
...
@@ -110,7 +110,7 @@ Run all unit tests with following commands:
➜ docker run -it-v$(pwd):/paddle paddle:latest-dev bash -c"cd /paddle/build && ctest"
```
Please refer to [Installation and run with Docker](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/v2/build_and_install/docker_install_cn.rst) about more information of construction and test.
Please refer to [Installation and run with Docker](../../../beginners_guide/install/install_Docker.html) about more information of construction and test.
For further information, please refer to: [multi_devices](https://github.com/PaddlePaddle/FluidDoc/tree/develop/doc/fluid/design/multi_devices) , [scope](https://github.com/PaddlePaddle/FluidDoc/Blob/develop/doc/fluid/design/concepts/scope.md) , [Developer's_Guide_to_Paddle_Fluid](https://github.com/PaddlePaddle/FluidDoc/blob/develop/doc/fluid/getstarted/Developer's_Guide_to_Paddle_Fluid.md)
For further information, please refer to: [multi_devices](https://github.com/PaddlePaddle/FluidDoc/tree/develop/doc/fluid/design/multi_devices) , [scope](https://github.com/PaddlePaddle/FluidDoc/Blob/develop/doc/fluid/design/concepts/scope.md) , [Developer's_Guide_to_Paddle_Fluid](https://github.com/PaddlePaddle/FluidDoc/blob/release/1.2/doc/fluid/getstarted/Developer's_Guide_to_Paddle_Fluid.md)
### 2.Op's registration logic
The registration entries for each Operator include:
...
...
@@ -67,7 +67,7 @@ The registration entries for each Operator include:
<tr>
<td>OpCreator </td>
<td>Functor </td>
<td>Create a new OperatorBase for each call </td>
<td>Create a new OperatorBase for each call </td>
<td>Call at runtime </td>
</tr>
</tbody>
...
...
@@ -150,7 +150,7 @@ The calculation speed of Op is related to the amount of data input. For some Op,
Since the call of CUDA Kernel has a certain overhead, multiple calls of the CUDA Kernel in Op may affect the execution speed of Op. For example, the previous sequence_expand_op contains many CUDA Kernels. Usually, these CUDA Kernels process a small amount of data, so frequent calls to such Kernels will affect the calculation speed of Op. In this case, it is better to combine these small CUDA Kernels into one. This idea is used in the optimization of the sequence_expand_op procedure (related PR[#9289](https://github.com/PaddlePaddle/Paddle/pull/9289)). The optimized sequence_expand_op is about twice as fast as the previous implementation, the relevant experiments are introduced in the PR ([#9289](https://github.com/PaddlePaddle/Paddle/pull/9289)).
Reduce the number of copy and sync operations between the CPU and the GPU. For example, the fetch operation will update the model parameters and get a loss after each iteration, and the copy of the data from the GPU to the Non-Pinned-Memory CPU is synchronous, so frequent fetching for multiple parameters will reduce the model training speed.
Reduce the number of copy and sync operations between the CPU and the GPU. For example, the fetch operation will update the model parameters and get a loss after each iteration, and the copy of the data from the GPU to the Non-Pinned-Memory CPU is synchronous, so frequent fetching for multiple parameters will reduce the model training speed.
@@ -16,7 +16,7 @@ gperftool mainly supports four functions:
- heap-profiling using tcmalloc
- CPU profiler
Paddle also provides a [tutorial on CPU performance analysis](https://github.com/PaddlePaddle/FluidDoc/blob/develop/doc/fluid/howto/optimization/cpu_profiling_en.md) based on gperftool.
Paddle also provides a [tutorial on CPU performance analysis](./cpu_profiling_en.html) based on gperftool.
For the analysis for heap, we mainly use thread-caching malloc and heap-profiling using tcmalloc.
...
...
@@ -29,7 +29,7 @@ This tutorial is based on the Docker development environment paddlepaddle/paddle
- Install google-perftools
```
apt-get install libunwind-dev
apt-get install libunwind-dev
apt-get install google-perftools
```
...
...
@@ -74,15 +74,15 @@ As the program runs, a lot of files will be generated in the perf_log folder as
```
The command above will generate a file of profile00x.pdf, which can be opened directly, for example, [memory_cpu_allocator](https://github.com/jacquesqiao/Paddle/blob/bd2ea0e1f84bb6522a66d44a072598153634cade/doc/fluid/howto/optimization/memory_cpu_allocator.pdf). As demonstrated in the chart below, during the running of the CPU version fluid, the module CPUAllocator is allocated with most memory. Other modules are allocated with relatively less memory, so they are ignored. It is very inconvenient for inspecting memory leak for memory leak is a chronic process which cannot be inspected in this picture.
- Diff mode. You can do diff on the heap at two moments, which removes some modules whose memory allocation has not changed, and displays the incremental part.
The generated result: [`memory_leak_protobuf`](https://github.com/jacquesqiao/Paddle/blob/bd2ea0e1f84bb6522a66d44a072598153634cade/doc/fluid/howto/optimization/memory_leak_protobuf.pdf)
As shown from the figure: The structure of ProgramDesc has increased by 200MB+ between the two versions, so there is a large possibility that memory leak happens here, and the final result does prove a leak here.
@@ -114,7 +114,7 @@ After completing the installation, you will also need to do:
In the docker container that compiles the code, or the corresponding location in the host machine:
- Run the script `paddle/scripts/paddle_build.sh` (under Paddle repo)
```bash
# Compile paddle's python library
cd Paddle
...
...
@@ -129,7 +129,7 @@ After completing the installation, you will also need to do:
docker run -it -v /Users/xxxx/workspace/paddlepaddle_workplace:/workplace -p 8000:8000 [images_id] /bin/bash
```
> Where `/Users/xxxx/workspace/paddlepaddle_workplace` should be replaced with your local host paddle workspace directory, `/workplace` should be replaced with the working directory in the docker. This mapping will ensure that we compile the python library, modify FluidDoc and use the preview tool at the same time.
> [images_id] is the image id of the paddlepaddle you use in docker.
...
...
@@ -158,7 +158,7 @@ After completing the installation, you will also need to do:
```
---
**Preview modification**
...
...
@@ -166,26 +166,26 @@ After completing the installation, you will also need to do:
Open your browser and navigate to http://localhost:8000 .
On the page to be updated, click Refresh Content at the top right corner.
After entering documentation page, the API section does not contain content. To preview the API document, please click on the API directory and you will see the generated API reference after a few minutes.
## Submit changes
If you wish to modify the code, please refer to [How to contribute code](contribute_to_paddle/index_en.html) under the `Paddle` repository.
If you wish to modify the code, please refer to [How to contribute code](../development/contribute_to_paddle/index_en.html) under the `Paddle` repository.
If you just modify the document:
- The modified content is in the `doc` folder, you only need to submit `PR` in the `FluidDoc` repository.
- The modified content is in the `external` folder:
1. Submit the PR in the repostory you modified. This is because the `FluidDoc` repository is just a wrapper that brings together the links of other repositories (namely, the "submodules" in git context).
2. When your changes are approved, update the corresponding `submodule` in FluidDoc to the latest commit-id of the source repository.
> For example, you updated the document on the develop branch in the book repository:
> - Go to the `FluidDoc/external/book` directory
> - Update commit-id to the latest commit: `git pull origin develop`
...
...
@@ -193,7 +193,7 @@ If you just modify the document:
3. Pull Request for your changes in the `FluidDoc` repository
The steps to submit changes and PR can refer to [How to contribute code](contribute_to_paddle/index_en.html)
The steps to submit changes and PR can refer to [How to contribute code](../development/contribute_to_paddle/index_en.html)
> It's not difficult to understand Docker. It takes about ten minutes to read this [article](https://zhuanlan.zhihu.com/p/19902938).
> It's not difficult to understand Docker. It takes about ten minutes to read this [article](https://zhuanlan.zhihu.com/p/19902938).
This can save you an hour of installing and configuring various development tools, as well as the need for new installations when switching machines. Don't forget that PaddlePaddle updates may lead to the need for new development tools. Not to mention the benefits of simplifying the recurrence of problems.
- Can I use an IDE?
> Of course, because the source code is on the machine. By default, the IDE calls a program like make to compile the source code. We only need to configure the IDE to call the Docker command to compile the source code.
Many PaddlePaddle developers use Emacs. They add two lines to their `~/.emacs` configuration file.
> Of course, because the source code is on the machine. By default, the IDE calls a program like make to compile the source code. We only need to configure the IDE to call the Docker command to compile the source code.
Many PaddlePaddle developers use Emacs. They add two lines to their `~/.emacs` configuration file.
You can start the compilation by pressing `Ctrl-C` and` c`.
- Can I compile in parallel?
> Yes. Our Docker image runs a [Bash script](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/paddle/scripts/paddle_build.sh). This script calls `make -j$(nproc)` to start as many processes as the CPU cores to compile in parallel.
> Yes. Our Docker image runs a [Bash script](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/scripts/paddle_build.sh). This script calls `make -j$(nproc)` to start as many processes as the CPU cores to compile in parallel.
PaddlePaddle supports two BLAS libraries, [MKL](https://software.intel.com/en-us/mkl) and [OpenBlAS](http://www.openblas.net/). MKL is used by default. If you use MKL and the machine contains the AVX2 instruction set, you will also download the MKL-DNN math library, for details please refer to [here](https://github.com/PaddlePaddle/Paddle/tree/develop/doc/design/mkldnn#cmake).
PaddlePaddle supports two BLAS libraries, [MKL](https://software.intel.com/en-us/mkl) and [OpenBlAS](http://www.openblas.net/). MKL is used by default. If you use MKL and the machine contains the AVX2 instruction set, you will also download the MKL-DNN math library, for details please refer to [here](https://github.com/PaddlePaddle/Paddle/tree/release/0.11.0/doc/design/mkldnn#cmake).
If you close MKL, OpenBLAS will be used as the BLAS library.
...
...
@@ -565,7 +565,7 @@ You can find the docker image for each release of PaddlePaddle in the [DockerHub
***
Suppose you have written a PaddlePaddle program in the current directory (such as /home/work): `train.py` ( refer to [PaddlePaddleBook](http://www.paddlepaddle.org/docs/develop/book/01.fit_a_line/index.cn.html) to write), you can start the training with the following command:
Suppose you have written a PaddlePaddle program in the current directory (such as /home/work): `train.py` ( refer to [PaddlePaddleBook](https://github.com/PaddlePaddle/book/blob/develop/01.fit_a_line/README.cn.md) to write), you can start the training with the following command:
cd /home/work
...
...
@@ -629,7 +629,7 @@ In order to ensure that the GPU driver works properly in the image, we recommend
**About AVX:**
AVX is a set of CPU instructions that speeds up the calculation of PaddlePaddle. The latest PaddlePaddle Docker image is enabled by default for AVX compilation, so if your computer does not support AVX, you need to [compile](/build_from_source_cn.html) PaddlePaddle to no-avx version separately.
AVX is a set of CPU instructions that speeds up the calculation of PaddlePaddle. The latest PaddlePaddle Docker image is enabled by default for AVX compilation, so if your computer does not support AVX, you need to compile PaddlePaddle to no-avx version separately.
The following instructions can check if the Linux computer supports AVX:
@@ -11,13 +11,13 @@ This document will instruct you to program and create a simple nueral network wi
Before building model, you need to figure out several core concepts of Fluid at first:
## Express data with Tensor
## Express data with Tensor
Like other mainstream frameworks, Fluid uses Tensor to hold data.
All data transferred in neural network are Tensor which can simply be regarded as a multi-dimensional array. In general, the number of dimensions can be any. Tensor features its own data type and shape. Data type of each element in single Tensor is the same. And **the shape of Tensor** refers to the dimensions of Tensor.
Picture below visually shows Tensor with dimension from one to six:
Picture below visually shows Tensor with dimension from one to six:
@@ -46,21 +46,21 @@ y = fluid.layers.fc(input=x, size=128, bias_attr=True)
**2. Input and Output Tensor**
The input data of the whole neural network is also a special Tensor in which the sizes of some dimensions can not be decided at the definition time of models. Such dimensions usually includes batch size, or width and height of image when such data formats in a mini-batch are not constant. Placeholders for these uncertain dimension are necessary at the definition phase of model.
The input data of the whole neural network is also a special Tensor in which the sizes of some dimensions can not be decided at the definition time of models. Such dimensions usually includes batch size, or width and height of image when such data formats in a mini-batch are not constant. Placeholders for these uncertain dimension are necessary at the definition phase of model.
`fluid.layers.data` is used to receive input data in Fluid, and it needs to be provided with the shape of input Tensor. When the shape is not certain, the correspondent dimension is defined as None.
`fluid.layers.data` is used to receive input data in Fluid, and it needs to be provided with the shape of input Tensor. When the shape is not certain, the correspondent dimension is defined as None.
The code below exemplifies the usage of `fluid.layers.data` :
```python
importpaddle.fluidasfluid
#Define the dimension of x : [3,None]. What we could make sure is that the first dimension of x is 3.
#Define the dimension of x : [3,None]. What we could make sure is that the first dimension of x is 3.
#The second dimension is unknown and can only be known at runtime.
Fluid is different from most other deep learning frameworks. In Fluid, static computing map is replaced by Program to dynamically describe the network. This dynamic method delivers both flexible modifications to network structure and convenience to build model. Moreover, the capability of expressing a model is enhanced significantly while the performance is guaranteed.
...
...
@@ -222,10 +222,10 @@ with fluid.layers.control_flow.Switch() as switch:
Now the network can predict output. Although the output is just a group of random numbers, which is far from expected results:
```python
#load library
import paddle.fluid as fluid
...
...
@@ -307,9 +307,9 @@ Firstly, define input data format, model structure,loss function and optimized a
#observe result
print outs
```
Output:
```
[array([[0.74079144],
[1.4815829 ],
...
...
@@ -317,8 +317,8 @@ Firstly, define input data format, model structure,loss function and optimized a
[2.9631658 ]], dtype=float32)]
```
4. Add loss function
4. Add loss function
After the construction of model, we need to evaluate the output result in order to make accurate predictions. How do we evaluate the result of prediction? We usually add loss function to network to compute the *distance* between ground-truth value and predict value.
In this example, we adopt [mean-square function](https://en.wikipedia.org/wiki/Mean_squared_error) as our loss function ：
...
...
@@ -328,7 +328,7 @@ Firstly, define input data format, model structure,loss function and optimized a
avg_cost = fluid.layers.mean(cost)
```
Output predicted value and loss function after a process of computing:
```python
#load library
import paddle.fluid as fluid
...
...
@@ -355,7 +355,7 @@ Firstly, define input data format, model structure,loss function and optimized a
print outs
```
Output:
```
[array([[0.9010564],
[1.8021128],
...
...
@@ -364,17 +364,17 @@ Firstly, define input data format, model structure,loss function and optimized a
```
We discover that the loss function after the first iteration of computing is 9.0, which shows there is a great improve space.
5. Optimization of network
After the definition of loss function,you can get loss value by forward computing and then get gradients of parameters with chain derivative method.
Parameters should be updated after you have obtained gradients. The simplest algorithm is random gradient algorithm: w=w−η⋅g,which is implemented by `fluid.optimizer.SGD`:
Now we discover that predicted value is nearly close to real value and the loss value descends from original value 9.05 to 0.01 after iteration for 100 times.
Congratulations! You have succeed to create a simple network. If you want to try advanced linear regression —— predict model of housing price, please read [linear regression](../../beginners_guide/quick_start/fit_a_line/README.en.html). More examples of model can be found in [models](../../user_guides/models/index_en.html).
