Skip to content

P
PaddleDetection

PaddlePaddle / PaddleDetection
大约 1 年 前同步成功

通知 695
Star 11112
Fork 2696
P
PaddleDetection

体验新版 GitCode，发现更多精彩内容 >>

提交 a0a67893 编写于 作者: N nhzlx
浏览文件
操作

Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into add_ut_for_trt

上级 0c511700 2bb8f5b1
展开全部 显示空白变更内容
内联 并排
Showing with 452 addition and 30736 deletion
.travis.yml
浏览文件 @ a0a67893
......@@ -12,7 +12,6 @@ services:
os:
- linux
env:
- JOB=doc
- JOB=check_style
- JOB=build_android
addons:
......
doc/fluid/CMakeLists.txt 已删除 100644 → 0
浏览文件 @ 0c511700
if(NOT DEFINED SPHINX_THEME)
set(SPHINX_THEME default)
endif()
if(NOT DEFINED SPHINX_THEME_DIR)
set(SPHINX_THEME_DIR)
endif()
# configured documentation tools and intermediate build results
set(BINARY_BUILD_DIR_EN "${CMAKE_CURRENT_BINARY_DIR}/en/_build")
# Sphinx cache with pickled ReST documents
set(SPHINX_CACHE_DIR_EN "${CMAKE_CURRENT_BINARY_DIR}/en/_doctrees")
# HTML output director
set(SPHINX_HTML_DIR_EN "${CMAKE_CURRENT_BINARY_DIR}/en/html")
set(IMPORT_PADDLE_STRING "")
set(IMPORT_PADDLEV2_STRING "")
configure_file(
"${CMAKE_CURRENT_SOURCE_DIR}/../templates/conf.py.en.in"
"${BINARY_BUILD_DIR_EN}/conf.py"
@ONLY)
sphinx_add_target(paddle_fluid_docs
html
${BINARY_BUILD_DIR_EN}
${SPHINX_CACHE_DIR_EN}
${CMAKE_CURRENT_SOURCE_DIR}
${SPHINX_HTML_DIR_EN})
# configured documentation tools and intermediate build results
set(BINARY_BUILD_DIR_CN "${CMAKE_CURRENT_BINARY_DIR}/cn/_build")
# Sphinx cache with pickled ReST documents
set(SPHINX_CACHE_DIR_CN "${CMAKE_CURRENT_BINARY_DIR}/cn/_doctrees")
# HTML output directory
set(SPHINX_HTML_DIR_CN "${CMAKE_CURRENT_BINARY_DIR}/cn/html")
configure_file(
"${CMAKE_CURRENT_SOURCE_DIR}/../templates/conf.py.cn.in"
"${BINARY_BUILD_DIR_CN}/conf.py"
@ONLY)
sphinx_add_target(paddle_fluid_docs_cn
html
${BINARY_BUILD_DIR_CN}
${SPHINX_CACHE_DIR_CN}
${CMAKE_CURRENT_SOURCE_DIR}
${SPHINX_HTML_DIR_CN})
add_subdirectory(api)
doc/fluid/api/CMakeLists.txt 已删除 100644 → 0
浏览文件 @ 0c511700
# configured documentation tools and intermediate build results
set(BINARY_BUILD_DIR_EN "${CMAKE_CURRENT_BINARY_DIR}/en/_build")
# Sphinx cache with pickled ReST documents
set(SPHINX_CACHE_DIR_EN "${CMAKE_CURRENT_BINARY_DIR}/en/_doctrees")
# HTML output director
set(SPHINX_HTML_DIR_EN "${CMAKE_CURRENT_BINARY_DIR}/en/html")
set(IMPORT_PADDLE_STRING "import paddle")
set(IMPORT_PADDLEV2_STRING "import paddle.v2")
configure_file(
"${CMAKE_CURRENT_SOURCE_DIR}/../../templates/conf.py.en.in"
"${BINARY_BUILD_DIR_EN}/conf.py"
@ONLY)
sphinx_add_target(paddle_fluid_apis
html
${BINARY_BUILD_DIR_EN}
${SPHINX_CACHE_DIR_EN}
${CMAKE_CURRENT_SOURCE_DIR}
${SPHINX_HTML_DIR_EN})
add_dependencies(paddle_fluid_apis gen_proto_py framework_py_proto copy_paddle_pybind paddle_python)
doc/fluid/api/average.rst 已删除 100644 → 0
浏览文件 @ 0c511700
.. THIS FILE IS GENERATED BY `gen_doc.{py|sh}`
!DO NOT EDIT THIS FILE MANUALLY!
=============
fluid.average
=============
.. _api_fluid_average_WeightedAverage:
WeightedAverage
---------------
.. autoclass:: paddle.fluid.average.WeightedAverage
:members:
:noindex:
doc/fluid/api/backward.rst 已删除 100644 → 0
浏览文件 @ 0c511700
.. THIS FILE IS GENERATED BY `gen_doc.{py|sh}`
!DO NOT EDIT THIS FILE MANUALLY!
==============
fluid.backward
==============
.. _api_fluid_backward_append_backward:
append_backward
---------------
.. autofunction:: paddle.fluid.backward.append_backward
:noindex:
.. _api_fluid_backward_calc_gradient:
calc_gradient
-------------
.. autofunction:: paddle.fluid.backward.calc_gradient
:noindex:
doc/fluid/api/clip.rst 已删除 100644 → 0
浏览文件 @ 0c511700
.. THIS FILE IS GENERATED BY `gen_doc.{py|sh}`
!DO NOT EDIT THIS FILE MANUALLY!
==========
fluid.clip
==========
.. _api_fluid_clip_ErrorClipByValue:
ErrorClipByValue
----------------
.. autoclass:: paddle.fluid.clip.ErrorClipByValue
:members:
:noindex:
.. _api_fluid_clip_GradientClipByValue:
GradientClipByValue
-------------------
.. autoclass:: paddle.fluid.clip.GradientClipByValue
:members:
:noindex:
.. _api_fluid_clip_GradientClipByNorm:
GradientClipByNorm
------------------
.. autoclass:: paddle.fluid.clip.GradientClipByNorm
:members:
:noindex:
.. _api_fluid_clip_GradientClipByGlobalNorm:
GradientClipByGlobalNorm
------------------------
.. autoclass:: paddle.fluid.clip.GradientClipByGlobalNorm
:members:
:noindex:
doc/fluid/api/data/data_reader.rst 已删除 100644 → 0
浏览文件 @ 0c511700
=====================
Data Reader Interface
=====================
DataTypes
=========
.. autofunction:: paddle.v2.data_type.dense_array
:noindex:
.. autofunction:: paddle.v2.data_type.integer_value
:noindex:
.. autofunction:: paddle.v2.data_type.integer_value_sequence
:noindex:
.. autofunction:: paddle.v2.data_type.integer_value_sub_sequence
:noindex:
.. autofunction:: paddle.v2.data_type.sparse_binary_vector
:noindex:
.. autofunction:: paddle.v2.data_type.sparse_binary_vector_sequence
:noindex:
.. autofunction:: paddle.v2.data_type.sparse_binary_vector_sub_sequence
:noindex:
.. autofunction:: paddle.v2.data_type.sparse_float_vector
:noindex:
.. autofunction:: paddle.v2.data_type.sparse_float_vector_sequence
:noindex:
.. autofunction:: paddle.v2.data_type.sparse_float_vector_sub_sequence
:noindex:
.. autofunction:: paddle.v2.data_type.sparse_non_value_slot
:noindex:
.. autofunction:: paddle.v2.data_type.sparse_value_slot
:noindex:
.. autoclass:: paddle.v2.data_type.InputType
:members:
:noindex:
DataFeeder
==========
.. automodule:: paddle.v2.data_feeder
:members:
:noindex:
Reader
======
.. automodule:: paddle.reader
:members:
:noindex:
.. automodule:: paddle.reader.creator
:members:
:noindex:
minibatch
=========
.. automodule:: paddle.v2.minibatch
:members:
:noindex:
doc/fluid/api/data/dataset.rst 已删除 100644 → 0
浏览文件 @ 0c511700
Dataset
=======
.. automodule:: paddle.dataset
:members:
:noindex:
mnist
+++++
.. automodule:: paddle.dataset.mnist
:members:
:noindex:
cifar
+++++
.. automodule:: paddle.dataset.cifar
:members:
:noindex:
conll05
+++++++
.. automodule:: paddle.dataset.conll05
:members: get_dict,get_embedding,test
:noindex:
imdb
++++
.. automodule:: paddle.dataset.imdb
:members:
:noindex:
imikolov
++++++++
.. automodule:: paddle.dataset.imikolov
:members:
:noindex:
movielens
+++++++++
.. automodule:: paddle.dataset.movielens
:members:
:noindex:
.. autoclass:: paddle.dataset.movielens.MovieInfo
:noindex:
.. autoclass:: paddle.dataset.movielens.UserInfo
:noindex:
sentiment
+++++++++
.. automodule:: paddle.dataset.sentiment
:members:
:noindex:
uci_housing
+++++++++++
.. automodule:: paddle.dataset.uci_housing
:members:
:noindex:
wmt14
+++++
.. automodule:: paddle.dataset.wmt14
:members:
:noindex:
wmt16
+++++
.. automodule:: paddle.dataset.wmt16
:members:
:noindex:
doc/fluid/api/data/image.rst 已删除 100644 → 0
浏览文件 @ 0c511700
Image Interface
===============
.. automodule:: paddle.v2.image
:members:
doc/fluid/api/data_feeder.rst 已删除 100644 → 0
浏览文件 @ 0c511700
.. THIS FILE IS GENERATED BY `gen_doc.{py|sh}`
!DO NOT EDIT THIS FILE MANUALLY!
=================
fluid.data_feeder
=================
.. _api_fluid_data_feeder_DataFeeder:
DataFeeder
----------
.. autoclass:: paddle.fluid.data_feeder.DataFeeder
:members:
:noindex:
doc/fluid/api/executor.rst 已删除 100644 → 0
浏览文件 @ 0c511700
.. THIS FILE IS GENERATED BY `gen_doc.{py|sh}`
!DO NOT EDIT THIS FILE MANUALLY!
==============
fluid.executor
==============
.. _api_fluid_executor_Executor:
Executor
--------
.. autoclass:: paddle.fluid.executor.Executor
:members:
:noindex:
.. _api_fluid_executor_global_scope:
global_scope
------------
.. autofunction:: paddle.fluid.executor.global_scope
:noindex:
.. _api_fluid_executor_scope_guard:
scope_guard
-----------
.. autofunction:: paddle.fluid.executor.scope_guard
:noindex:
.. _api_fluid_executor__switch_scope:
_switch_scope
-------------
.. autofunction:: paddle.fluid.executor._switch_scope
:noindex:
doc/fluid/api/fluid.rst 已删除 100644 → 0
浏览文件 @ 0c511700
.. THIS FILE IS GENERATED BY `gen_doc.{py|sh}`
!DO NOT EDIT THIS FILE MANUALLY!
=====
fluid
=====
.. _api_fluid_Block:
Block
-----
.. autoclass:: paddle.fluid.Block
:members:
:noindex:
.. _api_fluid_Variable:
Variable
--------
.. autoclass:: paddle.fluid.Variable
:members:
:noindex:
.. _api_fluid_Program:
Program
-------
.. autoclass:: paddle.fluid.Program
:members:
:noindex:
.. _api_fluid_Operator:
Operator
--------
.. autoclass:: paddle.fluid.Operator
:members:
:noindex:
.. _api_fluid_default_startup_program:
default_startup_program
-----------------------
.. autofunction:: paddle.fluid.default_startup_program
:noindex:
.. _api_fluid_default_main_program:
default_main_program
--------------------
.. autofunction:: paddle.fluid.default_main_program
:noindex:
.. _api_fluid_program_guard:
program_guard
-------------
.. autofunction:: paddle.fluid.program_guard
:noindex:
.. _api_fluid_get_var:
get_var
-------
.. autofunction:: paddle.fluid.get_var
:noindex:
.. _api_fluid_Executor:
Executor
--------
.. autoclass:: paddle.fluid.Executor
:members:
:noindex:
.. _api_fluid_global_scope:
global_scope
------------
.. autofunction:: paddle.fluid.global_scope
:noindex:
.. _api_fluid_scope_guard:
scope_guard
-----------
.. autofunction:: paddle.fluid.scope_guard
:noindex:
.. _api_fluid__switch_scope:
_switch_scope
-------------
.. autofunction:: paddle.fluid._switch_scope
:noindex:
.. _api_fluid_make_channel:
make_channel
------------
.. autofunction:: paddle.fluid.make_channel
:noindex:
.. _api_fluid_channel_send:
channel_send
------------
.. autofunction:: paddle.fluid.channel_send
:noindex:
.. _api_fluid_channel_recv:
channel_recv
------------
.. autofunction:: paddle.fluid.channel_recv
:noindex:
.. _api_fluid_channel_close:
channel_close
-------------
.. autofunction:: paddle.fluid.channel_close
:noindex:
.. _api_fluid_Select:
Select
------
.. autoclass:: paddle.fluid.Select
:members:
:noindex:
.. _api_fluid_Trainer:
Trainer
-------
.. autoclass:: paddle.fluid.Trainer
:members:
:noindex:
.. _api_fluid_BeginEpochEvent:
BeginEpochEvent
---------------
.. autoclass:: paddle.fluid.BeginEpochEvent
:members:
:noindex:
.. _api_fluid_EndEpochEvent:
EndEpochEvent
-------------
.. autoclass:: paddle.fluid.EndEpochEvent
:members:
:noindex:
.. _api_fluid_BeginStepEvent:
BeginStepEvent
--------------
.. autoclass:: paddle.fluid.BeginStepEvent
:members:
:noindex:
.. _api_fluid_EndStepEvent:
EndStepEvent
------------
.. autoclass:: paddle.fluid.EndStepEvent
:members:
:noindex:
.. _api_fluid_CheckpointConfig:
CheckpointConfig
----------------
.. autoclass:: paddle.fluid.CheckpointConfig
:members:
:noindex:
.. _api_fluid_Inferencer:
Inferencer
----------
.. autoclass:: paddle.fluid.Inferencer
:members:
:noindex:
.. _api_fluid_DistributeTranspiler:
DistributeTranspiler
--------------------
.. autoclass:: paddle.fluid.DistributeTranspiler
:members:
:noindex:
.. _api_fluid_memory_optimize:
memory_optimize
---------------
.. autofunction:: paddle.fluid.memory_optimize
:noindex:
.. _api_fluid_release_memory:
release_memory
--------------
.. autofunction:: paddle.fluid.release_memory
:noindex:
.. _api_fluid_ParallelExecutor:
ParallelExecutor
----------------
.. autoclass:: paddle.fluid.ParallelExecutor
:members:
:noindex:
.. _api_fluid_ExecutionStrategy:
ExecutionStrategy
-----------------
.. autoclass:: paddle.fluid.ExecutionStrategy
:members:
:noindex:
.. _api_fluid_BuildStrategy:
BuildStrategy
-------------
.. autoclass:: paddle.fluid.BuildStrategy
:members:
:noindex:
.. _api_fluid_create_lod_tensor:
create_lod_tensor
-----------------
.. autofunction:: paddle.fluid.create_lod_tensor
:noindex:
.. _api_fluid_create_random_int_lodtensor:
create_random_int_lodtensor
---------------------------
.. autofunction:: paddle.fluid.create_random_int_lodtensor
:noindex:
.. _api_fluid_LoDTensor:
LoDTensor
---------
.. autoclass:: paddle.fluid.LoDTensor
:members:
:noindex:
.. _api_fluid_CPUPlace:
CPUPlace
--------
.. autoclass:: paddle.fluid.CPUPlace
:members:
:noindex:
.. _api_fluid_CUDAPlace:
CUDAPlace
---------
.. autoclass:: paddle.fluid.CUDAPlace
:members:
:noindex:
.. _api_fluid_CUDAPinnedPlace:
CUDAPinnedPlace
---------------
.. autoclass:: paddle.fluid.CUDAPinnedPlace
:members:
:noindex:
.. _api_fluid_Tensor:
Tensor
------
.. autoclass:: paddle.fluid.Tensor
:members:
:noindex:
.. _api_fluid_ParamAttr:
ParamAttr
---------
.. autoclass:: paddle.fluid.ParamAttr
:members:
:noindex:
.. _api_fluid_WeightNormParamAttr:
WeightNormParamAttr
-------------------
.. autoclass:: paddle.fluid.WeightNormParamAttr
:members:
:noindex:
.. _api_fluid_DataFeeder:
DataFeeder
----------
.. autoclass:: paddle.fluid.DataFeeder
:members:
:noindex:
.. _api_fluid_Scope:
Scope
-----
.. autoclass:: paddle.fluid.Scope
:members:
:noindex:
doc/fluid/api/gen_doc.py 已删除 100644 → 0
浏览文件 @ 0c511700
# Copyright (c) 2018 PaddlePaddle Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
from __future__ import print_function
import argparse
import sys
import types
import paddle.fluid as fluid
def parse_arg():
parser = argparse.ArgumentParser()
parser.add_argument('--submodules', nargs="*")
parser.add_argument(
'module', type=str, help='Generate the documentation of which module')
return parser.parse_args()
class DocGenerator(object):
def __init__(self, module_name=None, stream=sys.stdout):
if module_name == "":
module_name = None
self.stream = stream
if module_name is None:
self.module_name = "fluid"
else:
self.module_name = "fluid." + module_name
if module_name is None:
self.module = fluid
else:
if not hasattr(fluid, module_name):
raise ValueError("Cannot find fluid.{0}".format(module_name))
else:
self.module = getattr(fluid, module_name)
self.stream.write('''.. THIS FILE IS GENERATED BY `gen_doc.{py|sh}`
!DO NOT EDIT THIS FILE MANUALLY!
''')
self._print_header_(self.module_name, dot='=', is_title=True)
def print_submodule(self, submodule_name):
submodule = getattr(self.module, submodule_name)
if submodule is None:
raise ValueError("Cannot find submodule {0}".format(submodule_name))
self.print_section(submodule_name)
for item in submodule.__all__:
self.print_item(item)
def print_current_module(self):
for item in self.module.__all__:
self.print_item(item)
def print_section(self, name):
self._print_header_(name, dot='=', is_title=False)
def print_item(self, name):
item = getattr(self.module, name, None)
if item is None:
return
if isinstance(item, types.TypeType):
self.print_class(name)
elif isinstance(item, types.FunctionType):
self.print_method(name)
else:
pass
def print_class(self, name):
self._print_ref_(name)
self._print_header_(name, dot='-', is_title=False)
self.stream.write('''.. autoclass:: paddle.{0}.{1}
:members:
:noindex:
'''.format(self.module_name, name))
def print_method(self, name):
self._print_ref_(name)
self._print_header_(name, dot='-', is_title=False)
self.stream.write('''.. autofunction:: paddle.{0}.{1}
:noindex:
'''.format(self.module_name, name))
def _print_header_(self, name, dot, is_title):
dot_line = dot * len(name)
if is_title:
self.stream.write(dot_line)
self.stream.write('\n')
self.stream.write(name)
self.stream.write('\n')
self.stream.write(dot_line)
self.stream.write('\n')
self.stream.write('\n')
def _print_ref_(self, name):
self.stream.write(".. _api_{0}_{1}:\n\n".format("_".join(
self.module_name.split(".")), name))
def main():
args = parse_arg()
gen = DocGenerator(args.module)
if args.submodules is None:
gen.print_current_module()
else:
for submodule_name in args.submodules:
gen.print_submodule(submodule_name)
if __name__ == '__main__':
main()
doc/fluid/api/gen_doc.sh 已删除 100755 → 0
浏览文件 @ 0c511700
#!/bin/bash
python gen_doc.py layers --submodules control_flow device io nn ops tensor learning_rate_scheduler detection metric_op tensor > layers.rst
for module in data_feeder clip metrics executor initializer io nets optimizer param_attr profiler regularizer transpiler recordio_writer backward average profiler
do
python gen_doc.py ${module} > ${module}.rst
done
python gen_doc.py "" > fluid.rst
doc/fluid/api/index_en.rst 已删除 100644 → 0
浏览文件 @ 0c511700
=============
API Reference
=============
.. toctree::
:maxdepth: 1
fluid.rst
layers.rst
data_feeder.rst
executor.rst
initializer.rst
metrics.rst
nets.rst
clip.rst
optimizer.rst
param_attr.rst
profiler.rst
regularizer.rst
io.rst
data.rst
transpiler.rst
recordio_writer.rst
backward.rst
average.rst
profiler.rst
doc/fluid/api/initializer.rst 已删除 100644 → 0
浏览文件 @ 0c511700
.. THIS FILE IS GENERATED BY `gen_doc.{py|sh}`
!DO NOT EDIT THIS FILE MANUALLY!
=================
fluid.initializer
=================
.. _api_fluid_initializer_Constant:
Constant
--------
.. autoclass:: paddle.fluid.initializer.Constant
:members:
:noindex:
.. _api_fluid_initializer_Uniform:
Uniform
-------
.. autoclass:: paddle.fluid.initializer.Uniform
:members:
:noindex:
.. _api_fluid_initializer_Normal:
Normal
------
.. autoclass:: paddle.fluid.initializer.Normal
:members:
:noindex:
.. _api_fluid_initializer_Normal:
TruncatedNormal
------
.. autoclass:: paddle.fluid.initializer.TruncatedNormal
:members:
:noindex:
.. _api_fluid_initializer_Xavier:
Xavier
------
.. autoclass:: paddle.fluid.initializer.Xavier
:members:
:noindex:
.. _api_fluid_initializer_Bilinear:
Bilinear
--------
.. autoclass:: paddle.fluid.initializer.Bilinear
:members:
:noindex:
.. _api_fluid_initializer_MSRA:
MSRA
----
.. autoclass:: paddle.fluid.initializer.MSRA
:members:
:noindex:
.. _api_fluid_initializer_force_init_on_cpu:
force_init_on_cpu
-----------------
.. autofunction:: paddle.fluid.initializer.force_init_on_cpu
:noindex:
.. _api_fluid_initializer_init_on_cpu:
init_on_cpu
-----------
.. autofunction:: paddle.fluid.initializer.init_on_cpu
:noindex:
.. _api_fluid_initializer_ConstantInitializer:
ConstantInitializer
-------------------
.. autoclass:: paddle.fluid.initializer.ConstantInitializer
:members:
:noindex:
.. _api_fluid_initializer_UniformInitializer:
UniformInitializer
------------------
.. autoclass:: paddle.fluid.initializer.UniformInitializer
:members:
:noindex:
.. _api_fluid_initializer_NormalInitializer:
NormalInitializer
-----------------
.. autoclass:: paddle.fluid.initializer.NormalInitializer
:members:
:noindex:
.. _api_fluid_initializer_XavierInitializer:
XavierInitializer
-----------------
.. autoclass:: paddle.fluid.initializer.XavierInitializer
:members:
:noindex:
.. _api_fluid_initializer_BilinearInitializer:
BilinearInitializer
-------------------
.. autoclass:: paddle.fluid.initializer.BilinearInitializer
:members:
:noindex:
.. _api_fluid_initializer_MSRAInitializer:
MSRAInitializer
---------------
.. autoclass:: paddle.fluid.initializer.MSRAInitializer
:members:
:noindex:
doc/fluid/api/io.rst 已删除 100644 → 0
浏览文件 @ 0c511700
.. THIS FILE IS GENERATED BY `gen_doc.{py|sh}`
!DO NOT EDIT THIS FILE MANUALLY!
========
fluid.io
========
.. _api_fluid_io_save_vars:
save_vars
---------
.. autofunction:: paddle.fluid.io.save_vars
:noindex:
.. _api_fluid_io_save_params:
save_params
-----------
.. autofunction:: paddle.fluid.io.save_params
:noindex:
.. _api_fluid_io_save_persistables:
save_persistables
-----------------
.. autofunction:: paddle.fluid.io.save_persistables
:noindex:
.. _api_fluid_io_load_vars:
load_vars
---------
.. autofunction:: paddle.fluid.io.load_vars
:noindex:
.. _api_fluid_io_load_params:
load_params
-----------
.. autofunction:: paddle.fluid.io.load_params
:noindex:
.. _api_fluid_io_load_persistables:
load_persistables
-----------------
.. autofunction:: paddle.fluid.io.load_persistables
:noindex:
.. _api_fluid_io_save_inference_model:
save_inference_model
--------------------
.. autofunction:: paddle.fluid.io.save_inference_model
:noindex:
.. _api_fluid_io_load_inference_model:
load_inference_model
--------------------
.. autofunction:: paddle.fluid.io.load_inference_model
:noindex:
.. _api_fluid_io_get_inference_program:
get_inference_program
---------------------
.. autofunction:: paddle.fluid.io.get_inference_program
:noindex:
.. _api_fluid_io_save_checkpoint:
save_checkpoint
---------------
.. autofunction:: paddle.fluid.io.save_checkpoint
:noindex:
.. _api_fluid_io_load_checkpoint:
load_checkpoint
---------------
.. autofunction:: paddle.fluid.io.load_checkpoint
:noindex:
.. _api_fluid_io_clean_checkpoint:
clean_checkpoint
----------------
.. autofunction:: paddle.fluid.io.clean_checkpoint
:noindex:
.. _api_fluid_io_load_persist_vars_without_grad:
load_persist_vars_without_grad
------------------------------
.. autofunction:: paddle.fluid.io.load_persist_vars_without_grad
:noindex:
.. _api_fluid_io_save_persist_vars_without_grad:
save_persist_vars_without_grad
------------------------------
.. autofunction:: paddle.fluid.io.save_persist_vars_without_grad
:noindex:
.. _api_fluid_io_get_latest_checkpoint_serial:
get_latest_checkpoint_serial
----------------------------
.. autofunction:: paddle.fluid.io.get_latest_checkpoint_serial
:noindex:
doc/fluid/api/layers.rst 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/api/metrics.rst 已删除 100644 → 0
浏览文件 @ 0c511700
.. THIS FILE IS GENERATED BY `gen_doc.{py|sh}`
!DO NOT EDIT THIS FILE MANUALLY!
=============
fluid.metrics
=============
.. _api_fluid_metrics_MetricBase:
MetricBase
----------
.. autoclass:: paddle.fluid.metrics.MetricBase
:members:
:noindex:
.. _api_fluid_metrics_CompositeMetric:
CompositeMetric
---------------
.. autoclass:: paddle.fluid.metrics.CompositeMetric
:members:
:noindex:
.. _api_fluid_metrics_Precision:
Precision
---------
.. autoclass:: paddle.fluid.metrics.Precision
:members:
:noindex:
.. _api_fluid_metrics_Recall:
Recall
------
.. autoclass:: paddle.fluid.metrics.Recall
:members:
:noindex:
.. _api_fluid_metrics_Accuracy:
Accuracy
--------
.. autoclass:: paddle.fluid.metrics.Accuracy
:members:
:noindex:
.. _api_fluid_metrics_ChunkEvaluator:
ChunkEvaluator
--------------
.. autoclass:: paddle.fluid.metrics.ChunkEvaluator
:members:
:noindex:
.. _api_fluid_metrics_EditDistance:
EditDistance
------------
.. autoclass:: paddle.fluid.metrics.EditDistance
:members:
:noindex:
.. _api_fluid_metrics_DetectionMAP:
DetectionMAP
------------
.. autoclass:: paddle.fluid.metrics.DetectionMAP
:members:
:noindex:
.. _api_fluid_metrics_Auc:
Auc
---
.. autoclass:: paddle.fluid.metrics.Auc
:members:
:noindex:
doc/fluid/api/nets.rst 已删除 100644 → 0
浏览文件 @ 0c511700
.. THIS FILE IS GENERATED BY `gen_doc.{py|sh}`
!DO NOT EDIT THIS FILE MANUALLY!
==========
fluid.nets
==========
.. _api_fluid_nets_simple_img_conv_pool:
simple_img_conv_pool
--------------------
.. autofunction:: paddle.fluid.nets.simple_img_conv_pool
:noindex:
.. _api_fluid_nets_sequence_conv_pool:
sequence_conv_pool
------------------
.. autofunction:: paddle.fluid.nets.sequence_conv_pool
:noindex:
.. _api_fluid_nets_glu:
glu
---
.. autofunction:: paddle.fluid.nets.glu
:noindex:
.. _api_fluid_nets_scaled_dot_product_attention:
scaled_dot_product_attention
----------------------------
.. autofunction:: paddle.fluid.nets.scaled_dot_product_attention
:noindex:
doc/fluid/api/optimizer.rst 已删除 100644 → 0
浏览文件 @ 0c511700
.. THIS FILE IS GENERATED BY `gen_doc.{py|sh}`
!DO NOT EDIT THIS FILE MANUALLY!
===============
fluid.optimizer
===============
.. _api_fluid_optimizer_SGD:
SGD
---
.. autoclass:: paddle.fluid.optimizer.SGD
:members:
:noindex:
.. _api_fluid_optimizer_Momentum:
Momentum
--------
.. autoclass:: paddle.fluid.optimizer.Momentum
:members:
:noindex:
.. _api_fluid_optimizer_Adagrad:
Adagrad
-------
.. autoclass:: paddle.fluid.optimizer.Adagrad
:members:
:noindex:
.. _api_fluid_optimizer_Adam:
Adam
----
.. autoclass:: paddle.fluid.optimizer.Adam
:members:
:noindex:
.. _api_fluid_optimizer_Adamax:
Adamax
------
.. autoclass:: paddle.fluid.optimizer.Adamax
:members:
:noindex:
.. _api_fluid_optimizer_DecayedAdagrad:
DecayedAdagrad
--------------
.. autoclass:: paddle.fluid.optimizer.DecayedAdagrad
:members:
:noindex:
.. _api_fluid_optimizer_Ftrl:
Ftrl
----
.. autoclass:: paddle.fluid.optimizer.Ftrl
:members:
:noindex:
.. _api_fluid_optimizer_SGDOptimizer:
SGDOptimizer
------------
.. autoclass:: paddle.fluid.optimizer.SGDOptimizer
:members:
:noindex:
.. _api_fluid_optimizer_MomentumOptimizer:
MomentumOptimizer
-----------------
.. autoclass:: paddle.fluid.optimizer.MomentumOptimizer
:members:
:noindex:
.. _api_fluid_optimizer_AdagradOptimizer:
AdagradOptimizer
----------------
.. autoclass:: paddle.fluid.optimizer.AdagradOptimizer
:members:
:noindex:
.. _api_fluid_optimizer_AdamOptimizer:
AdamOptimizer
-------------
.. autoclass:: paddle.fluid.optimizer.AdamOptimizer
:members:
:noindex:
.. _api_fluid_optimizer_AdamaxOptimizer:
AdamaxOptimizer
---------------
.. autoclass:: paddle.fluid.optimizer.AdamaxOptimizer
:members:
:noindex:
.. _api_fluid_optimizer_DecayedAdagradOptimizer:
DecayedAdagradOptimizer
-----------------------
.. autoclass:: paddle.fluid.optimizer.DecayedAdagradOptimizer
:members:
:noindex:
.. _api_fluid_optimizer_RMSPropOptimizer:
RMSPropOptimizer
----------------
.. autoclass:: paddle.fluid.optimizer.RMSPropOptimizer
:members:
:noindex:
.. _api_fluid_optimizer_FtrlOptimizer:
FtrlOptimizer
-------------
.. autoclass:: paddle.fluid.optimizer.FtrlOptimizer
:members:
:noindex:
.. _api_fluid_optimizer_Adadelta:
Adadelta
--------
.. autoclass:: paddle.fluid.optimizer.Adadelta
:members:
:noindex:
.. _api_fluid_optimizer_ModelAverage:
ModelAverage
------------
.. autoclass:: paddle.fluid.optimizer.ModelAverage
:members:
:noindex:
.. _api_fluid_optimizer_Optimizer:
Optimizer
---------
.. autoclass:: paddle.fluid.optimizer.Optimizer
:members:
:noindex:
.. _api_fluid_optimizer_RMSPropOptimizer:
RMSPropOptimizer
----------------
.. autoclass:: paddle.fluid.optimizer.RMSPropOptimizer
:members:
:noindex:
doc/fluid/api/param_attr.rst 已删除 100644 → 0
浏览文件 @ 0c511700
.. THIS FILE IS GENERATED BY `gen_doc.{py|sh}`
!DO NOT EDIT THIS FILE MANUALLY!
================
fluid.param_attr
================
.. _api_fluid_param_attr_ParamAttr:
ParamAttr
---------
.. autoclass:: paddle.fluid.param_attr.ParamAttr
:members:
:noindex:
.. _api_fluid_param_attr_WeightNormParamAttr:
WeightNormParamAttr
-------------------
.. autoclass:: paddle.fluid.param_attr.WeightNormParamAttr
:members:
:noindex:
doc/fluid/api/profiler.rst 已删除 100644 → 0
浏览文件 @ 0c511700
.. THIS FILE IS GENERATED BY `gen_doc.{py|sh}`
!DO NOT EDIT THIS FILE MANUALLY!
==============
fluid.profiler
==============
.. _api_fluid_profiler_cuda_profiler:
cuda_profiler
-------------
.. autofunction:: paddle.fluid.profiler.cuda_profiler
:noindex:
.. _api_fluid_profiler_reset_profiler:
reset_profiler
--------------
.. autofunction:: paddle.fluid.profiler.reset_profiler
:noindex:
.. _api_fluid_profiler_profiler:
profiler
--------
.. autofunction:: paddle.fluid.profiler.profiler
:noindex:
.. _api_fluid_profiler_start_profiler:
start_profiler
--------------
.. autofunction:: paddle.fluid.profiler.start_profiler
:noindex:
.. _api_fluid_profiler_stop_profiler:
stop_profiler
-------------
.. autofunction:: paddle.fluid.profiler.stop_profiler
:noindex:
doc/fluid/api/recordio_writer.rst 已删除 100644 → 0
浏览文件 @ 0c511700
.. THIS FILE IS GENERATED BY `gen_doc.{py|sh}`
!DO NOT EDIT THIS FILE MANUALLY!
=====================
fluid.recordio_writer
=====================
.. _api_fluid_recordio_writer_convert_reader_to_recordio_file:
convert_reader_to_recordio_file
-------------------------------
.. autofunction:: paddle.fluid.recordio_writer.convert_reader_to_recordio_file
:noindex:
.. _api_fluid_recordio_writer_convert_reader_to_recordio_files:
convert_reader_to_recordio_files
--------------------------------
.. autofunction:: paddle.fluid.recordio_writer.convert_reader_to_recordio_files
:noindex:
doc/fluid/api/regularizer.rst 已删除 100644 → 0
浏览文件 @ 0c511700
.. THIS FILE IS GENERATED BY `gen_doc.{py|sh}`
!DO NOT EDIT THIS FILE MANUALLY!
=================
fluid.regularizer
=================
.. _api_fluid_regularizer_append_regularization_ops:
append_regularization_ops
-------------------------
.. autofunction:: paddle.fluid.regularizer.append_regularization_ops
:noindex:
.. _api_fluid_regularizer_L1Decay:
L1Decay
-------
.. autoclass:: paddle.fluid.regularizer.L1Decay
:members:
:noindex:
.. _api_fluid_regularizer_L2Decay:
L2Decay
-------
.. autoclass:: paddle.fluid.regularizer.L2Decay
:members:
:noindex:
.. _api_fluid_regularizer_L1DecayRegularizer:
L1DecayRegularizer
------------------
.. autoclass:: paddle.fluid.regularizer.L1DecayRegularizer
:members:
:noindex:
.. _api_fluid_regularizer_L2DecayRegularizer:
L2DecayRegularizer
------------------
.. autoclass:: paddle.fluid.regularizer.L2DecayRegularizer
:members:
:noindex:
doc/fluid/api/transpiler.rst 已删除 100644 → 0
浏览文件 @ 0c511700
.. THIS FILE IS GENERATED BY `gen_doc.{py|sh}`
!DO NOT EDIT THIS FILE MANUALLY!
================
fluid.transpiler
================
.. _api_fluid_transpiler_DistributeTranspiler:
DistributeTranspiler
--------------------
.. autoclass:: paddle.fluid.transpiler.DistributeTranspiler
:members:
:noindex:
.. _api_fluid_transpiler_InferenceTranspiler:
InferenceTranspiler
-------------------
.. autoclass:: paddle.fluid.transpiler.InferenceTranspiler
:members:
:noindex:
.. _api_fluid_transpiler_memory_optimize:
memory_optimize
---------------
.. autofunction:: paddle.fluid.transpiler.memory_optimize
:noindex:
.. _api_fluid_transpiler_release_memory:
release_memory
--------------
.. autofunction:: paddle.fluid.transpiler.release_memory
:noindex:
.. _api_fluid_transpiler_HashName:
HashName
--------
.. autoclass:: paddle.fluid.transpiler.HashName
:members:
:noindex:
.. _api_fluid_transpiler_RoundRobin:
RoundRobin
----------
.. autoclass:: paddle.fluid.transpiler.RoundRobin
:members:
:noindex:
doc/fluid/build_and_install/build_from_source_cn.rst 已删除 120000 → 0
浏览文件 @ 0c511700
../../v2/build_and_install/build_from_source_cn.rst
\ No newline at end of file
doc/fluid/build_and_install/build_from_source_en.rst 已删除 120000 → 0
浏览文件 @ 0c511700
../../v2/build_and_install/build_from_source_en.rst
\ No newline at end of file
doc/fluid/build_and_install/docker_install_cn.rst 已删除 120000 → 0
浏览文件 @ 0c511700
../../v2/build_and_install/docker_install_cn.rst
\ No newline at end of file
doc/fluid/build_and_install/docker_install_en.rst 已删除 120000 → 0
浏览文件 @ 0c511700
../../v2/build_and_install/docker_install_en.rst
\ No newline at end of file
doc/fluid/build_and_install/index_cn.rst 已删除 120000 → 0
浏览文件 @ 0c511700
../../v2/build_and_install/index_cn.rst
\ No newline at end of file
doc/fluid/build_and_install/index_en.rst 已删除 120000 → 0
浏览文件 @ 0c511700
../../v2/build_and_install/index_en.rst
\ No newline at end of file
doc/fluid/build_and_install/paddleci.png 已删除 120000 → 0
浏览文件 @ 0c511700
../../v2/build_and_install/paddleci.png
\ No newline at end of file
doc/fluid/build_and_install/pip_install_cn.rst 已删除 120000 → 0
浏览文件 @ 0c511700
../../v2/build_and_install/pip_install_cn.rst
\ No newline at end of file
doc/fluid/build_and_install/pip_install_en.rst 已删除 120000 → 0
浏览文件 @ 0c511700
../../v2/build_and_install/pip_install_en.rst
\ No newline at end of file
doc/fluid/design/algorithm/index_cn.rst 已删除 100644 → 0
浏览文件 @ 0c511700
梯度更新算法
------------
.. toctree::
:maxdepth: 1
parameter_average.md
doc/fluid/design/algorithm/index_en.rst 已删除 100644 → 0
浏览文件 @ 0c511700
Gradient Update Algorithm
--------------------------------------
.. toctree::
:maxdepth: 1
parameter_average.md
doc/fluid/design/algorithm/parameter_average.md 已删除 100644 → 0
浏览文件 @ 0c511700
# Averaging Parameter in PaddlePaddle
## Why Averaging
In a large scale machine learning setup where the size of the training data is huge, it could take us a large number of iterations over the training data before we can achieve the optimal values of parameters of our model. Looking at the problem setup, it is desirable to obtain the optimal values of parameters by going through the data in as few passes as possible.
Polyak and Juditsky (1992) showed that the test performance of simple average of parameters obtained by Stochastic Gradient Descent (SGD) is as good as that of parameter values that are obtained by training the model over and over again, over the training dataset.
Hence, to accelerate the speed of Stochastic Gradient Descent, Averaged Stochastic Gradient Descent (ASGD) was proposed in Polyak and Juditsky (1992). For ASGD, the running average of parameters obtained by SGD, is used as the estimator for <img src="https://raw.githubusercontent.com/PaddlePaddle/Paddle/develop/doc/fluid/images/theta_star.gif"/><br/> . The averaging is done as follows:
<p align="center">
<img src="https://raw.githubusercontent.com/PaddlePaddle/Paddle/develop/doc/fluid/images/asgd.gif"><br />
</p>
We propose averaging for any optimizer similar to how ASGD performs it, as mentioned above.
### How to perform Parameter Averaging in PaddlePaddle
Parameter Averaging in PaddlePaddle works in the following way during training :
1. It will take in an instance of an optimizer as an input, e.g. RMSPropOptimizer
2. The optimizer itself is responsible for updating the parameters.
3. The ParameterAverageOptimizer maintains a separate copy of the parameters for itself:
1. In theory, the values of this copy are the average of the values of the parameters in the most recent N batches.
2. However, saving all N instances of the parameters in memory is not feasible.
3. Therefore, an approximation algorithm is used.
Hence, overall we have have two copies of the parameters: one for the optimizer itself, and one for the ParameterAverageOptimizer. The former should be used in back propagation, while the latter should be used during testing and should be saved.
During the testing/saving the model phase, we perform the following steps:
1. Perform the delayed operations.
2. Save current values of the parameters to a temporary variable.
3. Replace the values of the parameters with the averaged values.
4. Perform testing and/or save the parameters.
5. Restore the values of the parameters once done.
### How to implement Averaging of Parameter in PaddlePaddle
We can add the ParameterAverageOptimizer op to the graph through Python API. Using this approach, we manually add this op to the graph and direct the output of the optimizer op to this op during training.
**Advantages**:
- Allows for greater flexibility to the users of PaddlePaddle. Using this approach, the users can plug different optimizers into ParameterAverageOptimizer by passing in the optimizer to the op.
- Makes it easy for the users to customize and extend the framework.
**Disadvantages**:
- Implementation requires re-writing the averaging methodology in Python.
### Low-Level implementation
In the new design, we propose to create a new operation for averaging parameter updates (ParameterAverageOptimizer). For now, we can add an op that takes in the following as input:
- the optimizer
- the window_size to keep the updates
The ParameterAverageOptimizer op can be like any other operator with its own CPU/GPU implementation either using Eigen or separate CPU and GPU kernels. As the initial implementation, we can implement the kernel using Eigen following the abstraction pattern implemented for [Operators](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/fluid/operators/rmsprop_op.h). We also want to support the case when the Trainer/Optimizer runs on the GPU while ParameterAverageOptimizer runs on a CPU.
The idea of building an op for averaging is in sync with the refactored PaddlePaddle philosophy of using operators to represent any computation unit. The way the op will be added to the computation graph will be decided by the [layer functions](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/fluid/design/modules/python_api.md#layer-function) in Python API.
### Python API implementation for ParameterAverageOptimizer
Based on Polyak and Juditsky (1992), we can generalize the averaging of updates to any optimizer. The input to the op would be the following:
- Any optimizer (RMSProp , AdaGrad etc.)
- A window size. The op keeps accumulating updated parameter values over a window of N batches and takes an average. Move the averaged value to a buffer when window is full to avoid loss of precision.
Using the ParameterAverageOptimizer op, any user can add the operation to their computation graphs. However, this will require a lot of lines of code and we should design Python APIs that support averaging. As per the PaddlePaddle [Python API design](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/fluid/design/modules/python_api.md), the layer functions are responsible for creating operators, operator parameters and variables. Since ParameterAverageOptimizer will be an operator, it makes sense to create it in the layer functions.
We will have a wrapper written in Python that will support the functionality and implement the actual core computation in C++ core as we have done for other [Optimizers](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/fluid/operators/rmsprop_op.cc)
#### Creation of the ParameterAverageOptimizer operator
There are two ways for creating the ParameterAverageOptimizer op:
1. We create the op immediately while building the computation graph.
2. We add the op in a lazy manner, just before the backward pass, similar to the way the optimization ops are added.
The proposal is to add the op immediately while building the computation graph.
#### High-level API
In PaddlePaddle Python API, users will primarily rely on [layer functions](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/fluid/design/modules/python_api.md#layer-function) to create neural network layers. Hence, we also need to provide parameter average functionality in layer functions.
doc/fluid/design/concepts/README.md 已删除 100644 → 0
浏览文件 @ 0c511700
A few months ago when we were trying to replace CMake with Bazel, @emailweixu suggested that we rewrite those handy Bazel functions using CMake. Now it seems that it's the right time to get this done, as we are facing problems from the porting of Majel and the development of new the parameter server using Go and C++.
Here are some initial thoughts. Your comments are welcome!
# Required CMake Function
I think we need only the following few CMake functions to make a project description mean and clean:
<table>
<thead>
<tr>
<th>C++</th>
<th>CUDA C++</th>
<th>Go</th>
</tr>
</thead>
<tbody>
<tr>
<td>cc_library </td>
<td>nv_library </td>
<td>go_library </td>
</tr>
<tr>
<td>cc_binary </td>
<td>nv_binary </td>
<td>go_binary </td>
</tr>
<tr>
<td> cc_test </td>
<td> nv_test </td>
<td> go_test </td>
</tr>
</tbody>
</table>
- The `_library` functions generate .a files from source code.
- The `_binary` functions generate executable binary files.
- The `_test` functions generate executable unit test files. They work like `_binary` but links `-lgtest` and `-lgtest_main`.
The difference between `nv_` functions and `cc_` functions is that the former use `nvcc` instead of the system-default C++ compiler.
Both `nv_` and `cc_` functions enables C++11 (-std=c++11).
Also,
- to describe external dependencies, we need `external_library`.
- to build shared libraries, we need `shared_library`.
## An Example Project
Suppose that we have aforementioned functions defined in our `/cmake` directory. The following example `CMakeLists.txt` describes a project including the following source files:
- tensor.h
- tensor.cc
- tensor_test.cc
- ops.h
- ops.cu
- ops_test.cu
- api.go
- api_test.go
Suppose that ops.cu depends on CUDNN.
```cmake
# cc_binary parses tensor.cc and figures out that target also depend
# on tensor.h.
cc_binary(tensor
SRCS
tensor.cc)
# The dependency to target tensor implies that if any of
# tensor{.h,.cc,_test.cc} is changed, tensor_test need to be re-built.
cc_test(tensor_test
SRCS
tensor_test.cc
DEPS
tensor)
# I don't have a clear idea what parameters external_library need to
# have. @gangliao as a CMake expert would have better ideas.
external_library(cudnn
....)
# Suppose that ops.cu depends on external target CUDNN. Also, ops.cu
# include global functions that take Tensor as their parameters, so
# ops depend on tensor. This implies that if any of tensor.{h.cc},
# ops.{h,cu} is changed, ops need to be re-built.
nv_library(ops
SRCS
ops.cu
DEPS
tensor
cudnn) # cudnn is defined later.
nv_test(ops_test
SRCS
ops_test.cu
DEPS
ops)
# Because api.go defines a GO wrapper to ops and tensor, it depends on
# both. This implies that if any of tensor.{h,cc}, ops.{h,cu}, or
# api.go is changed, api need to be re-built.
go_library(api
SRCS
api.go
DEPS
tensor # Because ops depend on tensor, this line is optional.
ops)
go_test(api_test
SRCS
api_test.go
DEPS
api)
# This builds libapi.so. shared_library might use CMake target
# api_shared so to distinguish it from above target api.
shared_library(api
DEPS
api)
```
## Implementation
As above example CMakeLists.txt executes, each function invocation adds "nodes" to a dependency graph. It also use this graph to generate CMake commands including `add_executable`, `add_dependencies`, `target_link_libraries`, and `add_test`.
## Using Package Manager For Go
Building Go binaries and libraries need to satisfy their dependencies, generally
we can do `go get ./...` to download and compile all external dependencies. The
problems are:
1. `go get` will always get the latest code from the default branch of the
remote repo, so changes of dependents might break the build. This is very
different with what we already have in `cmake/external` which download a
specific version or commit id of the dependency.
1. Some locations can not access external dependencies through the internet, as mentioned
in https://github.com/PaddlePaddle/Paddle/issues/2605. Using package management
tools can package the dependencies as a "vendor" package, which can be mirrored
at many cloud file hosting, so users what to compile paddle by themselves can
download this "vendor" package from a mirror site.
### Choose A Suitable Tool
As mentioned by @wangkuiyi, [Here](https://github.com/golang/go/wiki/PackageManagementTools)
list dozens of Go package managers. We choose the tool using following principles:
- Most "active" projects with more stars, more pull requests or commits
- Widely used project
After comparing all these projects, we shall choose between the most popular
tools: Godep and Glide.
Here's a brief comparison between Godep and Glide
: https://github.com/Masterminds/glide/wiki/Go-Package-Manager-Comparison. There are
also many complaints about using `Godep`. There's also a new "official" pakcage
management tool has been started at: https://github.com/golang/dep to resolve
such problems, but it's currently at Alpha stage. So the best choice now is
glide obviously.
### Manage Go Packages
- Dependencies: `go/glide.yaml` will store the dependencies and their versions which
is directly imported by paddle. `go/glide.lock` will store all dependencies recursively
with their commit id. Builds will "lock" to these packages if we don't `glide up`
them
- Vendor package: `go/vendor` directory will generated when running `cmake` command. `cmake`
will download the code corresponding to `go/glide.lock`. If we put a vendor folder
under `go/`, cmake will just check the commit id to the packages under the folder,
if commit id matches, there will be no download at all.
doc/fluid/design/concepts/block.md 已删除 100644 → 0
浏览文件 @ 0c511700
# Design Doc: Block and Scope
## The Representation of Computation
Both deep learning systems and programming languages help users describe computation procedures. These systems use various representations of computation:
- Caffe, Torch, and Paddle: sequences of layers.
- TensorFlow, Caffe2, Mxnet: graph of operators.
- PaddlePaddle: nested blocks, like C++ and Java programs.
## Block in Programming Languages and Deep Learning
In programming languages, a block is a pair of curly braces that includes local variables definitions and a sequence of instructions or operators.
Blocks work with control flow structures like `if`, `else`, and `for`, which have equivalents in deep learning:
<table>
<thead>
<tr>
<th>programming languages</th>
<th>PaddlePaddle</th>
</tr>
</thead>
<tbody>
<tr>
<td>for, while loop </td>
<td>RNN, WhileOp </td>
</tr>
<tr>
<td>if, if-else, switch </td>
<td>IfElseOp, SwitchOp </td>
</tr>
<tr>
<td>sequential execution </td>
<td>a sequence of layers </td>
</tr>
</tbody>
</table>
A key difference is that a C++ program describes a one pass computation, whereas a deep learning program describes both the forward and backward passes.
## Stack Frames and the Scope Hierarchy
The existence of the backward pass makes the execution of a block of PaddlePaddle different from traditional programs:
<table>
<thead>
<tr>
<th>programming languages</th>
<th>PaddlePaddle</th>
</tr>
</thead>
<tbody>
<tr>
<td>stack </td>
<td>scope hierarchy </td>
</tr>
<tr>
<td>stack frame </td>
<td>scope </td>
</tr>
<tr>
<td>push at entering block </td>
<td>push at entering block </td>
</tr>
<tr>
<td>pop at leaving block </td>
<td>destroy when minibatch completes </td>
</tr>
</tbody>
</table>
1. In traditional programs:
- When the execution enters the left curly brace of a block, the runtime pushes a frame into the stack, where it realizes local variables.
- After the execution leaves the right curly brace, the runtime pops the frame.
- The maximum number of frames in the stack is the maximum depth of nested blocks.
1. In PaddlePaddle
- When the execution enters a block, PaddlePaddle adds a new scope, where it realizes variables.
- PaddlePaddle doesn't pop a scope after the execution of the block because variables therein are used by the backward pass. So it has a stack forest known as a *scope hierarchy*.
- The height of the highest tree is the maximum depth of nested blocks.
- After the processing of a minibatch, PaddlePaddle destroys the scope hierarchy.
## Use Blocks in C++ and PaddlePaddle Programs
Let us consolidate the discussion by presenting some examples.
### Blocks with `if-else` and `IfElseOp`
The following C++ programs shows how blocks are used with the `if-else` structure:
```c++
namespace pd = paddle;
int x = 10;
int y = 1;
int z = 10;
bool cond = false;
int o1, o2;
if (cond) {
int z = x + y;
o1 = z;
o2 = pd::layer::softmax(z);
} else {
int d = pd::layer::fc(z);
o1 = d;
o2 = d+1;
}
```
An equivalent PaddlePaddle program from the design doc of the [IfElseOp operator](../execution/if_else_op.md) is as follows:
```python
import paddle as pd
x = minibatch([10, 20, 30]) # shape=[None, 1]
y = var(1) # shape=[1], value=1
z = minibatch([10, 20, 30]) # shape=[None, 1]
cond = larger_than(x, 15) # [false, true, true]
ie = pd.ifelse()
with ie.true_block():
d = pd.layer.add_scalar(x, y)
ie.output(d, pd.layer.softmax(d))
with ie.false_block():
d = pd.layer.fc(z)
ie.output(d, d+1)
o1, o2 = ie(cond)
```
In both examples, the left branch computes `x+y` and `softmax(x+y)`, the right branch computes `fc(x)` and `x+1` .
The difference is that variables in the C++ program contain scalar values, whereas those in the PaddlePaddle programs are mini-batches of instances.
### Blocks with `for` and `RNNOp`
The following RNN model in PaddlePaddle from the [RNN design doc](../dynamic_rnn/rnn.md) :
```python
x = sequence([10, 20, 30]) # shape=[None, 1]
m = var(0) # shape=[1]
W = var(0.314, param=true) # shape=[1]
U = var(0.375, param=true) # shape=[1]
rnn = pd.rnn()
with rnn.step():
h = rnn.memory(init = m)
h_prev = rnn.previous_memory(h)
a = layer.fc(W, x)
b = layer.fc(U, h_prev)
s = pd.add(a, b)
act = pd.sigmoid(s)
rnn.update_memory(h, act)
rnn.output(a, b)
o1, o2 = rnn()
```
has its equivalent C++ program as follows
```c++
int* x = {10, 20, 30};
int* m = {0};
int* W = {0.314};
int* U = {0.375};
int mem[sizeof(x) / sizeof(x[0]) + 1];
int o1[sizeof(x) / sizeof(x[0]) + 1];
int o2[sizeof(x) / sizeof(x[0]) + 1];
for (int i = 1; i <= sizeof(x)/sizeof(x[0]); ++i) {
int x = x[i-1];
if (i == 1) mem[0] = m;
int a = W * x;
int b = Y * mem[i-1];
int s = fc_out + hidden_out;
int act = sigmoid(sum);
mem[i] = act;
o1[i] = act;
o2[i] = hidden_out;
}
```
## Compilation and Execution
Like TensorFlow, a PaddlePaddle program is written in Python. The first part describes a neural network as a protobuf message, and the rest executes the message for training or inference.
The generation of this protobuf message is similar to how a compiler generates a binary executable file. The execution of the message is similar to how the OS executes the binary file.
## The "Binary Executable File Format"
The definition of the protobuf message is as follows:
```protobuf
message BlockDesc {
repeated VarDesc vars = 1;
repeated OpDesc ops = 2;
}
```
The step net in above RNN example would look like
```
BlockDesc {
vars = {
VarDesc {...} // x
VarDesc {...} // h
VarDesc {...} // fc_out
VarDesc {...} // hidden_out
VarDesc {...} // sum
VarDesc {...} // act
}
ops = {
OpDesc {...} // matmul
OpDesc {...} // add_two
OpDesc {...} // sigmoid
}
};
```
Also, the RNN operator in above example is serialized into a protobuf message of type `OpDesc` and would look like:
```
OpDesc {
inputs = {0} // the index of x in vars of BlockDesc above
outputs = {5, 3} // indices of act and hidden_out in vars of BlockDesc above
attrs {
"states" : {1} // the index of h
"step_net" : <above step net>
}
};
```
This `OpDesc` value is in the `ops` field of the `BlockDesc` value representing the global block.
## The Compilation of Blocks
During the generation of the Protobuf message, the Block should store VarDesc (the Protobuf message which describes Variable) and OpDesc (the Protobuf message which describes Operator).
VarDesc in a block should have its name scope to avoid local variables affecting parent block's name scope.
Child block's name scopes should inherit the parent's so that OpDesc in child block can reference a VarDesc that is stored in the parent block. For example:
```python
a = pd.Variable(shape=[20, 20])
b = pd.fc(a, params=["fc.w", "fc.b"])
rnn = pd.create_rnn()
with rnn.stepnet():
x = a.as_step_input()
# reuse fc's parameter
fc_without_b = pd.get_variable("fc.w")
rnn.output(fc_without_b)
out = rnn()
```
The method `pd.get_variable` can help retrieve a Variable by the name. The Variable may be stored in a parent block, but might be retrieved in a child block, so block should have a variable scope that supports inheritance.
In compiler design, the symbol table is a data structure created and maintained by compilers to store information about the occurrence of various entities such as variable names, function names, classes, etc.
To store the definition of variables and operators, we define a C++ class `SymbolTable`, like the one used in compilers.
`SymbolTable` can do the following:
- store the definitions (some names and attributes) of variables and operators,
- verify if a variable was declared,
- make it possible to implement type checking (offer Protobuf message pointers to `InferShape` handlers).
```c++
// Information in SymbolTable is enough to trace the dependency graph. So maybe
// the Eval() interface takes a SymbolTable is enough.
class SymbolTable {
public:
SymbolTable(SymbolTable* parent) : parent_(parent) {}
OpDesc* NewOp(const string& name="");
// TODO determine whether name is generated by python or C++.
// Currently assume that a unique name will be generated by C++ if the
// argument name is left default.
VarDesc* Var(const string& name="");
// find a VarDesc by name, if recursive is true, find parent's SymbolTable
// recursively.
// this interface is introduced to support InferShape, find protobuf messages
// of variables and operators, pass pointers into InferShape.
//
// NOTE maybe some C++ classes such as VarDescBuilder and OpDescBuilder should
// be proposed and embedded into pybind to enable python operation on C++ pointers.
VarDesc* FindVar(const string& name, bool recursive=true);
OpDesc* FindOp(const string& name);
BlockDesc Compile() const;
private:
SymbolTable* parent_;
map<string, OpDesc> ops_;
map<string, VarDesc> vars_;
};
```
After all the description of variables and operators is added into SymbolTable,
the block has enough information to run.
The `Block` class takes a `BlockDesc` as input, and provides `Run` and `InferShape` functions.
```c++
namespace {
class Block : OperatorBase {
public:
Block(const BlockDesc& desc) desc_(desc) {}
void InferShape(const framework::Scope& scope) const override {
if (!symbols_ready_) {
CreateVariables(scope);
CreateOperators();
}
// should run InferShape first.
for (auto& op : runtime_table_.ops()) {
op->InferShape(scope);
}
}
void Run(const framework::Scope& scope,
const platform::Place& place) const override {
PADDLE_ENFORCE(symbols_ready_, "operators and variables should be created first.");
for (auto& op : runtime_table_.ops()) {
op->Run(scope, place);
}
}
void CreateVariables(const framework::Scope& scope);
void CreateOperators();
// some other necessary interfaces of NetOp are listed below
// ...
private:
BlockDesc desc_;
bool symbols_ready_{false};
};
```
## The Execution of Blocks
Block inherits from OperatorBase, which has a Run method.
Block's Run method will run its operators sequentially.
There is another important interface called `Eval`, which takes some arguments called targets and generates a minimal graph which treats targets as the end points and creates a new Block. After `Run`, `Eval` will get the latest value and return the targets.
The definition of Eval is as follows:
```c++
// clean a block description by targets using the corresponding dependency graph.
// return a new BlockDesc with minimal number of operators.
// NOTE: The return type is not a Block but the block's description so that this can be distributed
// to a cluster.
BlockDesc Prune(const BlockDesc& desc, vector<string> targets);
void Block::Eval(const vector<string>& targets,
const framework::Scope& scope,
const platform::DeviceContext& dev_ctx) {
BlockDesc min_desc = Prune(desc_, targets);
Block min_block(min_desc);
min_block.Run(scope, dev_ctx);
}
```
doc/fluid/design/concepts/cpp_data_feeding.md 已删除 100644 → 0
浏览文件 @ 0c511700
# C++ Data Feeding
While using Paddle V2 API for training, data feeding completely depends on the Python code. To get rid of the Python environment and achieve the goal of "wrapping the whole training by a while loop op" in Paddle Fluid, a C++ data feeding mechanism is required.
In this document, we show the fundamental design of a C++ data feeding process, which includes data reading, shuffling and batching.
## Overview
![](images/readers.png)
## Reader
In order to handle the above-mentioned problem, a new concept called 'Reader' is introduced. `Reader` is a series of inherited classes which can be held by our `Variable` and they are used to read or process file data.
### ReaderBase
`ReaderBase` is the abstract base class for all readers. It defines the interface for all readers.
```cpp
class ReaderBase {
public:
// Reads the next batch of data. (A 'batch' can be only one instance)
// If the next batch doesn't exist, it throws an exception
virtual void ReadNext(std::vector<LoDTensor>* out) = 0;
// Checks whether the next instance exists.
virtual bool HasNext() = 0;
// Reinitializes the reader and read the file from the beginning.
virtual void ReInit() = 0;
virtual ~ReaderBase();
};
```
### FileReader
`FileReader` is derived from the `ReaderBase`. It is still an abstract class and will further be derived by Readers of respective specific format.
```cpp
class FileReader : public ReaderBase {
public:
explicit FileReader(const std::vector<DDim>& dims);
void ReadNext(std::vector<LoDTensor>* out) override;
protected:
virtual void ReadNextImpl(std::vector<LoDTensor>* out) = 0;
private:
std::vector<DDim> dims_;
};
```
A file reader binds with a single file and reads one data instance at a time. Each type of file reader shall implement its own `ReadNextImpl()`, `HasNext()` and `ReInit()`.
The `ReadNextImpl()` is invoked by `ReadNext()`. Besides invoking `ReadNextImpl()`, `ReadNext()` is also responsible for checking the output, making sure that each shape of `LoDTensor` in `*out` is consistent with the one in `dims_`.
### DecoratedReader
A decorated reader takes another reader(both file reader and decorated reader are OK) as its 'underlying reader'. It gets data from its underlying reader, does some processing on them(shuffling, batching or something else), then yields processed data. The output data of a decorated reader can be a single instance or a batch. `ShuffleReader` and `BatchReader` are both decorated readers.
```cpp
class DecoratedReader : public ReaderBase {
public:
explicit DecoratedReader(ReaderBase* reader) : ReaderBase(), reader_(reader) {
PADDLE_ENFORCE_NOT_NULL(reader_);
}
void ReInit() override { reader_->ReInit(); }
bool HasNext() const override { return reader_->HasNext(); }
protected:
ReaderBase* reader_;
};
```
Both the `FileReader` and `DecoratedReader` share exactly the same interface as defined in `ReaderBase`. So they can be decorated for multiple times: We can **shuffle** a reader's outputs and then **batch** the shuffled outputs. The interface consistency also allows related ops use readers without knowing their underlying type.
### MultipleReader
All `FileReader` binds with a single file and are single-threaded. However, sometimes we need to read data from more than one file. In this case, it's not enough to only have `FileReader` and `DecoratedReader`.
So `MultipleReader` is introduced. It is also derived from `ReaderBase`. A `MultipleReader` holds several prefetching `FileReaders` and these readers run concurrently. Another pivotal part of a `MultipleReader` is a buffer channel. The channel collects data yield by all prefetching readers and makes subsequent OPs or decorated readers be able to fetch data without concerning about multiple readers scheduling.
![](images/multiple_reader.png)
This graph shows how a `MultipleReader` works with three prefetching file readers and two GPUs. There is a queue of files which are going to be read. Each time when a prefetching file reader is free(complete reading from one file), it fetches a new file from the queue. Each prefetching file reader runs in a separated prefetch thread and dumps their outputs to the same channel.
To the subsequent two decorated readers, the `MultipleReader` is **a single reader**. They don't need to concern about how prefetch readers are scheduled. They only need to invoke `MultipleReader::ReadNext()` to get the next data from the buffer channel.
### ReaderHolder
Different readers belong to different class types. This leads to a problem: How can we drop them into `Variable`s and fetch them out by a unified method? For example, if a Variable holds a `BatchReader`, we can not get it by the following code:
```cpp
var->Get<ReaderBase>("batch_reader");
```
We would have to write:
```cpp
var->Get<BatchReader>("batch_reader");
```
This requires that in order to get a reader from a variable, every time, we must know the reader's type exactly. This is nearly impossible.
To solve this problem, we introduce `ReaderHolder` as a wrapper. It acts as an empty decorator of `ReaderBase`, which hides reader's type. With `ReaderHolder` we are able to fetch all types of readers by `var->Get<ReaderHolder>("...")` and regard the obtained object as a reader.
## Related Operators
To create and invoke readers, some new ops are introduced:
### Operators That Create Readers
Each reader has its creation op. File readers' creation ops have no input and yield the created file reader as its output. Decorated readers' creation ops take the underlying readers as inputs and then yield new decorated readers.
However, direct usage of file readers' creation ops is not recommended because a file reader can only read one file via a single thread. Using `OpenFilesOp` is a better choice.
### OpenFilesOp
The `OpenFilesOp` is the creation op of `MultipleReader`. It takes no input but requires a list of file names as one of its attributes. The newly created `MultipleReader` then creates its own prefetching readers according to given file names.
To make sure that created prefetching readers match file formats, we need a name prefix rule to append file format tags to file names, as well as a file reader registry mechanism to map file format tags to their corresponding file readers' constructors.
### HasNextOp
`HasNextOp` is used to check whether the next data batch exists via the reader's `HasNext()` interface.
### ResetOp
`ResetOp` is used to reset a reader via its `ReInit()` interface.
### ReadOp
A reader is only a Variable. It cannot trigger the reading process by itself. So we add the `ReadOp` to execute it. A `ReadOp` takes a reader Variable as its input. Each time it runs, it invokes the reader‘s `ReadNext()` function and gets a new batch of data(or only one instance of data, if we use file reader directly). The output data of a reader are in the form of `std::vector<LoDTenosr>`, so the `ReadOp` also needs to split the vector and move LoDTensors to their respective output Variables.
## Program with Readers
A `Program` holds readers as its persistable variables. These variables are created by `CreateReaderOp` or `OpenFilesOp`. These ops shall run only once. So they shall be settled in the `startup_program`. `HasNextOp`, `ResetOp` and `ReadOp` are required by training loop, so they shall be in the `main_program`.
The ops of a `startup_program` with readers would be like this:
```
multiple_reader = open_files_op(...)
batch_reader = create_batch_reader_op(multiple_reader)
double_buffer_reader = create_double_buffer_op(batch_reader)
... (other initializers)
```
The forwarding ops of the corresponding `main_program` would be like this:
```
not_completed = true
pass_count = 0
while_op(not_completed) {
has_next = has_next_op(double_buffer_reader)
if_else_op(has_next) {
batch_data = read_op(double_buffer_reader)
... (subsequent training ops)
} else {
reset_op(double_buffer_reader)
increase_op(pass_count)
not_completed = less_than_op(pass_count, reqiured_pass_num)
}
}
```
A few important considerations for these programs are as follows:
1. `not_completed`, `pass_count` and other variables shown above are all Fluid Variables.
2. The multiple\_reader is the batch\_reader's underlying reader, and the batch\_reader is the double\_buffer\_reader's underlying reader. `read_op`, `has_next_op` and other reader related ops will only invoke the top-most reader. In this case, it's the double\_buffer\_reader.
3. All readers exist in both `startup_program` and `main_program`. And they are persistable.
### Simplify Configuration by MultiPassReader
The Program configuration mentioned above is complicated. Users need to be very familiar to concepts of Program and Block to prevent making mistakes in their code. To make the usage of C++ readers more friendly to new users, we introduce `MultiPassReader`.
`MultiPassReader` is a decorated reader. A multi-pass reader is used to continuously yield data for several training passes. It takes the number of passes to run as one of its attributes('pass_num') and maintains a counter to record how many passes it has completed. Each time its underlying reader reaches the EOF, the multi-pass reader checks whether it has completed the training of given number of pass. If not, the underlying reader will be re-initialized and starts a new pass automatically. Before completing the whole training, the return of MultiPassReader's `HasNext()` will always be `true`.
With `MultiPassReader`, the startup program would be like this:
```
multiple_reader = open_files_op(...)
batch_reader = create_batch_reader_op(multiple_reader)
multi_pass_reader = create_multi_pass_reader_op(batch_reader)
double_buffer_reader = create_double_buffer_op(multi_pass_reader)
... (other initializers)
```
The forwarding part of the corresponding `main_program` would be like this:
```
not_completed = true
while_op(not_completed) {
batch_data = read_op(double_buffer_reader)
... (subsequent training ops)
not_completed = has_next_op(double_buffer_reader)
}
```
doc/fluid/design/concepts/executor.md 已删除 100644 → 0
浏览文件 @ 0c511700
# Executor Design Doc
## Motivation
In [fluid](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/fluid/design/motivation/fluid.md), we encourage the user to use deep learning programming paradigms to describe the training process. When the user-written Python program is executed, it will first create a protobuf message
[`ProgramDesc`](https://github.com/PaddlePaddle/Paddle/blob/a91efdde6910ce92a78e3aa7157412c4c88d9ee8/paddle/framework/framework.proto#L145) that describes the process and is conceptually like an [abstract syntax tree](https://en.wikipedia.org/wiki/Abstract_syntax_tree).
The executor runs the `ProgramDesc` like an interpreter. `ProgramDesc` contains the intrinsics (operators in this case) and variables which will be used, executor explicitly executes the stored precompiled code.
## Overview
An executor takes a `ProgramDesc`, a `block_id` and a `Scope`. The `ProgramDesc` is a list of blocks and each block contains the protobuf definition of all the parameters and operators in the block. The `block_id` specifies the entrance block. And the `Scope` is the container of all the variable instances, which is persistent throughout different runs.
## Executor
The `Executor` explicitly executes all the intrinsics (operators here) in the `block_id`th block of a `ProgramDesc`. Essentially, it instantiates Variables and Operators, then runs all the operators in sequence one-by-one.
It is very similar to how a push stack frame works when entering a block, following which it cleans up all the temporary variables when a mini-batch is finished. It does not however, have the stack frame pop process.
### The interface
```c++
Executor(places);
```
A executor does not own any computing resources, a user can only construct an executor using the specified places.
### Running an Executor
```
void Run(ProgramDesc, Scope, block_id, create_local_scope);
```
An `Executor` only provides a unified way to execute `ProgramDesc`. `ProgramDesc` is the target that will be executed, the `Scope` specifies the variable container, the `block_id` indicates the entrance block and `create_local_scope` is a boolean that states whether it will destroy the temporary variables after the execution is finished.
doc/fluid/design/concepts/functions_operators_layers.md 已删除 100644 → 0
浏览文件 @ 0c511700
# Design Doc: Functions, Operators, and Layers
In a DL system, we can compose one or more fine grained operators into a coarse grained one. For example, the FC layer can be composed of a multiplication operator and an add operator.
Historically, some fine grained operations are known as operators, and some coarse level ones are known as layers. But we need a well-defined separation.
In general, operators are those very fine grained operations, e.g., mul and add. In the implementation, we can write them as C++ functions:
```c++
template <typename T> T add(T x, T y) { return x + y; }
template <typename T> T mul(T x, T y) { return x * y; }
```
Then we can wrap them into operators which are C++ classes and can be created from Python bindings by name. A C macro can do this. For example, the following macro invocation
```c++
#define MAKE_FUNCTION_OPERATOR(mul);
```
generates
```c++
template <typename T> class mulOp : public OperatorBase {...};
REGISTER_OP(mulOp<float32>, "mul");
```
so that in Python we can create operator mul by:
```python
X1 = Var()
X2 = Var()
Y = Var()
paddle.cpp.create_operator("mul", input=[X1, X2], output=Y)
```
Also, at the same time, we can compose a coarse level C++ operator class by composing functions `mul` and `add`:
```c++
template <typename T>
class FCOp : public OperatorBase {
public:
void Run(...) {
add(mul(Input<T>("X"), Input<T>("W")), Input<T>("b"));
}
};
REGISTER_OP(FCOp, "fc");
```
We need to support such composition in Python as well. To do so, we need a higher level Python wrapping of operator creation than `paddle.cpp.create_operator`. This higher level operator API should be compatible with the layer API.
Let's explain using an example. Suppose that we are going to compose the FC using mul and add in Python, we'd like to have Python functions `mul` and `add` defined in module `operator`:
```python
def operator.mul(X1, X2):
O = Var()
paddle.cpp.create_operator("mul", input={X1, Y1}, output=O)
return O
def operator.add(X1, X2):
O = Var()
paddle.cpp.create_operator("add", input={X1, X2}, output=O)
return O
```
Above code snippets are automatically generated. Given them, users can define
```python
def layer.fc(X):
W = Var()
b = Var()
return operator.add(operator.mul(X, W), b)
```
If we don't have `operator.mul` and `operator.add`, the definiton of `layer.fc` would be complicated:
```python
def layer.fc(X):
W = Var()
b = Var()
O1 = Var()
paddle.cpp.create_operator("mul", input=[X, W], output=O1)
O2 = Var()
paddle.cpp.create_operator("add", input=[O1, b], output=O2)
return O2
```
We'd like to have Python bindings to operators in package `paddle.operator`, and Python compositions of operators in package `paddle.layer`. So we have the following concepts in above illustrative example:
<table>
<thead>
<tr>
<th>C++ functions/functors</th>
<th>mul</th>
<th>add</th>
<th></th>
<th></th>
</tr>
</thead>
<tbody>
<tr>
<td>C++ operator class </td>
<td>mulOp</td>
<td>addOp </td>
<td>FCOp </td>
<td></td>
</tr>
<tr>
<td>Python binding </td>
<td>operator.mul</td>
<td> operator.add </td>
<td>operator.fc </td>
<td></td>
</tr>
<tr>
<td>Python function </td>
<td></td>
<td></td>
<td> </td>
<td>layer.fc</td>
</tr>
</tbody>
</table>
This is how we differentiate layer and operators in PaddlePaddle:
- those defined in C++ and have a lightweighted Python wrapper in module `operators` are operators; whereas
- those who don't have C++ implementations but a Python implementation that compose C++ operators are known as layers.
doc/fluid/design/concepts/images/parallel_executor_overview.dot 已删除 100644 → 0
浏览文件 @ 0c511700
digraph G {
subgraph cluster_init {
label="Initialization"
startup_program [label="startup", shape=box]
node_w_g0 [label="W\nGPU0"]
startup_program -> node_w_g0 [label="Initialize"]
node_w_g1 [label="W\nGPU1"]
node_w_g0 -> node_w_g1 [label="broadcast"]
}
subgraph cluster_train {
label="forward_backward"
subgraph cluster_gpu0 {
label="GPU0"
fc_0 [label="fc\nGPU0", shape=box]
hidden_0 [label="hidden\nGPU0"]
node_w_g0 -> fc_0
fc_0 -> hidden_0
loss0 [label="loss\nGPU0"]
hidden_0 -> loss0 [label="many ops omitted"]
scale_loss_0 [label="scale_loss_gradient\nGPU0", shape=box]
loss_g0 [label="loss_grad\nGPU0"]
scale_loss_0->loss_g0
fc_g_0 [label="w_grad\nGPU0", shape=box]
loss0 -> fc_g_0
loss_g0 -> fc_g_0
hidden_0 -> fc_g_0
}
subgraph cluster_gpu1 {
label="GPU1"
fc_1 [label="fc\nGPU1", shape=box]
hidden_1 [label="hidden\nGPU1"]
node_w_g1 -> fc_1
fc_1 -> hidden_1
loss1 [label="loss\nGPU1"]
hidden_1 -> loss1 [label="many ops omitted"]
scale_loss_1 [label="scale_loss_gradient\nGPU1", shape=box]
loss_g1 [label="loss_grad\nGPU1"]
scale_loss_1->loss_g1
fc_g_1 [label="w_grad\nGPU1", shape=box]
loss1 -> fc_g_1
loss_g1 -> fc_g_1
hidden_1 -> fc_g_1
}
}
all_reduce_w [label="Merge Gradients(AllReduce)", shape=box]
fc_g_0 -> all_reduce_w
fc_g_1 -> all_reduce_w
fc_g_0_merged [label="w_grad\nMerged\nGPU0"]
fc_g_1_merged [label="w_grad\nMerged\nGPU1"]
all_reduce_w -> fc_g_0_merged
all_reduce_w -> fc_g_1_merged
subgraph cluster_optimization {
label="Optimization"
subgraph cluster_opt_gpu0 {
label="GPU0"
sgd_0 [label="SGD Op\nGPU0", shape=box]
fc_g_0_merged -> sgd_0
node_w_g0 -> sgd_0
optimized_w_0 [label="Optimized W\nGPU0"]
sgd_0 -> optimized_w_0
}
subgraph cluster_opt_gpu1 {
label="GPU1"
sgd_1 [label="SGD Op\nGPU1", shape=box]
fc_g_1_merged -> sgd_1
node_w_g1 -> sgd_1
optimized_w_1 [label="Optimized W\nGPU0"]
sgd_1 -> optimized_w_1
}
}
}
doc/fluid/design/concepts/index_cn.rst 已删除 100644 → 0
浏览文件 @ 0c511700
核心概念
-------------
.. toctree::
:maxdepth: 1
README.md
cpp_data_feeding.md
functions_operators_layers.md
program.md
variable.md
var_desc.md
tensor.md
tensor_array.md
lod_tensor.md
block.md
scope.md
executor.md
parallel_executor.md
doc/fluid/design/concepts/index_en.rst 已删除 100644 → 0
浏览文件 @ 0c511700
Core Concepts
--------------------------------------
.. toctree::
:maxdepth: 1
README.md
cpp_data_feeding.md
functions_operators_layers.md
program.md
variable.md
var_desc.md
tensor.md
tensor_array.md
lod_tensor.md
block.md
scope.md
executor.md
parallel_executor.md
doc/fluid/design/concepts/lod_tensor.md 已删除 100644 → 0
浏览文件 @ 0c511700
# Design Doc: LoD (Level-of-Detail) Tensor
Like other deep learning systems, PaddlePaddle supports training models from sequence data. Also, like other systems, PaddlePaddle represent a mini-batch of sequences as a Tensor. What is different is that PaddlePaddle doesn't require all sequences in a mini-batch to be of the same length. Thus no need for padding zeros.
<table>
<thead>
<tr>
<th></th>
<th>TensorFlow</th>
<th>PaddlePaddle</th>
</tr>
</thead>
<tbody>
<tr>
<td>RNN </td>
<td>Support </td>
<td>Support </td>
</tr>
<tr>
<td>recursive RNN </td>
<td>Support </td>
<td>Support </td>
</tr>
<tr>
<td>padding zeros </td>
<td> Must </td>
<td>No need </td>
</tr>
<tr>
<td> blob data type </td>
<td> Tensor</td>
<td> LoDTensor </td>
</tr>
</tbody>
</table>
PaddlePaddle achieves this flexibility by passing through a new data type, *LoD Tensor*, which is a Tensor attached with segmentation index known as *LoD*, between operators. The LoD index doesn't only segment a tensor, but also recursively segments sub-sequences. This document presents the design of LoD and LoDTensor.
## The Challenge: Variable-length Sequences
Most deep learning systems represent a mini-batch as a Tensor. For example, a mini-batch of 10 images, each of size 32x32, is a 10x32x32 Tensor. Another example is that each mini-batch contains N sentences, where each word is a D-dimensional one-hot vector. Suppose that all sentences have the same length L, we can represent this mini-batch by a NxLxD tensor.
Both examples show that the elements of sequences are usually of the same size. In the first example, all images are 32x32, and in the second one, all words are D-dimensional vectors. It doesn't make sense to allow variable-sized images, as that would require transformations like convolution to handle variable-sized Tensors.
The real challenge is that in most cases, sentences have variable lengths, and we will need an index data structure to segment the tensor into sequences. Also, sequences might consist of sub-sequences.
## A Solution: The LoD Index
To understand our solution, it is best to look at some examples.
### A Mini-Batch of Sentences
Let's imagine a mini-batch of 3 variable lengths sentences composed of 3, 1, and 2 words, respectively. We can represent the mini-batch by a (3+1+2)xD tensor plus some index information:
```
3 1 2
||| | ||
```
where each `|` represents a D-dimensional word vector. The numbers, 3, 1, and 2, form a 1-level LoD.
### Recursive Sequences
Let check another example of a 2-level LoD Tensor. Consider a mini-batch of three articles with 3, 1, and 2 sentences, and each sentence consists of a variable number of words:
```
3 1 2
3 2 4 1 2 3
||| || |||| | || |||
```
### A Mini-Batch of Videos
LoD tensors generalize to the case where elements are higher dimensional objects, like images. Suppose that a mini-batch contains videos of the same frame size 640x480. Here is a mini-batch of 3 videos with 3, 1, and 2 frames, respectively.
```
3 1 2
口口口 口 口口
```
The underlying tensor is of size (3+1+2)x640x480, and each `口` represents a 640x480 image.
### A Mini-Batch of Images
In traditional cases like a mini-batch with N fixed-sized images, the LoD Tensor representation is as
```
1 1 1 1 1
口口口口 ... 口
```
In this case, we don't lose any information by ignoring the many 1's in the index and simply considering this LoD Tensor as a usual Tensor:
```
口口口口 ... 口
```
### Model Parameters
A model parameter is just a usual Tensor, which, just like the above example, is a **0-level LoD Tensor**.
## The LoD Tensor
Let us revisit above example of the 2-level LoD Tensor
```
3 1 2
3 2 4 1 2 3
||| || |||| | || |||
```
It is indeed a tree, where leaves are elementary sequences identified by **branches**.
For example, the third sentence in above example is identified by branch <0,2>, where 0 indicates the first article with length 3, and 2 indicates the third sentence in this article with length 4.
### The LoD Index
We can save the LoD index in the above example
```
3 1 2
3 2 4 1 2 3
```
in a not-full 2D matrix:
```c++
typedef std::vector<std::vector<int> > LoD;
```
where
- `LoD.size()` is the number of levels, or the maximum length of branches,
- `LoD[i][j]` is the length of the j-th segment at the i-th level.
## The Offset Representation
To quickly access elementary sequences, we adopt an offset representation -- instead of saving the lengths, we save the beginning and ending elements of sequences.
In the above example, we accumulate the length of elementary sequences:
```
3 2 4 1 2 3
```
into offsets
```
0 3 5 9 10 12 15
= = = = = =
3 2+3 4+5 1+9 2+10 3+12
```
so we know that the first sentence is from word 0 to word 3, and the second sentence from word 3 to word 5.
Similarly, the lengths in the top level LoD
```
3 1 2
```
are transformed into offsets of elements/words as follows:
```
0 3 4 6
= = =
3 3+1 4+2
```
## Slicing of LoD Tensors
When we use the above 2-level LoD Tensor as the input to a nested-RNN, we need to retrieve certain sequences. Here we define the sequence identified by branch <i,j,...> as the **<i,j,...>-slice**.
For example, the <2>-slice of above example is
```
10 15
10 12 15
|| |||
```
and the <2,0>-slice of above slice is
```
10 12
||
```
## Length Representation vs Offset Representation
The offset representation is an implementation-oriented decision and it makes understanding the idea behind LoDTensor difficult.
Hence, we encapsulate this implementation detail in C++ and expose the original length representation in our Python API.
Specifically, we call this length representation `recursive_sequence_lengths` and users can use the following code to set or get the `recursive_sequence_lengths` of a LoDTensor in Python:
```Python
# length representation of lod called recursive_sequence_lengths
recursive_seq_lens = [[3, 1, 2], [2, 2, 1, 3, 1, 2]]
# Create a LoDTensor that has the above recursive_sequence_lengths info.
# This recursive_sequence_lengths will be converted to an offset representation of LoD in the C++ implementation under the hood.
tensor = fluid.LoDTensor(lod)
# Set/Change the recursive_sequence_lengths info of LoDTensor
tensor.set_recursive_sequence_lengths([[3, 1, 2]])
# Get the recursive_sequence_lengths info of a LoDTensor (the offset-based LoD representation stored in C++ will be converted
# back to length-based recursive_sequence_lengths), new_recursive_seq_lens = [[3, 1, 2]]
new_recursive_seq_lens = tensor.recursive_sequence_lengths()
```
doc/fluid/design/concepts/parallel_executor.md 已删除 100644 → 0
浏览文件 @ 0c511700
# ParallelExecutor
## Background
Neural network models are defined as a `ProgramDesc` in Fluid. The `ProgramDesc` can be executed by an interpreter(i.e. the `executor` concept in Fluid). The instructions or operators in a `Program` will be executed, and the results will be fetched in Python side.
The executor is a very naive interpreter. It runs operators one by one. We can use `Parallel.Do` to support data parallelism, however, lacking device information in `ProgramDesc`; it is not possible to optimize the performance of `Parallel.Do`.
We want a `ProgramDesc` can be run on different nodes. It is better not to contain device information in `ProgramDesc`. However, we can write a high-performance interpreter, which can hold an alternative intermediate representation of `ProgramDesc`, to take full usage of Multi-GPUs.
ParallelExecutor is an interpreter of `ProgramDesc` which will [out-of-order execute](https://en.wikipedia.org/wiki/Out-of-order_execution) `Program` in data parallelism mode and maximise the utility of Multi-GPUs.
## Overview of MultiGPUs logic
The ParallelExecutor takes the startup program and main program as inputs. The parameters will be initialised on `GPU0` by startup program and will broadcast to multi-GPUs. The main program will be duplicated into multi-GPUs. The gradient will be merged during each iteration, and each device will optimize parameters independently. Since the gradients on each device will be merged before parameter optimization, the parameters will be the same on each device and it does not need to be broadcast the parameters.
![alt](images/parallel_executor_overview.png)
There are several optimizations for this logic.
1. We use an alternate representation in ParallelExecutor. It because the device information is critical for performance optimization.
2. The execution is out-of-order, i.e., an operator will be executed whenever the inputs of the operator are ready.
* GPU is a high-performance device; only one CPU thread cannot fulfil one GPU. So there is a thread pool to execute operators.
* Out-of-order also helps transpilers to generate `ProgramDesc`. It is no need to concern about the best order of performance when implementing a transpiler.
3. The streams of computation, merge gradients and fetch data are different.
The performance of `ResNeXt152` on `TitanX` which `batch_size=12` is shown below.
| Number of GPUs | 1 | 2 | 3 | 4|
| --- | --- | --- | --- | --- |
| Image/Sec | 17.9906 | 25.771 | 36.911 | 48.8428 |
| Speed Up | N/A | 1.43247029 | 2.05168255 | 2.71490667 |
## Static single assignment Graph
[Static single assignment form](https://en.wikipedia.org/wiki/Static_single_assignment_form)(`SSA` for short) is a common form for compiler optimization. To implement concurrent execution, we uses an `SSA` graph as an intermedia representation of `ProgramDesc`.
The `Program` is a directed acyclic graph, since a variable can be assigned multiple times. We enforce a variable will be assigned once, by adding version number to varaibles. We parsing the `Program` into a `SSA` graph. Also, ProgramExecutor duplicate `Program` into multi-devices. We also add a device number to varaibles and insert `NCCLAllReduce` into Graph.
The data structure of `SSA` graph is:
```c++
struct VarHandleBase {
OpHandleBase* generated_op_;
vector<OpHandleBase*> pending_ops_;
string name;
Place place;
size_t version;
};
struct OpHandleBase {
vector<OpHandleBase*> inputs_;
vector<OpHnadleBase*> outputs_;
};
struct SSAGraph {
// vars on each devices.
// * the vars in each map in vector is on different device.
// * the map is mapping a variable name to variable handles
// with different versions
vector<std::unordered_map<string, vector<VarHandleBase>>> vars_;
// All ops
vector<OpHandleBase> ops_;
};
```
The variable handles are the wrapper of `Variables`. The operator handles are the wrapper of `OperatorBase`. Some `OpHandle` is not an `OperatorBase`, such as `NCCLAllReduceOpHandle`, because `AllReduceOpHandle` will use new device contexts.
When the `ProgramDesc` converted into an `SSA` Graph, the [data hazard](https://en.wikipedia.org/wiki/Hazard_(computer_architecture)) problem is also need to be taken care. The dummy variables, which represent the dependency between operators, will be manually inserted into SSA graph to resolve the [data hazard](https://en.wikipedia.org/wiki/Hazard_(computer_architecture)) problem.
## Execute SSA Graph
The SSA graph can be out-of-order executed by an approximate [topological sorting](https://en.wikipedia.org/wiki/Topological_sorting) algorithm. The algorithm is
1. Maintaining a map of an operator and its needed input number.
2. If a variable is not generated by an operator, i.e., `var.generated_op == nullptr`, decrease the needed input number of its pending operators.
3. If there is an operator which needed input number is decreased to zero, just run this operator.
4. After run this operator, just mark the variables are generated and repeat step 2 until all variables are generated.
Running an operator can be asynchronized. There is a thread pool to execute an `SSA` graph.
## Synchronize GPU Kernels
The GPU is a non-blocking device. The different streams need be synchronized when switching streams. In current implementation, the synchronization based on the following algorithm:
1. `OpHandle` will record `DeviceContext` that it is used.
2. In `OpHandle::Run`, if the `DeviceContext` of current operator is different from `DeviceContext` of any input variable, just wait the generate operator of this input variable.
The `wait` are implemented by two strategies:
1. Invoke `DeviceContext->Wait()`, It will wait all operators on this device contexts complete.
2. Uses `cudaStreamWaitEvent` to sending a event to the stream. It is a non-blocking call. The wait operators will be executed in GPU.
Generally, the `cudaStreamWaitEvent` will have a better perforamnce. However, `DeviceContext->Wait()` strategy is easier to debug. The strategy can be changed in runtime.
## What's next?
* Merging gradient of dense parameters has been done. However, the merging of sparse parameters has not been done.
* The CPU version of Parallel Executor has not been implemented. The out-of-order logic will make CPU compuatation faster, too.
* A better strategy to merge gradients can be introduced. We can shrink the gradients from `float32` to `int8` or `int4` while merging. It will significantly speed up multi-GPUs training without much loss of precision.
* Combine multi-Nodes implementation. By the benifit of out-of-order, sending and recving operator can be an blocking operator, and the transpiler does not need to concern about the best position of operator.
doc/fluid/design/concepts/program.md 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/design/concepts/scope.md 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/design/concepts/tensor.md 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/design/concepts/tensor_array.md 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/design/concepts/var_desc.md 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/design/concepts/variable.md 已删除 100644 → 0
浏览文件 @ 0c511700
# Design Doc: Variable
Variable is also known as *blob* in MxNet and Caffe2. It is the input and output type of operators, where a neural network is a graph of operators.
## Requirements: Lazy Memory Allocation
For the flexibility of a DL system, a variable should be able to contain any typed value -- a tensor in most cases, but could also be some integer IDs or a scope of other variables in the case of RNN.
To use the minimum amount of memory, we would like that a variable allocates memory only when it has to, or, lazy memory allocation. Let's take the following example:
```cpp
Variable vr, v1, v2;
Tensor* t1 = new Tensor();
Tensor* t2 = new Tensor();
Randomize(
/* malloc */ v1.GetMutable<Tensor>().mutable_data<float16>(DDim(100,200)),
/* size */ t1.Size());
Randomize(
/* malloc */ v2.GetMutable<Tensor>().mutable_data<float16>(DDim(200,300)),
/* size */ t2.Size());
Mult(
/*result*/ vr.GetMutable<Tensor>().mutable_data<v1.Type()>(SizeOfMult(v1, v2)),
/*input1*/ v1.Get<Tensor>().data(),
/*input2*/ v2.Get<Tensor>().data());
```
We see that a variable holds nothing until `Variable::GetMutable<Tensor>()` allocates a tensor and puts it in the variable. Similarly, a tensor gets its memory until `Tensor::mutable_data()`.
This syntax for lazy memory allocation when we call `Randomize` and `Mult`, those functions that mutate the variable, so it saves us some line of C++ code.
## Implementation: Type Hiding
To make memory allocation lazy, we cannot assume that we know the type held by a variable at definition time. In other words, `class Variable` cannot be a template `template <T> class Variable`.
Because we don't know the type `T`, we cannot save a `T*` as `Variable's` data member. Instead, we save an interface object `Placeholder`, which can return the pointer to the saved object via `Placeholder::Ptr()` as `void*`.
But anyway, Variable needs to know `T` so could it `delete<T>(ptr)` and so could `Variable::Get` checks the expected type and the saved object's type.
We save `T` in `PlaceholderImpl`, the implementation of `Placeholder`. Please be aware that `PlaceholderImpl` is a class template and `T` is passed in as a template parameter.
Because `PlaceholderImpl` knows `T`, it can save and return `typeid(T)` for the type comparison in `Variable::Get` and `Variable::GetMutable`.
## Conclusion
The technique type hiding utilizes C++ class templates, interface and derivation, and C++ RTTI (typeid). This combination saves us from defining something like `caffe2::TypeMeta`, which takes hundreds of lines of C++ code.
doc/fluid/design/concurrent/channel.md 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/design/concurrent/csp.md 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/design/concurrent/go_op.md 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/design/concurrent/index_cn.rst 已删除 100644 → 0
浏览文件 @ 0c511700
并发编程
------------
.. toctree::
:maxdepth: 1
concurrent_programming.md
parallel_do.md
doc/fluid/design/concurrent/index_en.rst 已删除 100644 → 0
浏览文件 @ 0c511700
Concurrent Programming
-------------------------
.. toctree::
:maxdepth: 1
concurrent_programming.md
parallel_do.md
doc/fluid/design/concurrent/parallel_do.md 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/design/concurrent/select_op.md 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/design/data_type/float16.md 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/design/data_type/index_cn.rst 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/design/data_type/index_en.rst 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/design/dist_train/README.md 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/design/dist_train/async_update.md 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/design/dist_train/index_cn.rst 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/design/dist_train/index_en.rst 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/design/dist_train/multi_cpu.md 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/design/dist_train/src/compiler.png 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/design/dist_train/src/dist-graph.png 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/design/dist_train/src/local-graph.png 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/design/dist_train/src/mpi_module.png 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/design/dist_train/src/ncc2_design.png 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/design/dynamic_rnn/2_level_rnn.dot 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/design/dynamic_rnn/2_level_rnn.png 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/design/dynamic_rnn/index_cn.rst 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/design/dynamic_rnn/index_en.rst 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/design/dynamic_rnn/rnn.dot 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/design/dynamic_rnn/rnn.jpg 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/design/dynamic_rnn/rnn.md 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/design/dynamic_rnn/rnn.png 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/design/dynamic_rnn/rnn_design.md 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/design/dynamic_rnn/rnn_design_en.md 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/design/execution/if_else_op.md 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/design/execution/index_cn.rst 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/design/execution/index_en.rst 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/design/execution/switch.md 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/design/index_cn.rst 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/design/index_en.rst 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/design/interface/index_cn.rst 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/design/interface/index_en.rst 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/design/ir/overview.md 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/design/memory/README.md 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/design/memory/index_cn.rst 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/design/memory/index_en.rst 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/design/modules/backward.md 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/design/modules/batch_norm_op.md 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/design/modules/evaluator.md 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/design/modules/index_cn.rst 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/design/modules/index_en.rst 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/design/modules/infer_var_type.md 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/design/modules/net_op_design.md 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/design/modules/optimizer.md 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/design/modules/prune.md 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/design/modules/python_api.md 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/design/modules/register_grad_op.md 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/design/modules/regularization.md 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/design/modules/selected_rows.md 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/design/motivation/api.md 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/design/motivation/fluid-compiler.png 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/design/motivation/fluid.md 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/design/motivation/fluid_compiler.md 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/design/motivation/index_cn.rst 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/design/motivation/index_en.rst 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/design/multi_devices/index_cn.rst 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/design/multi_devices/index_en.rst 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/design/network/deep_speech_2.md 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/design/network/images/beam_search.png 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/design/network/images/ds2_network.png 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/design/network/index_cn.rst 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/design/network/index_en.rst 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/design/network/sequence_decoder.md 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/design/onnx/onnx_convertor.md 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/design/others/dcgan.png 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/design/others/gan_api.md 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/design/others/graph.md 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/design/others/graph_survey.md 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/design/others/parameters_in_cpp.md 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/design/others/simple_op_design.md 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/design/others/test.dot 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/design/others/test.dot.png 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/dev/api_doc_std_cn.md 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/dev/api_doc_std_en.md 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/dev/ci_build_whl.png 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/dev/contribute_to_paddle_cn.md 已删除 120000 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/dev/contribute_to_paddle_en.md 已删除 120000 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/dev/index_cn.rst 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/dev/index_en.rst 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/dev/name_convention.md 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/dev/new_op_cn.md 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/dev/new_op_en.md 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/dev/new_op_kernel.md 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/dev/op_markdown_format.md 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/dev/releasing_process_cn.md 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/dev/releasing_process_en.md 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/dev/src/fc.py 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/dev/support_new_device.md 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/dev/use_eigen_cn.md 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/dev/use_eigen_en.md 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/dev/versioning_en.md 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/dev/write_docs_cn.rst 已删除 120000 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/dev/write_docs_en.rst 已删除 120000 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/faq/index_cn.rst 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/faq/index_en.rst 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/getstarted/concepts/index_cn.rst 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/getstarted/concepts/index_en.rst 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/getstarted/index_cn.rst 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/getstarted/index_en.rst 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/getstarted/quickstart_cn.rst 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/getstarted/quickstart_en.rst 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/howto/cluster/fluid_recordio.md 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/howto/index_cn.rst 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/howto/index_en.rst 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/howto/inference/index_cn.rst 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/howto/optimization/index_cn.rst 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/howto/optimization/index_en.rst 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/howto/optimization/pprof_1.png 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/howto/optimization/pprof_2.png 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/howto/optimization/timeline.jpeg 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/howto/optimization/timeline_cn.md 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/howto/optimization/timeline_en.md 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/howto/optimization/tracing.jpeg 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/howto/performance/error_clip.md 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/howto/performance/images/profiler.png 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/howto/performance/profiler.md 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/howto/third_party/mkldnn_fluid.md 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/howto/third_party/paddle_nccl.md 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/images/1.png 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/images/2.png 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/images/2_level_rnn.dot 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/images/2_level_rnn.png 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/images/3.png 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/images/4.png 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/images/LoDTensor.png 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/images/asgd.gif 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/images/batch_norm_fork.dot 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/images/batch_norm_fork.png 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/images/batch_norm_op_kernel.png 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/images/beam_search.png 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/images/ci_build_whl.png 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/images/compile_run_time.png 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/images/compiler.graffle 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/images/compiler.png 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/images/control_flow_graph.png 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/images/dataflow_equations.png 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/images/dcgan.png 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/images/deep_learning.png 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/images/dist-graph.graffle 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/images/dist-graph.png 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/images/distributed_architecture.png 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/images/ds2_network.png 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/images/executor.png 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/images/feed_forward.png 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/images/feed_forward_regularized.png 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/images/fluid-compiler.graffle 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/images/fluid-compiler.png 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/images/fluid_examples.png 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/images/fluid_module_1.png 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/images/fluid_module_2.png 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/images/l1_regularization.png 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/images/l2_regularization.png 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/images/layer.png 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/images/local-graph.graffle 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/images/local-graph.png 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/images/local_architecture.graffle 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/images/local_architecture.png 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/images/lookup_table.png 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/images/lookup_table_training.png 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/images/loss_equation.png 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/images/multi-threads.graffle 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/images/multi-threads@3x.png 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/images/multigpu_allreduce.graffle 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/images/multigpu_allreduce.png 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/images/multigpu_before_convert.png 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/images/multiple_reader.png 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/images/op.dot 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/images/op_with_kernel.dot 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/images/operator1.png 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/images/operator2.png 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/images/paddle-compile.graffle 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/images/paddle-compile.png 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/images/place.png 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/images/pprof_1.png 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/images/pprof_2.png 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/images/print_fluid_program.png 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/images/profiler.png 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/images/program_desc1.png 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/images/program_desc2.png 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/images/raw_input.png 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/images/readers.png 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/images/remote_executor.graffle 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/images/remote_executor.png 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/images/rnn.dot 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/images/rnn.jpg 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/images/rnn.png 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/images/rnn_2level_data.dot 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/images/rnn_2level_data.png 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/images/scope_variable_tensor.png 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/images/single-thread@3x.png 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/images/sorted_input.png 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/images/sparse_update.graffle 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/images/sparse_update.png 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/images/test.dot 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/images/test.dot.png 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/images/theta_star.gif 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/images/timeline.jpeg 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/images/tracing.jpeg 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/images/transpiler.png 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/images/user_interface.png 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/index_cn.rst 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/index_en.rst 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/new_docs/advanced_usage/index.rst 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/new_docs/beginners_guide/index.rst 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/new_docs/faq/faq.rst 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/new_docs/faq/index_cn.rst 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/new_docs/user_guides/index.rst 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
doc/fluid/read_source.md 已删除 100644 → 0
浏览文件 @ 0c511700
此差异已折叠。
paddle/fluid/API.spec
浏览文件 @ a0a67893
此差异已折叠。
paddle/fluid/operators/sequence_scatter_op.cc 0 → 100644
浏览文件 @ a0a67893
此差异已折叠。
paddle/fluid/operators/sequence_scatter_op.h 0 → 100644
浏览文件 @ a0a67893
此差异已折叠。
python/paddle/fluid/data_feeder.py
浏览文件 @ a0a67893
此差异已折叠。
python/paddle/fluid/framework.py
浏览文件 @ a0a67893
此差异已折叠。
python/paddle/fluid/io.py
浏览文件 @ a0a67893
此差异已折叠。
python/paddle/fluid/layers/nn.py
浏览文件 @ a0a67893
此差异已折叠。
Markdown is supported
0% 或 .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑！
想要评论请 注册 或