diff --git a/RELEASE.cn.md b/RELEASE.cn.md
index 5deaf230a8f5dd3089993f0fc79b9460fd049750..494c59730dd3c2830514e8924aa3d59a34ac412e 100644
--- a/RELEASE.cn.md
+++ b/RELEASE.cn.md
@@ -1,3 +1,62 @@
+# v0.11.0版本
+
+## PaddlePaddle Fluid
+
+- PaddlePaddle发布版本v0.11.0包含一个新的特性*PaddlePaddle Fluid*. Fluid 是设计用来让用户像Pytorch和Tensorflow Eager Execution一样执行程序。在这些系统中，不再有*模型*这个概念，应用也不再包含一个用于描述Operator图或者一系列层的符号描述，而是像通用程序那样描述训练或者预测的过程。而Fluid与PyTorch或Eager Execution的区别在于Fluid不依赖Python提供的控制流，例如 if-else-then或者for，而是提供了基于C++实现的控制流并暴露了对应的用with语法实现的Python接口。例如：
+
+ https://github.com/PaddlePaddle/Paddle/blob/3df78ed2a98d37f7ae6725894cc7514effd5664b/python/paddle/v2/fluid/tests/test_while_op.py#L36-L44
+
+- 在v0.11.0版本中，我们提供了一个C++类`Executor`用于运行一个Fluid程序。Executor类似一个解释器。在未来的版本中，我们将提升和优化Executor成为一个调试器，就像GDB。并可能提供一些编译器，这个编译器会读取一个上文所描述的应用然后编译成一个等价的
+源代码，这个源代码可以被nvcc编译成可以使用CUDA的二进制，或者被icc编译成可以充分利用Intel CPU的二进制。
+
+
+## 新特点
+
+* 发布 `PaddlePaddle Fluid`。
+* 增加了用于模型预测的C-API。
+* 用Fluid API实现了一个简单的GAN的例子。
+* 增加了关于性能调优的文档。
+* 为`paddle.v2.dataset`下载数据集提供了重试机制.
+* C++中使用protobuf-lite替换protobuf减少了二进制的大小。
+* 发布了新特性 [Elastic Deep Learning (EDL)](https://github.com/PaddlePaddle/cloud/tree/develop/doc/autoscale/experiment).
+* 基于Bazel API利用cmake实现了一个的新的构建系统函数库。
+* 当使用编译选项`WITH_MKL=ON`时自动下载和编译Intel® [MKLML](https://github.com/01org/mkl-dnn/releases/download/v0.11/mklml_lnx_2018.0.1.20171007.tgz) 函数库.
+* [Intel® MKL-DNN on PaddlePaddle](https://github.com/PaddlePaddle/Paddle/tree/develop/doc/design/mkldnn):
+ - 完成了 11个 MKL-DNN 层: Convolution, Fully connectivity, Pooling, ReLU, Tanh, ELU, Softmax, BatchNorm, AddTo, Concat, LRN。
+ - 完成了 3个 MKL-DNN 网络: VGG-19, ResNet-50, GoogleNet
+ - 基于Intel Skylake 6148 CPU的[性能测试](https://github.com/PaddlePaddle/Paddle/blob/develop/benchmark/IntelOptimizedPaddle.md) : 相对于MKLML有2~3倍的训练加速。
+* 增加 [softsign activation](http://www.paddlepaddle.org/docs/develop/documentation/zh/api/v2/config/activation.html#softsign)
+* 增加 [dot product layer](http://www.paddlepaddle.org/docs/develop/documentation/zh/api/v2/config/layer.html#dot-prod)
+* 增加 [L2 distance layer](http://www.paddlepaddle.org/docs/develop/documentation/zh/api/v2/config/layer.html#l2-distance)
+* 增加 [sub-nested sequence layer](http://www.paddlepaddle.org/docs/develop/documentation/zh/api/v2/config/layer.html#sub-nested-seq)
+* 增加 [kmax sequence score layer](http://www.paddlepaddle.org/docs/develop/documentation/zh/api/v2/config/layer.html#kmax-sequence-score)
+* 增加 [sequence slice layer](http://www.paddlepaddle.org/docs/develop/documentation/zh/api/v2/config/layer.html#seq-slice)
+* 增加 [row convolution layer](http://www.paddlepaddle.org/docs/develop/documentation/zh/api/v2/config/layer.html#row-conv)
+* 增加移动端友好的网页
+
+## 改进
+
+* 使用一个Python`whl`包即可安装.
+* [V2 API可以实现用户定制化评估](https://github.com/PaddlePaddle/models/tree/develop/ltr#训练过程中输出自定义评估指标)。
+* 将 `PADDLE_ONLY_CPU` 改为 `PADDLE_WITH_GPU`, 因为我们会支持多种设备。
+* 删除了有一些bug的BarrierStat。
+* 清理和删除了paddle::Parameter中未使用的函数。
+* 删除了ProtoDataProvider。
+* Huber loss同时支持回归和分类。
+* 为sequence pooling 层增加`stride`参数。
+* v2 API自动使用cudnn batch normalization。
+* 可以使用一个固定的参数名共享BN层的参数。
+* 2D convolution operation支持variable-dimension input特性。
+* 重构cmake中关于CUDA的部分并实现自动检测GPU架构的功能。
+* 优化网页导航。
+
+## 错误修复
+
+* 修复ROI pooling的Bug. cc9a761
+* 修复当label是dense vector是AUC变成0的问题. #5274
+* 修复WarpCTC 层的Bug.
+
+
# v0.10.0版本
我们非常高兴发布了PaddlePaddle V0.10.0版，并开发了新的[Python API](http://research.baidu.com/paddlepaddles-new-api-simplifies-deep-learning-programs/)。
diff --git a/RELEASE.md b/RELEASE.md
index 146f7afa7dfbc152500b82fde28445ae3155c16c..5a62c955131007c9f3329d162c20d1b462550019 100644
--- a/RELEASE.md
+++ b/RELEASE.md
@@ -1,3 +1,75 @@
+# Release v0.11.0
+
+## PaddlePaddle Fluid
+
+- Release 0.11.0 includes a new feature *PaddlePaddle Fluid*. Fluid is
+ designed to allow users to program like PyTorch and TensorFlow Eager Execution.
+ In these systems, there is no longer the concept *model* and applications
+ do not include a symbolic description of a graph of operators nor a sequence
+ of layers. Instead, applications look exactly like a usual program that
+ describes a process of training or inference. The difference between
+ Fluid and PyTorch or Eager Execution is that Fluid doesn't rely on Python's
+ control-flow, `if-then-else` nor `for`. Instead, Fluid provides its
+ C++ implementations and their Python binding using the `with` statement. For an example
+
+ https://github.com/PaddlePaddle/Paddle/blob/3df78ed2a98d37f7ae6725894cc7514effd5664b/python/paddle/v2/fluid/tests/test_while_op.py#L36-L44
+
+- In 0.11.0, we provides a C++ class `Executor` to run a Fluid program.
+Executor works like an interpreter. In future version, we will improve
+`Executor` into a debugger like GDB, and we might provide some compilers,
+which, for example, takes an application like the above one, and outputs
+an equivalent C++ source program, which can be compiled using
+[`nvcc`](http://docs.nvidia.com/cuda/cuda-compiler-driver-nvcc/index.html)
+to generate binaries that use CUDA, or using
+[`icc`](https://software.intel.com/en-us/c-compilers) to generate binaries
+that make full use of Intel CPUs.
+
+## New Features
+
+* Release `PaddlePaddle Fluid`.
+* Add C-API for model inference
+* Use fluid API to create a simple GAN demo.
+* Add develop guide about performance tunning.
+* Add retry when download `paddle.v2.dataset`.
+* Linking protobuf-lite not protobuf in C++. Reduce the binary size.
+* Feature [Elastic Deep Learning (EDL)](https://github.com/PaddlePaddle/cloud/tree/develop/doc/autoscale/experiment) released.
+* A new style cmake functions for Paddle. It is based on Bazel API.
+* Automatically download and compile with Intel® [MKLML](https://github.com/01org/mkl-dnn/releases/download/v0.11/mklml_lnx_2018.0.1.20171007.tgz) library as CBLAS when build `WITH_MKL=ON`.
+* [Intel® MKL-DNN on PaddlePaddle](https://github.com/PaddlePaddle/Paddle/tree/develop/doc/design/mkldnn):
+ - Complete 11 MKL-DNN layers: Convolution, Fully connectivity, Pooling, ReLU, Tanh, ELU, Softmax, BatchNorm, AddTo, Concat, LRN.
+ - Complete 3 MKL-DNN networks: VGG-19, ResNet-50, GoogleNet
+ - [Benchmark](https://github.com/PaddlePaddle/Paddle/blob/develop/benchmark/IntelOptimizedPaddle.md) on Intel Skylake 6148 CPU: 2~3x training speedup compared with MKLML.
+* Add the [`softsign` activation](http://www.paddlepaddle.org/docs/develop/documentation/zh/api/v2/config/activation.html#softsign).
+* Add the [dot product layer](http://www.paddlepaddle.org/docs/develop/documentation/zh/api/v2/config/layer.html#dot-prod).
+* Add the [L2 distance layer](http://www.paddlepaddle.org/docs/develop/documentation/zh/api/v2/config/layer.html#l2-distance).
+* Add the [sub-nested sequence layer](http://www.paddlepaddle.org/docs/develop/documentation/zh/api/v2/config/layer.html#sub-nested-seq).
+* Add the [kmax sequence score layer](http://www.paddlepaddle.org/docs/develop/documentation/zh/api/v2/config/layer.html#kmax-sequence-score).
+* Add the [sequence slice layer](http://www.paddlepaddle.org/docs/develop/documentation/zh/api/v2/config/layer.html#seq-slice).
+* Add the [row convolution layer](http://www.paddlepaddle.org/docs/develop/documentation/zh/api/v2/config/layer.html#row-conv)
+* Add mobile friendly webpages.
+
+## Improvements
+
+* Build and install using a single `whl` package.
+* [Custom evaluating in V2 API](https://github.com/PaddlePaddle/models/tree/develop/ltr#训练过程中输出自定义评估指标).
+* Change `PADDLE_ONLY_CPU` to `PADDLE_WITH_GPU`, since we will support many kinds of devices.
+* Remove buggy BarrierStat.
+* Clean and remove unused functions in paddle::Parameter.
+* Remove ProtoDataProvider.
+* Huber loss supports both regression and classification.
+* Add the `stride` parameter for sequence pooling layers.
+* Enable v2 API use cudnn batch normalization automatically.
+* The BN layer's parameter can be shared by a fixed the parameter name.
+* Support variable-dimension input feature for 2D convolution operation.
+* Refine cmake about CUDA to automatically detect GPU architecture.
+* Improved website navigation.
+
+## Bug Fixes
+
+* Fix bug in ROI pooling. cc9a761
+* Fix AUC is zero when label is dense vector. #5274
+* Fix bug in WarpCTC layer.
+
# Release v0.10.0
We are glad to release version 0.10.0. In this version, we are happy to release the new
diff --git a/benchmark/IntelOptimizedPaddle.md b/benchmark/IntelOptimizedPaddle.md
index 26930a76377397cc36af8cf411b49b02e4f67748..8ee7fd28c58f2a2bcb82040eb824a37062bd4e9c 100644
--- a/benchmark/IntelOptimizedPaddle.md
+++ b/benchmark/IntelOptimizedPaddle.md
@@ -19,6 +19,8 @@ On each machine, we will test and compare the performance of training on single
## Benchmark Model
### Server
+
+#### Training
Test on batch size 64, 128, 256 on Intel(R) Xeon(R) Gold 6148 CPU @ 2.40GHz
Input image size - 3 * 224 * 224, Time: images/second
@@ -53,5 +55,33 @@ Input image size - 3 * 224 * 224, Time: images/second
+#### Inference
+Test on batch size 1, 2, 4, 8, 16 on Intel(R) Xeon(R) Gold 6148 CPU @ 2.40GHz
+- VGG-19
+
+| BatchSize | 1 | 2 | 4 | 8 | 16 |
+|-----------|-------|-------|-------|-------|-------|
+| OpenBLAS | 1.07 | 1.08 | 1.06 | 0.88 | 0.65 |
+| MKLML | 5.58 | 9.80 | 15.15 | 21.21 | 28.67 |
+| MKL-DNN | 75.07 | 88.64 | 82.58 | 92.29 | 96.75 |
+
+- ResNet-50
+
+| BatchSize | 1 | 2 | 4 | 8 | 16 |
+|-----------|-------|--------|--------|--------|--------|
+| OpenBLAS | 3.35 | 3.19 | 3.09 | 2.55 | 1.96 |
+| MKLML | 6.33 | 12.02 | 22.88 | 40.53 | 63.09 |
+| MKL-DNN | 107.83| 148.84 | 177.78 | 189.35 | 217.69 |
+
+
+- GoogLeNet
+
+| BatchSize | 1 | 2 | 4 | 8 | 16 |
+|-----------|--------|--------|--------|--------|--------|
+| OpenBLAS | 12.04 | 11.31 | 10.00 | 9.07 | 4.34 |
+| MKLML | 22.74 | 41.56 | 81.22 | 133.47 | 210.53 |
+| MKL-DNN | 175.10 | 272.92 | 450.70 | 512.00 | 600.94 |
+
+
### Laptop
TBD
diff --git a/cmake/cblas.cmake b/cmake/cblas.cmake
index b21fc43904d9aafe9f7d019dfbe5b1c0d3f9e2d6..13294c054849092399c60917673c0a6d3dafa85c 100644
--- a/cmake/cblas.cmake
+++ b/cmake/cblas.cmake
@@ -17,7 +17,7 @@ if(WITH_MKLML AND MKLML_INC_DIR AND MKLML_LIB)
set(CBLAS_INC_DIR ${MKLML_INC_DIR})
set(CBLAS_LIBRARIES ${MKLML_LIB})
- add_definitions(-DPADDLE_USE_MKLML)
+ add_definitions(-DPADDLE_WITH_MKLML)
add_definitions(-DLAPACK_FOUND)
message(STATUS "Found cblas and lapack in MKLML "
diff --git a/cmake/external/mkldnn.cmake b/cmake/external/mkldnn.cmake
index fc52d339d7a336b44c97f2e0a9fc8d6604854365..5d24caebdcc5a28823164d718fb1628be5c4179d 100644
--- a/cmake/external/mkldnn.cmake
+++ b/cmake/external/mkldnn.cmake
@@ -67,5 +67,5 @@ ADD_LIBRARY(mkldnn SHARED IMPORTED GLOBAL)
SET_PROPERTY(TARGET mkldnn PROPERTY IMPORTED_LOCATION ${MKLDNN_LIB})
ADD_DEPENDENCIES(mkldnn ${MKLDNN_PROJECT})
MESSAGE(STATUS "MKLDNN library: ${MKLDNN_LIB}")
-add_definitions(-DPADDLE_USE_MKLDNN)
+add_definitions(-DPADDLE_WITH_MKLDNN)
LIST(APPEND external_project_dependencies mkldnn)
diff --git a/paddle/api/CMakeLists.txt b/paddle/api/CMakeLists.txt
index d6b8464100d4497876aa3f6f7cbc666aafae4bfc..cf84568ecdf1227b0d0ed3606a4a9a6e5186af72 100644
--- a/paddle/api/CMakeLists.txt
+++ b/paddle/api/CMakeLists.txt
@@ -25,8 +25,18 @@ FILE(GLOB PY_PADDLE_PYTHON_FILES ${PADDLE_SOURCE_DIR}/paddle/py_paddle/*.py)
SET_SOURCE_FILES_PROPERTIES(Paddle.i PROPERTIES CPLUSPLUS ON)
+SET(SWIG_NEED_FLAGS
+ -ftls-model=global-dynamic
+ -Wno-parentheses-equality
+ -Wno-self-assign
+ -Wno-maybe-uninitialized
+ -Wno-missing-field-initializers)
+ FOREACH(flag ${SWIG_NEED_FLAGS})
+ safe_set_cxxflag(SWIG_CXX_FLAGS ${flag})
+ENDFOREACH()
+
SET(CMAKE_SWIG_OUTDIR ${CMAKE_CURRENT_BINARY_DIR})
-SET(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -Wno-parentheses-equality -Wno-missing-field-initializers -Wno-self-assign -ftls-model=global-dynamic")
+SET(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} ${SWIG_CXX_FLAGS}")
SET(SWIG_MODULE_swig_paddle_EXTRA_DEPS
paddle_parameter
diff --git a/paddle/framework/backward.cc b/paddle/framework/backward.cc
index 7294ba1a9c55bd628493ecb23ba806c4442adad7..a17036c6527da3a4a32f021a57542b6b6d68a395 100644
--- a/paddle/framework/backward.cc
+++ b/paddle/framework/backward.cc
@@ -190,8 +190,9 @@ static std::unique_ptr BackwardRecursive(
// collect all the offset for each alias,
// insert a sum operator to add all aliases to output
insert_position.push_back(
- {dup_op.back(), OpRegistry::CreateOp("sum", {{"X", dup_outputs}},
- {{"Out", {name}}}, {})});
+ {dup_op.back(),
+ OpRegistry::CreateOp("sum", {{"X", dup_outputs}}, {{"Out", {name}}},
+ AttributeMap{})});
}
// make sure the inserted `sum` ops follow the BFS order.
@@ -216,7 +217,8 @@ static std::unique_ptr BackwardRecursive(
// If part of input gradient of that operator is not calculated, fill
// zero variables to that input gradient.
net->AppendOp(OpRegistry::CreateOp("fill_zeros_like", {{"X", {prefix}}},
- {{"Y", {grad_input}}}, {}));
+ {{"Y", {grad_input}}},
+ AttributeMap{}));
}
return false;
});
@@ -392,8 +394,9 @@ std::vector> MakeOpGrad(
0, in_name.size() - sizeof(kGradVarSuffix) / sizeof(char) + 1);
std::string new_name = prefix + kZeroVarSuffix;
desc->Rename(in_name, new_name);
- std::unique_ptr fill_zeros_op(new OpDescBind(
- "fill_zeros_like", {{"X", {prefix}}}, {{"Y", {new_name}}}, {}));
+ std::unique_ptr fill_zeros_op(
+ new OpDescBind("fill_zeros_like", {{"X", {prefix}}},
+ {{"Y", {new_name}}}, AttributeMap{}));
pending_fill_zeros_ops.push_back(std::move(fill_zeros_op));
}
}
@@ -483,8 +486,9 @@ std::vector> MakeBlockBackward(
sum_op_inputs.emplace_back(new_name);
next_g_name = sum_op_inputs.back();
}
- std::unique_ptr sum_op(new OpDescBind(
- "sum", {{"X", sum_op_inputs}}, {{"Out", {out_name}}}, {}));
+ std::unique_ptr sum_op(
+ new OpDescBind("sum", {{"X", sum_op_inputs}}, {{"Out", {out_name}}},
+ AttributeMap{}));
pending_sum_ops.push_back({dup_op.back(), std::move(sum_op)});
}
}
diff --git a/paddle/framework/backward_test.cc b/paddle/framework/backward_test.cc
index 2b858f5ea0874d7bf1a9cf38529f5d0d70cca7f2..9fe49881d5b740655432f6e83a7886878ceb17e8 100644
--- a/paddle/framework/backward_test.cc
+++ b/paddle/framework/backward_test.cc
@@ -106,15 +106,15 @@ class FcOp : public operators::NetOp {
FcOp(const std::string &type, const VariableNameMap &inputs,
const VariableNameMap &outputs, const AttributeMap &attrs)
: NetOp(type, inputs, outputs, attrs) {
- AppendOp(OpRegistry::CreateOp("mul",
- {{"X", {Input("X")}}, {"Y", {Input("W")}}},
- {{"Out", {Output("mul_result")}}}, {}));
+ AppendOp(OpRegistry::CreateOp(
+ "mul", {{"X", {Input("X")}}, {"Y", {Input("W")}}},
+ {{"Out", {Output("mul_result")}}}, AttributeMap{}));
auto input_b = Inputs("b");
std::string before_act = "mul_result";
if (input_b.size() != 0) {
AppendOp(OpRegistry::CreateOp(
"rowwise_add", {{"X", {Output("mul_result")}}, {"b", {input_b[0]}}},
- {{"Out", {Output("add_result")}}}, {}));
+ {{"Out", {Output("add_result")}}}, AttributeMap{}));
before_act = "add_result";
} else {
auto out_varname = Output("add_result");
@@ -124,7 +124,7 @@ class FcOp : public operators::NetOp {
}
AppendOp(OpRegistry::CreateOp("sigmoid", {{"X", {Output(before_act)}}},
- {{"Out", {Output("Out")}}}, {}));
+ {{"Out", {Output("Out")}}}, AttributeMap{}));
CompleteAddOp(false);
}
};
@@ -278,8 +278,9 @@ REGISTER_OPERATOR(scale, f::NoneOp);
REGISTER_OP_CPU_KERNEL(scale, f::NoneKernel);
TEST(Backward, simple_op_not_need_grad) {
- auto fwd = f::OpRegistry::CreateOp(
- "rowwise_add", {{"X", {"x"}}, {"b", {"b"}}}, {{"Out", {"out"}}}, {});
+ auto fwd =
+ f::OpRegistry::CreateOp("rowwise_add", {{"X", {"x"}}, {"b", {"b"}}},
+ {{"Out", {"out"}}}, f::AttributeMap{});
ASSERT_NE(fwd, nullptr);
auto gop = f::Backward(*fwd, {"x"});
ASSERT_EQ(gop->Output(f::GradVarName("X")), f::kEmptyVarName);
@@ -296,9 +297,10 @@ TEST(Backward, net_fc_backward_normal) {
{{"mul_result", {"mul_res"}},
{"add_result", {"add_re"}},
{"Out", {"out"}}},
- {});
+ f::AttributeMap{});
ASSERT_NE(fwd, nullptr);
- std::shared_ptr gop = f::Backward(*fwd, {});
+ std::shared_ptr gop =
+ f::Backward(*fwd, std::unordered_set{});
ASSERT_TRUE(gop->IsNetOp());
auto net = static_cast(gop.get());
@@ -322,9 +324,10 @@ TEST(Backward, net_fc_backward_not_have_b) {
{{"mul_result", {"mul_res"}},
{"add_result", {"add_res"}},
{"Out", {"tmp"}}},
- {});
+ f::AttributeMap{});
ASSERT_NE(fwd, nullptr);
- std::shared_ptr gop = f::Backward(*fwd, {});
+ std::shared_ptr gop =
+ f::Backward(*fwd, std::unordered_set{});
ASSERT_TRUE(gop->IsNetOp());
auto net = static_cast(gop.get());
@@ -346,13 +349,13 @@ TEST(Backward, net_input_of_network_not_need_grad) {
{{"mul_result", {"mul_tmp_0"}},
{"add_result", {"add_tmp_0"}},
{"Out", {"hidden0"}}},
- {}));
+ f::AttributeMap{}));
net.AppendOp(f::OpRegistry::CreateOp(
"fc", {{"X", {"hidden0"}}, {"W", {"W2"}}, {"b", {"b2"}}},
{{"mul_result", {"mul_tmp_1"}},
{"add_result", {"add_tmp_1"}},
{"Out", {"hidden1"}}},
- {}));
+ f::AttributeMap{}));
net.CompleteAddOp();
auto bwd = Backward(net, {"x"}); // x@GRAD is not need.
ASSERT_TRUE(bwd->IsNetOp());
@@ -381,12 +384,13 @@ TEST(Backward, net_input_of_network_not_need_grad) {
TEST(Backward, net_shared_weight) {
ops::NetOp net;
net.AppendOp(f::OpRegistry::CreateOp("mul", {{"X", {"x"}}, {"Y", {"w"}}},
- {{"Out", {"out"}}}, {}));
+ {{"Out", {"out"}}}, f::AttributeMap{}));
net.AppendOp(f::OpRegistry::CreateOp("mul", {{"X", {"out"}}, {"Y", {"w"}}},
- {{"Out", {"FinalOut"}}}, {}));
+ {{"Out", {"FinalOut"}}},
+ f::AttributeMap{}));
net.CompleteAddOp();
- auto bwd = f::Backward(net, {});
+ auto bwd = f::Backward(net, std::unordered_set{});
ASSERT_TRUE(bwd->IsNetOp());
auto bwd_net = static_cast(bwd.get());
ASSERT_EQ(3UL, bwd_net->ops_.size());
@@ -394,8 +398,9 @@ TEST(Backward, net_shared_weight) {
}
TEST(Backward, op_all_input_are_not_need) {
- auto fwd = f::OpRegistry::CreateOp(
- "rowwise_add", {{"X", {"x"}}, {"b", {"b"}}}, {{"Out", {"out"}}}, {});
+ auto fwd =
+ f::OpRegistry::CreateOp("rowwise_add", {{"X", {"x"}}, {"b", {"b"}}},
+ {{"Out", {"out"}}}, f::AttributeMap{});
auto backward = f::Backward(*fwd, {"x", "b"});
ASSERT_TRUE(backward->IsNetOp());
auto net = static_cast(backward.get());
@@ -403,8 +408,9 @@ TEST(Backward, op_all_input_are_not_need) {
}
TEST(Backward, op_all_output_are_not_need) {
- auto fwd = f::OpRegistry::CreateOp(
- "rowwise_add", {{"X", {"x"}}, {"b", {"b"}}}, {{"Out", {"out"}}}, {});
+ auto fwd =
+ f::OpRegistry::CreateOp("rowwise_add", {{"X", {"x"}}, {"b", {"b"}}},
+ {{"Out", {"out"}}}, f::AttributeMap{});
auto backward = f::Backward(*fwd, {"out"});
ASSERT_TRUE(backward->IsNetOp());
auto net = static_cast(backward.get());
@@ -412,8 +418,9 @@ TEST(Backward, op_all_output_are_not_need) {
}
TEST(Backward, op_part_of_output_are_not_need) {
- auto fwd = f::OpRegistry::CreateOp("many_output_op", {{"x", {"X"}}},
- {{"y", {"Y"}}, {"z", {"Z"}}}, {});
+ auto fwd =
+ f::OpRegistry::CreateOp("many_output_op", {{"x", {"X"}}},
+ {{"y", {"Y"}}, {"z", {"Z"}}}, f::AttributeMap{});
auto backward = f::Backward(*fwd, {"Z"});
ASSERT_TRUE(backward->IsNetOp());
auto net = static_cast(backward.get());
@@ -437,7 +444,7 @@ TEST(Backward, op_part_of_output_are_not_need) {
TEST(Backward, op_part_of_input_are_not_need) {
auto fwd = f::OpRegistry::CreateOp("mul", {{"X", {"a"}}, {"Y", {"b"}}},
- {{"Out", {"out"}}}, {});
+ {{"Out", {"out"}}}, f::AttributeMap{});
auto backward = f::Backward(*fwd, {"a"});
auto &grad_mul = *backward;
ASSERT_EQ(grad_mul.Type(), "mul_grad");
@@ -458,19 +465,19 @@ TEST(Backward, linear_net_intermediate_variable_has_no_grad) {
{{"mul_result", {"mul_out1"}},
{"add_result", {"add_out1"}},
{"Out", {"out1"}}},
- {}));
+ f::AttributeMap{}));
net.AppendOp(f::OpRegistry::CreateOp(
"fc", {{"X", {"out1"}}, {"W", {"w2"}}, {"b", {"b2"}}},
{{"mul_result", {"mul_out2"}},
{"add_result", {"tmp_out2"}},
{"Out", {"out2"}}},
- {}));
+ f::AttributeMap{}));
net.AppendOp(f::OpRegistry::CreateOp(
"fc", {{"X", {"out2"}}, {"W", {"w3"}}, {"b", {"b3"}}},
{{"mul_result", {"mul_out3"}},
{"add_result", {"tmp_out3"}},
{"Out", {"out3"}}},
- {}));
+ f::AttributeMap{}));
net.CompleteAddOp();
auto backward = f::Backward(net, {"mul_out2", "tmp_out2", "out2"});
@@ -509,7 +516,8 @@ TEST(Backward, simple_single_op) {
auto target = f::VarDescBind("out");
target.SetShape({1});
- auto var_to_grad = AppendBackward(program, target, {});
+ auto var_to_grad =
+ AppendBackward(program, target, std::unordered_set{});
ASSERT_EQ(block->AllOps().size(), 3UL);
f::OpDescBind *fill_op = block->AllOps()[1];
@@ -546,7 +554,7 @@ TEST(Backward, default_attribute) {
auto target = f::VarDescBind("out");
target.SetShape({1});
- AppendBackward(program, target, {});
+ AppendBackward(program, target, std::unordered_set{});
ASSERT_EQ(block->AllOps().size(), 3UL);
EXPECT_EQ(boost::get(op->GetAttr("x_num_col_dims")), 1);
@@ -585,7 +593,8 @@ TEST(Backward, simple_mult_op) {
auto target = f::VarDescBind("out3");
target.SetShape({1});
size_t forward_len = block->AllOps().size();
- auto var_to_grad = AppendBackward(program, target, {});
+ auto var_to_grad =
+ AppendBackward(program, target, std::unordered_set{});
ASSERT_EQ(block->AllOps().size(), 6UL + 1);
f::OpDescBind *fill_op = block->AllOps()[forward_len];
@@ -817,7 +826,8 @@ TEST(Backward, shared_var) {
auto target = f::VarDescBind("out3");
target.SetShape({1});
size_t forward_len = block->AllOps().size();
- auto var_to_grad = AppendBackward(program, target, {});
+ auto var_to_grad =
+ AppendBackward(program, target, std::unordered_set{});
ASSERT_EQ(block->AllOps().size(), 8UL);
f::OpDescBind *fill_op = block->AllOps()[forward_len];
diff --git a/paddle/framework/op_desc.cc b/paddle/framework/op_desc.cc
index cde3f1ac2e411a79f948e0c15a90ec8278a93a29..7ba1e3e4e3270f4cd88e41e245f24c3cfc8aaab7 100644
--- a/paddle/framework/op_desc.cc
+++ b/paddle/framework/op_desc.cc
@@ -316,8 +316,8 @@ static void InitInferShapeFuncs() {
for (auto &kern_pair : OperatorWithKernel::AllOpKernels()) {
auto op_type = kern_pair.first;
auto &op_info = info_map.at(op_type);
- auto op =
- static_cast(op_info.Creator()("", {}, {}, {}));
+ auto op = static_cast(op_info.Creator()(
+ "", VariableNameMap{}, VariableNameMap{}, AttributeMap{}));
if (op_info.infer_shape_) { // infer_shape has been registered.
continue;
}
diff --git a/paddle/framework/operator_test.cc b/paddle/framework/operator_test.cc
index 1e19f82b341768142258ba4a5dfa246d87ba4c43..59ddbc77912decf56192e542916ab012c929ed0a 100644
--- a/paddle/framework/operator_test.cc
+++ b/paddle/framework/operator_test.cc
@@ -261,7 +261,9 @@ class OperatorClone : public paddle::framework::OperatorBase {
};
TEST(Operator, Clone) {
- OperatorClone a("ABC", {}, {}, {});
+ OperatorClone a("ABC", paddle::framework::VariableNameMap{},
+ paddle::framework::VariableNameMap{},
+ paddle::framework::AttributeMap{});
auto b = a.Clone();
ASSERT_EQ(a.Type(), b->Type());
}
diff --git a/paddle/framework/prune_test.cc b/paddle/framework/prune_test.cc
index 5988874809f51c09b3d3d279be6c1e8d43d7a782..f21df37a292fd1e039ee8f8fa26244e26c978cae 100644
--- a/paddle/framework/prune_test.cc
+++ b/paddle/framework/prune_test.cc
@@ -54,7 +54,8 @@ TEST(Prune, one_operator) {
f::ProgramDescBind program;
f::BlockDescBind *block = program.MutableBlock(0);
- AddOp("one_one", {{"input", {"a"}}}, {{"output", {"b"}}}, {}, block);
+ AddOp("one_one", {{"input", {"a"}}}, {{"output", {"b"}}}, f::AttributeMap{},
+ block);
f::ProgramDesc *pdesc = program.Proto();
f::ProgramDesc pruned;
@@ -71,10 +72,14 @@ TEST(Prune, forward) {
f::ProgramDescBind program;
f::BlockDescBind *block = program.MutableBlock(0);
- AddOp("one_one", {{"input", {"a"}}}, {{"output", {"b"}}}, {}, block);
- AddOp("one_one", {{"input", {"b"}}}, {{"output", {"c"}}}, {}, block);
- AddOp("one_one", {{"input", {"c"}}}, {{"output", {"d"}}}, {}, block);
- AddOp("one_one", {{"input", {"d"}}}, {{"output", {"e"}}}, {}, block);
+ AddOp("one_one", {{"input", {"a"}}}, {{"output", {"b"}}}, f::AttributeMap{},
+ block);
+ AddOp("one_one", {{"input", {"b"}}}, {{"output", {"c"}}}, f::AttributeMap{},
+ block);
+ AddOp("one_one", {{"input", {"c"}}}, {{"output", {"d"}}}, f::AttributeMap{},
+ block);
+ AddOp("one_one", {{"input", {"d"}}}, {{"output", {"e"}}}, f::AttributeMap{},
+ block);
f::ProgramDesc *pdesc = program.Proto();
@@ -90,11 +95,14 @@ TEST(Prune, multi_input_op) {
f::ProgramDescBind program;
f::BlockDescBind *block = program.MutableBlock(0);
- AddOp("one_one", {{"input", {"a0"}}}, {{"output", {"b0"}}}, {}, block);
- AddOp("one_one", {{"input", {"a1"}}}, {{"output", {"b1"}}}, {}, block);
- AddOp("one_one", {{"input", {"a2"}}}, {{"output", {"b2"}}}, {}, block);
- AddOp("three_one", {{"input", {"b0", "b1", "b2"}}}, {{"output", {"c"}}}, {},
+ AddOp("one_one", {{"input", {"a0"}}}, {{"output", {"b0"}}}, f::AttributeMap{},
+ block);
+ AddOp("one_one", {{"input", {"a1"}}}, {{"output", {"b1"}}}, f::AttributeMap{},
block);
+ AddOp("one_one", {{"input", {"a2"}}}, {{"output", {"b2"}}}, f::AttributeMap{},
+ block);
+ AddOp("three_one", {{"input", {"b0", "b1", "b2"}}}, {{"output", {"c"}}},
+ f::AttributeMap{}, block);
f::ProgramDesc *pdesc = program.Proto();
pdesc->mutable_blocks(0)->mutable_ops(3)->set_is_target(true);
@@ -108,9 +116,12 @@ TEST(Prune, multi_output_op) {
f::ProgramDescBind program;
f::BlockDescBind *block = program.MutableBlock(0);
- AddOp("one_two", {{"input", {"a"}}}, {{"output", {"b", "c"}}}, {}, block);
- AddOp("one_one", {{"input", {"b"}}}, {{"output", {"b1"}}}, {}, block);
- AddOp("one_one", {{"input", {"c"}}}, {{"output", {"c1"}}}, {}, block);
+ AddOp("one_two", {{"input", {"a"}}}, {{"output", {"b", "c"}}},
+ f::AttributeMap{}, block);
+ AddOp("one_one", {{"input", {"b"}}}, {{"output", {"b1"}}}, f::AttributeMap{},
+ block);
+ AddOp("one_one", {{"input", {"c"}}}, {{"output", {"c1"}}}, f::AttributeMap{},
+ block);
f::ProgramDesc *pdesc = program.Proto();
pdesc->mutable_blocks(0)->mutable_ops(2)->set_is_target(true);
@@ -124,9 +135,12 @@ TEST(Prune, multi_target) {
f::ProgramDescBind program;
f::BlockDescBind *block = program.MutableBlock(0);
- AddOp("one_two", {{"input", {"a"}}}, {{"output", {"b", "c"}}}, {}, block);
- AddOp("one_one", {{"input", {"b"}}}, {{"output", {"b1"}}}, {}, block);
- AddOp("one_one", {{"input", {"c"}}}, {{"output", {"c1"}}}, {}, block);
+ AddOp("one_two", {{"input", {"a"}}}, {{"output", {"b", "c"}}},
+ f::AttributeMap{}, block);
+ AddOp("one_one", {{"input", {"b"}}}, {{"output", {"b1"}}}, f::AttributeMap{},
+ block);
+ AddOp("one_one", {{"input", {"c"}}}, {{"output", {"c1"}}}, f::AttributeMap{},
+ block);
f::ProgramDesc *pdesc = program.Proto();
pdesc->mutable_blocks(0)->mutable_ops(1)->set_is_target(true);
diff --git a/paddle/gserver/activations/ActivationFunction.cpp b/paddle/gserver/activations/ActivationFunction.cpp
index f5a41b66bf09a4abc5ae7b64f227ca52461408f5..57c890e4884da38e2087d89dc199e20af51495ea 100644
--- a/paddle/gserver/activations/ActivationFunction.cpp
+++ b/paddle/gserver/activations/ActivationFunction.cpp
@@ -24,7 +24,7 @@ limitations under the License. */
#include "paddle/utils/ClassRegistrar.h"
#include "paddle/utils/Logging.h"
-#ifdef PADDLE_USE_MKLDNN
+#ifdef PADDLE_WITH_MKLDNN
#include "MKLDNNActivation.h"
#endif
@@ -490,7 +490,7 @@ Error __must_check backward(Argument& act) {
END_DEFINE_ACTIVATION(log)
ActivationFunction* ActivationFunction::create(const std::string& type) {
-#ifdef PADDLE_USE_MKLDNN
+#ifdef PADDLE_WITH_MKLDNN
if (!type.empty() && type.compare(0, 7, "mkldnn_") == 0) {
return MKLDNNActivation::create(type);
}
diff --git a/paddle/gserver/gradientmachines/NeuralNetwork.cpp b/paddle/gserver/gradientmachines/NeuralNetwork.cpp
index be112b41239cace3fa9b9ee97923f8c3c7a9a98f..68bf37d59db65ddc8096e2db3391be25c37b57e6 100644
--- a/paddle/gserver/gradientmachines/NeuralNetwork.cpp
+++ b/paddle/gserver/gradientmachines/NeuralNetwork.cpp
@@ -20,7 +20,7 @@ limitations under the License. */
#include "paddle/utils/Logging.h"
#include "paddle/utils/Stat.h"
-#ifdef PADDLE_USE_MKLDNN
+#ifdef PADDLE_WITH_MKLDNN
#include "paddle/gserver/layers/MKLDNNLayer.h"
#endif
@@ -307,7 +307,7 @@ void NeuralNetwork::backward(const UpdateCallback& callback) {
}
void NeuralNetwork::finish() {
-#ifdef PADDLE_USE_MKLDNN
+#ifdef PADDLE_WITH_MKLDNN
FOR_EACH_R(layer, layers_) {
MKLDNNLayerPtr dnnLayer = std::dynamic_pointer_cast(*layer);
if (dnnLayer) {
diff --git a/paddle/math/Allocator.h b/paddle/math/Allocator.h
index 94ef561f066a127496e2849a419835e175c526d7..17563bf5e1649361b83b896bf864b922296a5487 100644
--- a/paddle/math/Allocator.h
+++ b/paddle/math/Allocator.h
@@ -48,7 +48,7 @@ public:
*/
virtual void* alloc(size_t size) {
void* ptr;
-#ifdef PADDLE_USE_MKLDNN
+#ifdef PADDLE_WITH_MKLDNN
// refer to https://github.com/01org/mkl-dnn/blob/master/include/mkldnn.hpp
// memory alignment
CHECK_EQ(posix_memalign(&ptr, 4096ul, size), 0);
diff --git a/paddle/math/MathFunctions.cpp b/paddle/math/MathFunctions.cpp
index ba86eacbb5d53ee43a60d2cd1dd922333a5d48f0..28ab54b450c96b4bdefdf36813595766162b1434 100644
--- a/paddle/math/MathFunctions.cpp
+++ b/paddle/math/MathFunctions.cpp
@@ -206,7 +206,7 @@ double dotProduct(const int n, const double* x, const double* y) {
}
#endif
-#if defined(PADDLE_USE_MKLML)
+#if defined(PADDLE_WITH_MKLML)
template <>
void vExp(const int n, const float* a, float* r) {
diff --git a/paddle/math/MathFunctions.h b/paddle/math/MathFunctions.h
index f6e77029bdd75a602f88b688ca810f47ba4ee615..29fe36e3a4bd5e5d372480950a03142822262d41 100644
--- a/paddle/math/MathFunctions.h
+++ b/paddle/math/MathFunctions.h
@@ -15,7 +15,7 @@ limitations under the License. */
#ifndef MATHFUNCTIONS_H_
#define MATHFUNCTIONS_H_
-#ifdef PADDLE_USE_MKLML
+#ifdef PADDLE_WITH_MKLML
#include
#include
#include
diff --git a/paddle/math/tests/CMakeLists.txt b/paddle/math/tests/CMakeLists.txt
index 215bac12715ca789b51f58f6817617cb9fe101ef..dcd2a34583417993a4bf2976f7a3bc5a10d496ac 100644
--- a/paddle/math/tests/CMakeLists.txt
+++ b/paddle/math/tests/CMakeLists.txt
@@ -34,4 +34,4 @@ add_simple_unittest(test_FPException)
add_simple_unittest(test_GpuProfiler)
add_simple_unittest(test_BaseMatrix)
add_simple_unittest(test_Matrix)
-cc_test(test_float16 SRCS test_float16.cpp)
+add_simple_unittest(test_float16)
diff --git a/paddle/memory/detail/system_allocator.cc b/paddle/memory/detail/system_allocator.cc
index b543b767e8015f3fd0449fdf4c2922701837d3dc..6a815a1b57db1d833781ca224f34e4559af9b9a5 100644
--- a/paddle/memory/detail/system_allocator.cc
+++ b/paddle/memory/detail/system_allocator.cc
@@ -43,7 +43,7 @@ void* CPUAllocator::Alloc(size_t& index, size_t size) {
void* p;
-#ifdef PADDLE_USE_MKLDNN
+#ifdef PADDLE_WITH_MKLDNN
// refer to https://github.com/01org/mkl-dnn/blob/master/include/mkldnn.hpp
// memory alignment
PADDLE_ENFORCE_EQ(posix_memalign(&p, 4096ul, size), 0);
diff --git a/paddle/operators/conditional_block_op.cc b/paddle/operators/conditional_block_op.cc
index d5b124682d755ffb39f32c9f001a3cf113a01a2c..03c58a7eab8b2071a3a0b75ac0c665e32ef39876 100644
--- a/paddle/operators/conditional_block_op.cc
+++ b/paddle/operators/conditional_block_op.cc
@@ -142,9 +142,9 @@ class ConditionalBlockGradOp : public ConditionalOp {
continue;
}
auto new_in_grad_name = cur_scope.Rename(in_grad_name);
- auto assign =
- framework::OpRegistry::CreateOp("assign", {{"X", {new_in_grad_name}}},
- {{"Out", {out_grad_name}}}, {});
+ auto assign = framework::OpRegistry::CreateOp(
+ "assign", {{"X", {new_in_grad_name}}}, {{"Out", {out_grad_name}}},
+ framework::AttributeMap{});
assign->Run(cur_scope, dev_ctx);
cur_scope.Rename(new_in_grad_name, in_grad_name);
}
diff --git a/paddle/operators/math/math_function.cc b/paddle/operators/math/math_function.cc
index 2e333a8cde721f8e65dbf2cf5e3aac6272172cc0..e099a6a43917f2c1213ecb1f07965ee97ff195db 100644
--- a/paddle/operators/math/math_function.cc
+++ b/paddle/operators/math/math_function.cc
@@ -132,7 +132,7 @@ void matmul(
matrix_b.data(), beta, matrix_out->data());
}
-#ifdef PADDLE_USE_MKLML
+#ifdef PADDLE_WITH_MKLML
// Use cblas_{s,d}gemm_batched if available: Run with 1 group of size batchSize.
template <>
void batched_gemm(
diff --git a/paddle/operators/math/math_function.h b/paddle/operators/math/math_function.h
index 5a42854f22234629b3405ec2397143ef761a9d08..f2b025b78b68decee43fa1f80378e51eaf4c90de 100644
--- a/paddle/operators/math/math_function.h
+++ b/paddle/operators/math/math_function.h
@@ -13,7 +13,7 @@ See the License for the specific language governing permissions and
limitations under the License. */
#pragma once
-#ifdef PADDLE_USE_MKLML
+#ifdef PADDLE_WITH_MKLML
#include
#include
#include
diff --git a/paddle/operators/net_op.h b/paddle/operators/net_op.h
index ebeb262d9621fa35c870b6407992f6b6d2bf7c70..8935751f15ccc4861c9e06d8d9031c8dff1a4af3 100644
--- a/paddle/operators/net_op.h
+++ b/paddle/operators/net_op.h
@@ -38,7 +38,10 @@ namespace operators {
class NetOp : public framework::OperatorBase {
public:
static const char kAll[];
- NetOp() : framework::OperatorBase("plain_net", {}, {}, {}) {}
+ NetOp()
+ : framework::OperatorBase("plain_net", framework::VariableNameMap{},
+ framework::VariableNameMap{},
+ framework::AttributeMap{}) {}
NetOp(const std::string& type, const framework::VariableNameMap& inputs,
const framework::VariableNameMap& outputs,
diff --git a/paddle/operators/net_op_test.cc b/paddle/operators/net_op_test.cc
index 63bebd5b44719868a38ddf2b023955d1ab05245c..22fba9568d018586b4622884b7d6145fd646adb0 100644
--- a/paddle/operators/net_op_test.cc
+++ b/paddle/operators/net_op_test.cc
@@ -38,10 +38,10 @@ TEST(OpKernel, all) {
net->AppendOp(std::unique_ptr(
new TestOp("test", {{"X", {"x"}}, {"W", {"w1"}}, {"b", {"b1"}}},
- {{"Out", {"y"}}}, {})));
+ {{"Out", {"y"}}}, framework::AttributeMap{})));
net->AppendOp(std::unique_ptr(
new TestOp("test", {{"X", {"y"}}, {"W", {"w2"}}, {"b", {"b2"}}},
- {{"Out", {"z"}}}, {})));
+ {{"Out", {"z"}}}, framework::AttributeMap{})));
net->CompleteAddOp();
AssertSameVectorWithoutOrder({"x", "w1", "b1", "w2", "b2"},
@@ -58,7 +58,7 @@ TEST(NetOp, insert_op) {
NetOp net;
auto op1 = std::unique_ptr(
new framework::NOP("empty", {{"X", {"x"}}, {"W", {"w1"}}, {"b", {"b1"}}},
- {{"Out", {"y"}}}, {}));
+ {{"Out", {"y"}}}, framework::AttributeMap{}));
net.AppendOp(*op1);
net.InsertOp(0, *op1);
ASSERT_EQ(2UL, net.ops_.size());
@@ -68,10 +68,12 @@ TEST(NetOp, insert_op) {
TEST(NetOp, Clone) {
NetOp net;
- net.AppendOp(
- std::unique_ptr(new framework::NOP{"empty", {}, {}, {}}));
- net.AppendOp(std::unique_ptr(
- new framework::NOP{"empty2", {}, {}, {}}));
+ net.AppendOp(std::unique_ptr(new framework::NOP{
+ "empty", framework::VariableNameMap{}, framework::VariableNameMap{},
+ framework::AttributeMap{}}));
+ net.AppendOp(std::unique_ptr(new framework::NOP{
+ "empty2", framework::VariableNameMap{}, framework::VariableNameMap{},
+ framework::AttributeMap{}}));
net.CompleteAddOp(true);
auto new_net_op = net.Clone();
ASSERT_NE(new_net_op, nullptr);
diff --git a/paddle/operators/recurrent_op.cc b/paddle/operators/recurrent_op.cc
index 8b60b9c9122956e96441b67c752786bddd2d71fc..29f91636438449f90ea3ffee8adc21595aabe202 100644
--- a/paddle/operators/recurrent_op.cc
+++ b/paddle/operators/recurrent_op.cc
@@ -408,7 +408,8 @@ class RecurrentGradOp : public RecurrentBase {
attrs["value"] = 0.0f;
auto zero_op = framework::OpRegistry::CreateOp(
- "fill_constant", {}, {{"Out", {pg_names[param_id]}}}, attrs);
+ "fill_constant", framework::VariableNameMap{},
+ {{"Out", {pg_names[param_id]}}}, attrs);
zero_op->Run(scope, dev_ctx);
}
@@ -417,7 +418,7 @@ class RecurrentGradOp : public RecurrentBase {
auto sum_op = framework::OpRegistry::CreateOp(
"sum", {{"X", {pg_names[param_id], new_inside_name}}},
- {{"Out", {pg_names[param_id]}}}, {});
+ {{"Out", {pg_names[param_id]}}}, framework::AttributeMap{});
sum_op->Run(cur_scope, dev_ctx);
cur_scope.Rename(new_inside_name, inside_grad_name);
diff --git a/paddle/operators/recv_op.cc b/paddle/operators/recv_op.cc
index c69e416e10f2a9ced1f1b22c39235e4c9338e77c..eed482c1b458cd442ede523838b400d85c23a155 100644
--- a/paddle/operators/recv_op.cc
+++ b/paddle/operators/recv_op.cc
@@ -72,11 +72,13 @@ class RecvOp : public framework::OperatorBase {
// FIXME(typhoonzero): do not copy
framework::CopyFrom(t, dev_ctx.GetPlace(), dev_ctx, tensor);
- auto *block = Attr("OptimizeBlock");
- auto *program = block->Program();
+ std::string program_str = Attr("OptimizeProgram");
+ framework::ProgramDesc program_desc;
+ program_desc.ParseFromString(program_str);
+ framework::ProgramDescBind program(program_desc);
framework::Executor executor(dev_ctx);
// Run sub graph to get optimized tensor
- executor.Run(*program, &recv_scope, block->ID(),
+ executor.Run(program, &recv_scope, 0, /*global_block*/
false /*create_local_scope*/);
auto *out_var = recv_scope.FindVar("Out");
@@ -108,8 +110,8 @@ This operator will recv tensor from send_op
"IP address to listen on.")
.SetDefault("127.0.0.1:6164")
.AddCustomChecker([](const std::string &ip) { return !ip.empty(); });
- AddAttr("OptimizeBlock", "type BlockDescBind*",
- "optimize network run in server");
+ AddAttr("OptimizeProgram", "type string",
+ "Serialized ProgramDesc string for recv to run.");
}
};
diff --git a/paddle/operators/send_recv_op_test.cc b/paddle/operators/send_recv_op_test.cc
index ac03eb3752e7cd31dd80f4caa39dc0625f0409d5..3e2e2051afacb748877e3b0c3dec8d6662ac4e72 100644
--- a/paddle/operators/send_recv_op_test.cc
+++ b/paddle/operators/send_recv_op_test.cc
@@ -85,7 +85,10 @@ void StartServerNet() {
paddle::framework::AttributeMap attrs;
attrs.insert({"endpoint", std::string("127.0.0.1:6174")});
- attrs.insert({"OptimizeBlock", block});
+ std::string program_proto;
+ PADDLE_ENFORCE(program.Proto()->SerializeToString(&program_proto));
+
+ attrs.insert({"OptimizeProgram", program_proto});
recv_op = paddle::framework::OpRegistry::CreateOp("recv", {{"RX", {"RX"}}},
{{"Out", {"Out"}}}, attrs);
paddle::platform::CPUDeviceContext ctx(place);
diff --git a/paddle/operators/while_op.cc b/paddle/operators/while_op.cc
index 9b3f21cf94db338dee1173a76393dc66c3cbddaf..b8e44bcc5a99380fdf08cc2819b20045695eaf87 100644
--- a/paddle/operators/while_op.cc
+++ b/paddle/operators/while_op.cc
@@ -187,7 +187,8 @@ class WhileGradOp : public framework::OperatorBase {
attrs["value"] = 0.0f;
auto zero_op = framework::OpRegistry::CreateOp(
- "fill_constant", {}, {{"Out", {pg_names[param_id]}}}, attrs);
+ "fill_constant", framework::VariableNameMap{},
+ {{"Out", {pg_names[param_id]}}}, attrs);
zero_op->Run(scope, dev_ctx);
}
}
@@ -195,7 +196,7 @@ class WhileGradOp : public framework::OperatorBase {
auto new_inside_name = cur_scope.Rename(inside_grad_name);
auto sum_op = framework::OpRegistry::CreateOp(
"sum", {{"X", {pg_names[param_id], new_inside_name}}},
- {{"Out", {pg_names[param_id]}}}, {});
+ {{"Out", {pg_names[param_id]}}}, framework::AttributeMap{});
sum_op->Run(cur_scope, dev_ctx);
cur_scope.Rename(new_inside_name, inside_grad_name);
}
diff --git a/paddle/parameter/FirstOrderOptimizer.h b/paddle/parameter/FirstOrderOptimizer.h
index f157188a4f736319ea187052b90a17f8be9e9edb..5b0c52a30dfbc34f0db4a03366e31da54b86c9fc 100644
--- a/paddle/parameter/FirstOrderOptimizer.h
+++ b/paddle/parameter/FirstOrderOptimizer.h
@@ -38,7 +38,7 @@ public:
real torch_learningRate = optConfig_.learning_method() == "torch_momentum"
? 1.0 - paraConfig.momentum()
: 1.0;
-#ifdef PADDLE_USE_MKLDNN
+#ifdef PADDLE_WITH_MKLDNN
sgdUpdate(learningRate_ * paraConfig.learning_rate() *
(firstTime_ ? 1.0 : torch_learningRate),
paraConfig.momentum(),
diff --git a/paddle/parameter/ParameterUpdateFunctions.cpp b/paddle/parameter/ParameterUpdateFunctions.cpp
index 1898598e49652a2829e57329bab6017304cec662..d60cb363830ff26a1f5054fb4cebf37afdfd1d03 100644
--- a/paddle/parameter/ParameterUpdateFunctions.cpp
+++ b/paddle/parameter/ParameterUpdateFunctions.cpp
@@ -30,7 +30,7 @@ void sgdUpdateCpu(real learningRate,
const real* grad,
real* momentumVec) {
decayRate *= learningRate;
-#ifdef PADDLE_USE_MKLML
+#ifdef PADDLE_WITH_MKLML
#pragma omp parallel for
#endif
for (size_t i = 0; i < size; ++i) {
diff --git a/paddle/utils/Flags.cpp b/paddle/utils/Flags.cpp
index 8f100f02e90bcbc7fdcf6f053aec6f95cfb09c1a..9a7dc0e35622383a190f8b3a80736e6b42c9c959 100644
--- a/paddle/utils/Flags.cpp
+++ b/paddle/utils/Flags.cpp
@@ -20,7 +20,7 @@ DEFINE_bool(use_gpu, false, "Only support CPU training");
DEFINE_bool(use_gpu, true, "Whether to use GPU for training");
#endif
-#ifdef PADDLE_USE_MKLDNN
+#ifdef PADDLE_WITH_MKLDNN
// TODO(TJ): change to true when MKLDNN layers support multi-inputs
DEFINE_bool(use_mkldnn, false, "Default still keep use CPU training");
#else
diff --git a/python/paddle/trainer_config_helpers/layers.py b/python/paddle/trainer_config_helpers/layers.py
index 2b6c92460dadf9a03236763d9d7fb3291a0813fc..d0b14cf63c31713691b21742a45a3eb12af70ceb 100644
--- a/python/paddle/trainer_config_helpers/layers.py
+++ b/python/paddle/trainer_config_helpers/layers.py
@@ -1519,34 +1519,33 @@ def lstmemory(input,
NOTE: This is a low level user interface. You can use network.simple_lstm
to config a simple plain lstm layer.
- Please refer to **Generating Sequences With Recurrent Neural Networks** for
- more details about LSTM.
-
- Link_ goes as below.
-
- .. _Link: http://arxiv.org/abs/1308.0850
+ Reference:
+ `Generating Sequences With Recurrent Neural Networks
+ `_
- :param name: The lstmemory layer name.
+ :param name: The name of this layer. It is optional.
:type name: basestring
- :param size: DEPRECATED. size of the lstm cell
+ :param size: DEPRECATED. The dimension of the lstm cell.
:type size: int
:param input: The input of this layer.
:type input: LayerOutput
- :param reverse: is sequence process reversed or not.
+ :param reverse: Whether the input sequence is processed in a reverse order.
:type reverse: bool
:param act: Activation type. TanhActivation is the default activation.
:type act: BaseActivation
- :param gate_act: gate activation type, SigmoidActivation by default.
+ :param gate_act: Activation type of this layer's gates. SigmoidActivation is the
+ default activation.
:type gate_act: BaseActivation
- :param state_act: state activation type, TanhActivation by default.
+ :param state_act: Activation type of the state. TanhActivation is the default activation.
:type state_act: BaseActivation
:param bias_attr: The bias attribute. If the parameter is set to False or an object
whose type is not ParameterAttribute, no bias is defined. If the
parameter is set to True, the bias is initialized to zero.
:type bias_attr: ParameterAttribute | None | bool | Any
- :param param_attr: Parameter Attribute.
- :type param_attr: ParameterAttribute | None | False
- :param layer_attr: Extra Layer attribute
+ :param param_attr: The parameter attribute. See ParameterAttribute for details.
+ :type param_attr: ParameterAttribute
+ :param layer_attr: The extra layer attribute. See ExtraLayerAttribute for
+ details.
:type layer_attr: ExtraLayerAttribute | None
:return: LayerOutput object.
:rtype: LayerOutput
@@ -1635,14 +1634,14 @@ def grumemory(input,
h_t = (1 - z_t) h_{t-1} + z_t {\\tilde{h_t}}
NOTE: In PaddlePaddle's implementation, the multiplication operations
- :math:`W_{r}x_{t}`, :math:`W_{z}x_{t}` and :math:`W x_t` are not computed in
- gate_recurrent layer. Consequently, an additional mixed_layer with
+ :math:`W_{r}x_{t}`, :math:`W_{z}x_{t}` and :math:`W x_t` are not performed
+ in gate_recurrent layer. Consequently, an additional mixed_layer with
full_matrix_projection or a fc_layer must be included before grumemory
is called.
- More details can be found by referring to `Empirical Evaluation of Gated
- Recurrent Neural Networks on Sequence Modeling.
- `_
+ Reference:
+ `Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
+ `_
The simple usage is:
@@ -1650,28 +1649,29 @@ def grumemory(input,
gru = grumemory(input)
- :param name: The gru layer name.
- :type name: None | basestring
+ :param name: The name of this layer. It is optional.
+ :type name: basestring
:param input: The input of this layer.
:type input: LayerOutput.
- :param size: DEPRECATED. size of the gru cell
+ :param size: DEPRECATED. The dimension of the gru cell.
:type size: int
- :param reverse: Whether sequence process is reversed or not.
+ :param reverse: Whether the input sequence is processed in a reverse order.
:type reverse: bool
:param act: Activation type, TanhActivation is the default. This activation
affects the :math:`{\\tilde{h_t}}`.
:type act: BaseActivation
- :param gate_act: gate activation type, SigmoidActivation by default.
- This activation affects the :math:`z_t` and :math:`r_t`. It is the
- :math:`\\sigma` in the above formula.
+ :param gate_act: Activation type of this layer's two gates. SigmoidActivation is
+ the default activation. This activation affects the :math:`z_t`
+ and :math:`r_t`. It is the :math:`\\sigma` in the above formula.
:type gate_act: BaseActivation
:param bias_attr: The bias attribute. If the parameter is set to False or an object
whose type is not ParameterAttribute, no bias is defined. If the
parameter is set to True, the bias is initialized to zero.
:type bias_attr: ParameterAttribute | None | bool | Any
- :param param_attr: Parameter Attribute.
- :type param_attr: ParameterAttribute | None | False
- :param layer_attr: Extra Layer attribute
+ :param param_attr: The parameter attribute. See ParameterAttribute for details.
+ :type param_attr: ParameterAttribute
+ :param layer_attr: The extra layer attribute. See ExtraLayerAttribute for
+ details.
:type layer_attr: ExtraLayerAttribute | None
:return: LayerOutput object.
:rtype: LayerOutput
@@ -1715,10 +1715,10 @@ def last_seq(input,
"""
Get Last Timestamp Activation of a sequence.
- If stride > 0, this layer slides a window whose size is determined by stride,
- and return the last value of the window as the output. Thus, a long sequence
- will be shorten. Note that for sequence with sub-sequence, the default value
- of stride is -1.
+ If stride > 0, this layer will slide a window whose size is determined by stride,
+ and return the last value of the sequence in the window as the output. Thus, a
+ long sequence will be shortened. Note that for sequence with sub-sequence, the
+ default value of stride is -1.
The simple usage is:
@@ -1727,14 +1727,16 @@ def last_seq(input,
seq = last_seq(input=layer)
:param agg_level: Aggregated level
+ :type agg_level: AggregateLevel
:param name: The name of this layer. It is optional.
:type name: basestring
:param input: The input of this layer.
:type input: LayerOutput
:param stride: The step size between successive pooling regions.
- :type stride: Int
- :param layer_attr: extra layer attributes.
- :type layer_attr: ExtraLayerAttribute.
+ :type stride: int
+ :param layer_attr: The extra layer attribute. See ExtraLayerAttribute for
+ details.
+ :type layer_attr: ExtraLayerAttribute
:return: LayerOutput object.
:rtype: LayerOutput
"""
@@ -1771,10 +1773,10 @@ def first_seq(input,
"""
Get First Timestamp Activation of a sequence.
- If stride > 0, this layer slides a window whose size is determined by stride,
- and return the first value of the window as the output. Thus, a long sequence
- will be shorten. Note that for sequence with sub-sequence, the default value
- of stride is -1.
+ If stride > 0, this layer will slide a window whose size is determined by stride,
+ and return the first value of the sequence in the window as the output. Thus, a
+ long sequence will be shortened. Note that for sequence with sub-sequence, the
+ default value of stride is -1.
The simple usage is:
@@ -1783,13 +1785,15 @@ def first_seq(input,
seq = first_seq(input=layer)
:param agg_level: aggregation level
+ :type agg_level: AggregateLevel
:param name: The name of this layer. It is optional.
:type name: basestring
:param input: The input of this layer.
:type input: LayerOutput
:param stride: The step size between successive pooling regions.
- :type stride: Int
- :param layer_attr: extra layer attributes.
+ :type stride: int
+ :param layer_attr: The extra layer attribute. See ExtraLayerAttribute for
+ details.
:type layer_attr: ExtraLayerAttribute.
:return: LayerOutput object.
:rtype: LayerOutput
@@ -1847,8 +1851,8 @@ def expand_layer(input,
expand_level=ExpandLevel.FROM_NO_SEQUENCE,
layer_attr=None):
"""
- A layer for "Expand Dense data or (sequence data where the length of each
- sequence is one) to sequence data."
+ A layer for expanding dense data or (sequence data where the length of each
+ sequence is one) to sequence data.
The example usage is:
@@ -1860,7 +1864,9 @@ def expand_layer(input,
:param input: The input of this layer.
:type input: LayerOutput
- :param expand_as: Expand as this layer's sequence info.
+ :param expand_as: Expand the input according to this layer's sequence infomation. And
+ after the operation, the input expanded will have the same number of
+ elememts as this layer.
:type expand_as: LayerOutput
:param name: The name of this layer. It is optional.
:type name: basestring
@@ -1868,9 +1874,10 @@ def expand_layer(input,
whose type is not ParameterAttribute, no bias is defined. If the
parameter is set to True, the bias is initialized to zero.
:type bias_attr: ParameterAttribute | None | bool | Any
- :param expand_level: whether input layer is timestep(default) or sequence.
+ :param expand_level: Whether the input layer is a sequence or the element of a sequence.
:type expand_level: ExpandLevel
- :param layer_attr: extra layer attributes.
+ :param layer_attr: The extra layer attribute. See ExtraLayerAttribute for
+ details.
:type layer_attr: ExtraLayerAttribute.
:return: LayerOutput object.
:rtype: LayerOutput
@@ -3304,7 +3311,7 @@ def row_l2_norm_layer(input, name=None, layer_attr=None):
A layer for L2-normalization in each row.
.. math::
- out[i] = \frac{in[i]}{\sqrt{\sum_{k=1}^N in[k]^{2}}}
+ out[i] = \\frac{in[i]} {\\sqrt{\\sum_{k=1}^N in[k]^{2}}}
where the size of :math:`in` is (batchSize x dataDim) ,
and the size of :math:`out` is a (batchSize x dataDim) .
@@ -6173,9 +6180,11 @@ def huber_regression_cost(input,
Given a prediction f(x), a label y and :math:`\delta`, the loss function
is defined as:
- .. math:
- loss = 0.5*\left ( y-f(x) \right )^2, \left | y-f(x) \right |\leq \delta
- loss = \delta \left | y-f(x) \right |-0.5\delta ^2, otherwise
+ .. math::
+
+ loss = 0.5*(y-f(x))^{2}, | y-f(x) | < \delta
+
+ loss = \delta | y-f(x) | - 0.5 \delta ^2, otherwise
The example usage is:
@@ -6222,12 +6231,14 @@ def huber_classification_cost(input,
"""
For classification purposes, a variant of the Huber loss called modified Huber
is sometimes used. Given a prediction f(x) (a real-valued classifier score) and
- a true binary class label :math:`y\in \left \{-1, 1 \right \}`, the modified Huber
+ a true binary class label :math:`y\in \{-1, 1 \}`, the modified Huber
loss is defined as:
.. math:
- loss = \max \left ( 0, 1-yf(x) \right )^2, yf(x)\geq 1
- loss = -4yf(x), \text{otherwise}
+
+ loss = \max ( 0, 1-yf(x) )^2, yf(x) \geq -1
+
+ loss = -4yf(x), otherwise
The example usage is:
@@ -6972,7 +6983,7 @@ def clip_layer(input, min, max, name=None):
.. math::
- out[i] = \min\left(\max\left(in[i],p_{1}\right),p_{2}\right)
+ out[i] = \min (\max (in[i],p_{1} ),p_{2} )
.. code-block:: python
diff --git a/python/paddle/v2/fluid/layers.py b/python/paddle/v2/fluid/layers.py
index b4426bad1499419a6b512aa32abfed4fc21ef4c5..fd8a2ed18c9690ab55f62aea6c7b9dd7a92e68d5 100644
--- a/python/paddle/v2/fluid/layers.py
+++ b/python/paddle/v2/fluid/layers.py
@@ -762,7 +762,7 @@ def sequence_conv(input,
helper = LayerHelper('sequence_conv', **locals())
dtype = helper.input_dtype()
filter_shape = [filter_size * input.shape[1], num_filters]
- filter = helper.create_parameter(
+ filter_param = helper.create_parameter(
attr=helper.param_attr, shape=filter_shape, dtype=dtype)
pre_bias = helper.create_tmp_variable(dtype)
@@ -770,7 +770,7 @@ def sequence_conv(input,
type='sequence_conv',
inputs={
'X': [input],
- 'Filter': [filter],
+ 'Filter': [filter_param],
},
outputs={"Out": pre_bias},
attrs={
@@ -785,7 +785,7 @@ def sequence_conv(input,
def conv2d(input,
num_filters,
filter_size,
- stride=[1, 1],
+ stride=None,
padding=None,
groups=None,
param_attr=None,
@@ -802,6 +802,8 @@ def conv2d(input,
conv-2d output, if mentioned in the input parameters.
"""
+ if stride is None:
+ stride = [1, 1]
helper = LayerHelper('conv2d', **locals())
dtype = helper.input_dtype()
@@ -827,7 +829,7 @@ def conv2d(input,
std = (2.0 / (filter_size[0]**2 * num_channels))**0.5
return Normal(0.0, std, 0)
- filter = helper.create_parameter(
+ filter_param = helper.create_parameter(
attr=helper.param_attr,
shape=filter_shape,
dtype=dtype,
@@ -839,7 +841,7 @@ def conv2d(input,
type='conv2d_cudnn',
inputs={
'Input': input,
- 'Filter': filter,
+ 'Filter': filter_param,
},
outputs={"Output": pre_bias},
attrs={'strides': stride,
@@ -875,8 +877,8 @@ def sequence_pool(input, pool_type, **kwargs):
def pool2d(input,
pool_size,
pool_type,
- pool_stride=[1, 1],
- pool_padding=[0, 0],
+ pool_stride=None,
+ pool_padding=None,
global_pooling=False,
main_program=None,
startup_program=None):
@@ -884,6 +886,10 @@ def pool2d(input,
This function adds the operator for pooling in 2 dimensions, using the
pooling configurations mentioned in input parameters.
"""
+ if pool_padding is None:
+ pool_padding = [0, 0]
+ if pool_stride is None:
+ pool_stride = [1, 1]
if pool_type not in ["max", "avg"]:
raise ValueError(
"Unknown pool_type: '%s'. It can only be 'max' or 'avg'.",