提交 87ba6cbf 编写于 作者: Q qijun

merge baidu/develop

# Design Doc: Block and Scope
## The Representation of Computation
Both deep learning systems and programming languages help users describe computation procedures. These systems use various representations of computation:
- Caffe, Torch, and Paddle: sequences of layers.
- TensorFlow, Caffe2, Mxnet: graphs of operators.
- PaddlePaddle: nested blocks, like C++ and Java programs.
## Block in Programming Languages and Deep Learning
In programming languages, a block is a pair of curly braces that includes local variables definitions and a sequence of instructions, or operators.
Blocks work with control flow structures like `if`, `else`, and `for`, which have equivalents in deep learning:
| programming languages | PaddlePaddle |
|-----------------------|-----------------------|
| for, while loop | RNN, WhileOp |
| if, if-else, switch | IfElseOp, SwitchOp |
| sequential execution | a sequence of layers |
A key difference is that a C++ program describes a one pass computation, whereas a deep learning program describes both the forward and backward passes.
## Stack Frames and the Scope Hierarchy
The existence of the backward makes the execution of a block of traditional programs and PaddlePaddle different to each other:
| programming languages | PaddlePaddle |
|-----------------------|-------------------------------|
| stack | scope hierarchy |
| stack frame | scope |
| push at entering block| push at entering block |
| pop at leaving block | destroy at minibatch completes|
1. In traditional programs:
- When the execution enters the left curly brace of a block, the runtime pushes a frame into the stack, where it realizes local variables.
- After the execution leaves the right curly brace, the runtime pops the frame.
- The maximum number of frames in the stack is the maximum depth of nested blocks.
1. In PaddlePaddle
- When the execution enters a block, PaddlePaddle adds a new scope, where it realizes variables.
- PaddlePaddle doesn't pop a scope after the execution of the block because variables therein are to be used by the backward pass. So it has a stack forest known as a *scope hierarchy*.
- The height of the highest tree is the maximum depth of nested blocks.
- After the process of a minibatch, PaddlePaddle destroys the scope hierarchy.
## Use Blocks in C++ and PaddlePaddle Programs
Let us consolidate the discussion by presenting some examples.
### Blocks with `if-else` and `IfElseOp`
The following C++ programs shows how blocks are used with the `if-else` structure:
```c++
int x = 10;
int y = 20;
int out;
bool cond = false;
if (cond) {
int z = x + y;
out = softmax(z);
} else {
int z = fc(x);
out = z;
}
```
An equivalent PaddlePaddle program from the design doc of the [IfElseOp operator](./if_else_op.md) is as follows:
```python
import paddle as pd
x = var(10)
y = var(20)
cond = var(false)
ie = pd.create_ifelseop(inputs=[x], output_num=1)
with ie.true_block():
x = ie.inputs(true, 0)
z = operator.add(x, y)
ie.set_output(true, 0, operator.softmax(z))
with ie.false_block():
x = ie.inputs(false, 0)
z = layer.fc(x)
ie.set_output(true, 0, operator.softmax(z))
out = b(cond)
```
In both examples, the left branch computes `softmax(x+y)` and the right branch computes `fc(x)`.
A difference is that variables in the C++ program contain scalar values, whereas those in the PaddlePaddle programs are mini-batches of instances. The `ie.input(true, 0)` invocation returns instances in the 0-th input, `x`, that corresponds to true values in `cond` as the local variable `x`, where `ie.input(false, 0)` returns instances corresponding to false values.
### Blocks with `for` and `RNNOp`
The following RNN model from the [RNN design doc](./rnn.md)
```python
x = sequence([10, 20, 30])
m = var(0)
W = tensor()
U = tensor()
rnn = create_rnn(inputs=[input])
with rnn.stepnet() as net:
x = net.set_inputs(0)
h = net.add_memory(init=m)
fc_out = pd.matmul(W, x)
hidden_out = pd.matmul(U, h.pre(n=1))
sum = pd.add_two(fc_out, hidden_out)
act = pd.sigmoid(sum)
h.update(act) # update memory with act
net.set_outputs(0, act, hidden_out) # two outputs
o1, o2 = rnn()
print o1, o2
```
has its equivalent C++ program as follows
```c++
int* x = {10, 20, 30};
int m = 0;
int W = some_value();
int U = some_other_value();
int mem[sizeof(x) / sizeof(x[0]) + 1];
int o1[sizeof(x) / sizeof(x[0]) + 1];
int o2[sizeof(x) / sizeof(x[0]) + 1];
for (int i = 1; i <= sizeof(x)/sizeof(x[0]); ++i) {
int x = x[i-1];
if (i == 1) mem[0] = m;
int fc_out = W * x;
int hidden_out = Y * mem[i-1];
int sum = fc_out + hidden_out;
int act = sigmoid(sum);
mem[i] = act;
o1[i] = act;
o2[i] = hidden_out;
}
print_array(o1);
print_array(o2);
```
## Compilation and Execution
Like TensorFlow programs, a PaddlePaddle program is written in Python. The first part describes a neural network as a protobuf message, and the rest part executes the message for training or inference.
The generation of this protobuf message is like what a compiler generates a binary executable file. The execution of the message that the OS executes the binary file.
## The "Binary Executable File Format"
The definition of the protobuf message is as follows:
```protobuf
message BlockDesc {
repeated VarDesc vars = 1;
repeated OpDesc ops = 2;
}
```
The step net in above RNN example would look like
```
BlockDesc {
vars = {
VarDesc {...} // x
VarDesc {...} // h
VarDesc {...} // fc_out
VarDesc {...} // hidden_out
VarDesc {...} // sum
VarDesc {...} // act
}
ops = {
OpDesc {...} // matmul
OpDesc {...} // add_two
OpDesc {...} // sigmoid
}
};
```
Also, the RNN operator in above example is serialized into a protobuf message of type `OpDesc` and would look like:
```
OpDesc {
inputs = {0} // the index of x
outputs = {5, 3} // indices of act and hidden_out
attrs {
"memories" : {1} // the index of h
"step_net" : <above step net>
}
};
```
This `OpDesc` value is in the `ops` field of the `BlockDesc` value representing the global block.
## The Compilation of Blocks
During the generation of the Protobuf message, the Block should store VarDesc (the Protobuf message which describes Variable) and OpDesc (the Protobuf message which describes Operator).
VarDesc in a block should have its name scope to avoid local variables affect parent block's name scope.
Child block's name scopes should inherit the parent's so that OpDesc in child block can reference a VarDesc that stored in parent block. For example
```python
a = pd.Varaible(shape=[20, 20])
b = pd.fc(a, params=["fc.w", "fc.b"])
rnn = pd.create_rnn()
with rnn.stepnet() as net:
x = net.set_inputs(a)
# reuse fc's parameter
fc_without_b = pd.get_variable("fc.w")
net.set_outputs(fc_without_b)
out = rnn()
```
the method `pd.get_variable` can help retrieve a Variable by a name, a Variable may store in a parent block, but might be retrieved in a child block, so block should have a variable scope that supports inheritance.
In compiler design, the symbol table is a data structure created and maintained by compilers to store information about the occurrence of various entities such as variable names, function names, classes, etc.
To store the definition of variables and operators, we define a C++ class `SymbolTable`, like the one used in compilers.
`SymbolTable` can do the following stuff:
- store the definitions (some names and attributes) of variables and operators,
- to verify if a variable was declared,
- to make it possible to implement type checking (offer Protobuf message pointers to `InferShape` handlers).
```c++
// Information in SymbolTable is enough to trace the dependency graph. So maybe
// the Eval() interface takes a SymbolTable is enough.
class SymbolTable {
public:
SymbolTable(SymbolTable* parent) : parent_(parent) {}
OpDesc* NewOp(const string& name="");
// TODO determine whether name is generated by python or C++
// currently assume that a unique name will be generated by C++ if the
// argument name left default.
VarDesc* NewVar(const string& name="");
// find a VarDesc by name, if recursive true, find parent's SymbolTable
// recursively.
// this interface is introduced to support InferShape, find protobuf messages
// of variables and operators, pass pointers into InferShape.
// operator
//
// NOTE maybe some C++ classes such as VarDescBuilder and OpDescBuilder should
// be proposed and embedded into pybind to enable python operate on C++ pointers.
VarDesc* FindVar(const string& name, bool recursive=true);
OpDesc* FindOp(const string& name);
BlockDesc Compile() const;
private:
SymbolTable* parent_;
map<string, OpDesc> ops_;
map<string, VarDesc> vars_;
};
```
After all the description of variables and operators is added into SymbolTable,
the block has enough information to run.
The `Block` class takes a `BlockDesc` as input, and provide `Run` and `InferShape` functions.
```c++
namespace {
class Block : OperatorBase {
public:
Block(const BlockDesc& desc) desc_(desc) {}
void InferShape(const framework::Scope& scope) const override {
if (!symbols_ready_) {
CreateVariables(scope);
CreateOperators();
}
// should run InferShape first.
for (auto& op : runtime_table_.ops()) {
op->InferShape(scope);
}
}
void Run(const framework::Scope& scope,
const platform::DeviceContext& dev_ctx) const override {
PADDLE_ENFORCE(symbols_ready_, "operators and variables should be created first.");
for (auto& op : runtime_table_.ops()) {
op->Run(scope, dev_ctx);
}
}
void CreateVariables(const framework::Scope& scope);
void CreateOperators();
// some other necessary interfaces of NetOp are list below
// ...
private:
BlockDesc desc_;
bool symbols_ready_{false};
};
```
## The Execution of Blocks
Block inherits from OperatorBase, which has a Run method.
Block's Run method will run its operators sequentially.
There is another important interface called `Eval`, which take some arguments called targets, and generate a minimal graph which takes targets as the end points and creates a new Block,
after `Run`, `Eval` will get the latest value and return the targets.
The definition of Eval is as follows:
```c++
// clean a block description by targets using the corresponding dependency graph.
// return a new BlockDesc with minimal number of operators.
// NOTE not return a Block but the block's description so that this can be distributed
// to a cluster.
BlockDesc Prune(const BlockDesc& desc, vector<string> targets);
void Block::Eval(const vector<string>& targets,
const framework::Scope& scope,
const platform::DeviceContext& dev_ctx) {
BlockDesc min_desc = Prune(desc_, targets);
Block min_block(min_desc);
min_block.Run(scope, dev_ctx);
}
```
/* Copyright (c) 2016 PaddlePaddle Authors. All Rights Reserve.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License. */
#include "ExpandConvBaseLayer.h"
#include "paddle/utils/Logging.h"
namespace paddle {
bool ExpandConvBaseLayer::init(const LayerMap &layerMap,
const ParameterMap &parameterMap) {
/* Initialize the basic convolutional parent class */
ConvBaseLayer::init(layerMap, parameterMap);
int index = 0;
for (auto &inputConfig : config_.inputs()) {
const ConvConfig &conf = inputConfig.conv_conf();
/* Consistent caffe mode for multiple input */
caffeMode_ = conf.caffe_mode();
// create a new weight
size_t height, width;
height = filterPixels_[index] * filterChannels_[index];
width = (!isDeconv_) ? numFilters_ : channels_[index];
CHECK_EQ(parameters_[index]->getSize(), width * height);
Weight *w = new Weight(height, width, parameters_[index]);
weights_.emplace_back(w);
index++;
}
if (biasParameter_.get()) {
if (sharedBiases_) {
CHECK_EQ((size_t)numFilters_, biasParameter_->getSize());
biases_ =
std::unique_ptr<Weight>(new Weight(numFilters_, 1, biasParameter_));
} else {
biases_ =
std::unique_ptr<Weight>(new Weight(getSize(), 1, biasParameter_));
}
}
getOutputSize();
return true;
}
size_t ExpandConvBaseLayer::getOutputSize() {
CHECK_NE(inputLayers_.size(), 0UL);
size_t layerSize = ConvBaseLayer::calOutputSize();
return layerSize;
}
void ExpandConvBaseLayer::addSharedBias() {
size_t mapW = getOutputSize() / numFilters_;
size_t mapH = getOutputValue()->getElementCnt() / mapW;
MatrixPtr out =
Matrix::create(getOutputValue()->getData(), mapH, mapW, false, useGpu_);
Matrix::resizeOrCreate(transOutValue_, mapW, mapH, false, useGpu_);
out->transpose(transOutValue_, false); // false means no memory allocation
transOutValue_->reshape(transOutValue_->getElementCnt() / numFilters_,
numFilters_);
MatrixPtr bias = Matrix::create(biases_->getW()->getData(),
1,
biases_->getW()->getElementCnt(),
false,
useGpu_);
transOutValue_->addBias(*bias, 1.0f);
transOutValue_->reshape(mapW, mapH);
transOutValue_->transpose(out, false); // false means no memory allocation
out->clear();
bias->clear();
}
void ExpandConvBaseLayer::addUnsharedBias() {
MatrixPtr outValue = getOutputValue();
MatrixPtr bias = Matrix::create(biases_->getW()->getData(),
1,
biases_->getW()->getElementCnt(),
false,
useGpu_);
outValue->addBias(*bias, 1.0f);
}
void ExpandConvBaseLayer::bpropSharedBias(MatrixPtr biases, MatrixPtr v) {
size_t mapW = getOutputSize() / numFilters_;
size_t mapH = v->getElementCnt() / mapW;
MatrixPtr vTmp = Matrix::create(v->getData(), mapH, mapW, false, useGpu_);
Matrix::resizeOrCreate(transOutValue_, mapW, mapH, false, useGpu_);
vTmp->transpose(transOutValue_, false); // false means no memory allocation
transOutValue_->reshape(transOutValue_->getElementCnt() / numFilters_,
numFilters_);
biases->collectBias(*transOutValue_, 1.0f);
}
void ExpandConvBaseLayer::bpropBiases(MatrixPtr v) {
MatrixPtr biases = Matrix::create(biases_->getWGrad()->getData(),
1,
biases_->getWGrad()->getElementCnt(),
false,
useGpu_);
if (sharedBiases_) {
bpropSharedBias(biases, v);
} else {
biases->collectBias(*v, 1.0f);
}
biases->clear();
}
} // namespace paddle
/* Copyright (c) 2016 PaddlePaddle Authors. All Rights Reserve.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License. */
#pragma once
#include <vector>
#include "ConvBaseLayer.h"
#include "paddle/math/Matrix.h"
namespace paddle {
/**
* @brief A subclass of ConvBaseLayer that is a superclass of both
* ExpandConvLayer and ExpandConvTransLayer
*/
class ExpandConvBaseLayer : public ConvBaseLayer {
protected:
/// The transpose of output, which is an auxiliary matrix.
MatrixPtr transOutValue_;
public:
explicit ExpandConvBaseLayer(const LayerConfig& config)
: ConvBaseLayer(config) {}
~ExpandConvBaseLayer() {}
bool init(const LayerMap& layerMap,
const ParameterMap& parameterMap) override;
size_t getOutputSize();
/**
* Add shared bias.
*/
void addSharedBias();
/**
* Add unshared bias.
*/
void addUnsharedBias();
void bpropSharedBias(MatrixPtr biases, MatrixPtr v);
void bpropBiases(MatrixPtr v);
};
} // namespace paddle
...@@ -36,7 +36,36 @@ inline bool isDepthwiseConv(int channels, int groups) { ...@@ -36,7 +36,36 @@ inline bool isDepthwiseConv(int channels, int groups) {
bool ExpandConvLayer::init(const LayerMap &layerMap, bool ExpandConvLayer::init(const LayerMap &layerMap,
const ParameterMap &parameterMap) { const ParameterMap &parameterMap) {
/* Initialize the basic convolutional parent class */ /* Initialize the basic convolutional parent class */
ExpandConvBaseLayer::init(layerMap, parameterMap); ConvBaseLayer::init(layerMap, parameterMap);
int index = 0;
for (auto &inputConfig : config_.inputs()) {
const ConvConfig &conf = inputConfig.conv_conf();
/* Consistent caffe mode for multiple input */
caffeMode_ = conf.caffe_mode();
// create a new weight
size_t height, width;
height = filterPixels_[index] * filterChannels_[index];
width = (!isDeconv_) ? numFilters_ : channels_[index];
CHECK_EQ(parameters_[index]->getSize(), width * height);
Weight *w = new Weight(height, width, parameters_[index]);
weights_.emplace_back(w);
index++;
}
if (biasParameter_.get()) {
if (sharedBiases_) {
CHECK_EQ((size_t)numFilters_, biasParameter_->getSize());
biases_ = std::unique_ptr<Weight>(
new Weight(1, numFilters_, biasParameter_, 0));
} else {
biases_ =
std::unique_ptr<Weight>(new Weight(1, getSize(), biasParameter_, 0));
}
}
getOutputSize();
size_t numInputs = config_.inputs_size(); size_t numInputs = config_.inputs_size();
inputShape_.resize(numInputs); inputShape_.resize(numInputs);
...@@ -108,6 +137,12 @@ bool ExpandConvLayer::init(const LayerMap &layerMap, ...@@ -108,6 +137,12 @@ bool ExpandConvLayer::init(const LayerMap &layerMap,
return true; return true;
} }
size_t ExpandConvLayer::getOutputSize() {
CHECK_NE(inputLayers_.size(), 0UL);
size_t layerSize = ConvBaseLayer::calOutputSize();
return layerSize;
}
// i is the index of input layers // i is the index of input layers
#define BACKWARD_INPUT(i, inputs, outputs) \ #define BACKWARD_INPUT(i, inputs, outputs) \
backward_[2 * i]->calc(inputs, outputs) backward_[2 * i]->calc(inputs, outputs)
...@@ -155,11 +190,7 @@ void ExpandConvLayer::forward(PassType passType) { ...@@ -155,11 +190,7 @@ void ExpandConvLayer::forward(PassType passType) {
/* add the bias-vector */ /* add the bias-vector */
if (biases_.get()) { if (biases_.get()) {
if (sharedBiases_) { output_.value->addBias(*biases_->getW(), 1.0, sharedBiases_);
addSharedBias();
} else {
addUnsharedBias();
}
} }
/* activation */ /* activation */
...@@ -171,7 +202,7 @@ void ExpandConvLayer::backward(const UpdateCallback &callback) { ...@@ -171,7 +202,7 @@ void ExpandConvLayer::backward(const UpdateCallback &callback) {
MatrixPtr outGrad = getOutputGrad(); MatrixPtr outGrad = getOutputGrad();
if (biases_ && biases_->getWGrad()) { if (biases_ && biases_->getWGrad()) {
bpropBiases(outGrad); biases_->getWGrad()->collectBias(*getOutputGrad(), 1, sharedBiases_);
/* Increasing the number of gradient */ /* Increasing the number of gradient */
biases_->getParameterPtr()->incUpdate(callback); biases_->getParameterPtr()->incUpdate(callback);
} }
......
...@@ -15,7 +15,7 @@ limitations under the License. */ ...@@ -15,7 +15,7 @@ limitations under the License. */
#pragma once #pragma once
#include <vector> #include <vector>
#include "ExpandConvBaseLayer.h" #include "ConvBaseLayer.h"
#include "paddle/math/Matrix.h" #include "paddle/math/Matrix.h"
namespace paddle { namespace paddle {
...@@ -28,10 +28,9 @@ namespace paddle { ...@@ -28,10 +28,9 @@ namespace paddle {
* The config file api is img_conv_layer. * The config file api is img_conv_layer.
*/ */
class ExpandConvLayer : public ExpandConvBaseLayer { class ExpandConvLayer : public ConvBaseLayer {
public: public:
explicit ExpandConvLayer(const LayerConfig& config) explicit ExpandConvLayer(const LayerConfig& config) : ConvBaseLayer(config) {}
: ExpandConvBaseLayer(config) {}
~ExpandConvLayer() {} ~ExpandConvLayer() {}
...@@ -41,6 +40,8 @@ public: ...@@ -41,6 +40,8 @@ public:
void forward(PassType passType) override; void forward(PassType passType) override;
void backward(const UpdateCallback& callback) override; void backward(const UpdateCallback& callback) override;
size_t getOutputSize();
protected: protected:
std::vector<TensorShape> inputShape_; std::vector<TensorShape> inputShape_;
std::vector<TensorShape> filterShape_; std::vector<TensorShape> filterShape_;
......
...@@ -285,10 +285,9 @@ void MKLDNNConvLayer::resetWgtBiasValue( ...@@ -285,10 +285,9 @@ void MKLDNNConvLayer::resetWgtBiasValue(
wgt = MKLDNNMatrix::create(weight_->getW(), pd->weights_primitive_desc()); wgt = MKLDNNMatrix::create(weight_->getW(), pd->weights_primitive_desc());
VLOG(MKLDNN_FMTS) << "Weight value format: " << wgt->getFormat(); VLOG(MKLDNN_FMTS) << "Weight value format: " << wgt->getFormat();
bias = nullptr; bias = (biases_ && biases_->getW())
if (biases_ && biases_->getW()) { ? MKLDNNMatrix::create(biases_->getW(), pd->bias_primitive_desc())
bias = MKLDNNMatrix::create(biases_->getW(), pd->bias_primitive_desc()); : nullptr;
}
} }
void MKLDNNConvLayer::resetOutValue( void MKLDNNConvLayer::resetOutValue(
...@@ -356,6 +355,7 @@ void MKLDNNConvLayer::resetBwdWgtPD( ...@@ -356,6 +355,7 @@ void MKLDNNConvLayer::resetBwdWgtPD(
void MKLDNNConvLayer::resetBwdDataPD( void MKLDNNConvLayer::resetBwdDataPD(
std::shared_ptr<conv_bwdData::primitive_desc>& pd) { std::shared_ptr<conv_bwdData::primitive_desc>& pd) {
pd = nullptr;
if (inputLayers_[0]->getOutput().grad == nullptr) { if (inputLayers_[0]->getOutput().grad == nullptr) {
return; return;
} }
...@@ -476,6 +476,7 @@ void MKLDNNConvLayer::resetWgtBiasGrad( ...@@ -476,6 +476,7 @@ void MKLDNNConvLayer::resetWgtBiasGrad(
<< "primitive desc of weight grad and value should be equal"; << "primitive desc of weight grad and value should be equal";
VLOG(MKLDNN_FMTS) << "weight grad format: " << wgt->getFormat(); VLOG(MKLDNN_FMTS) << "weight grad format: " << wgt->getFormat();
bias = nullptr;
if (biasVal_ == nullptr) { if (biasVal_ == nullptr) {
return; return;
} }
......
...@@ -17,9 +17,6 @@ limitations under the License. */ ...@@ -17,9 +17,6 @@ limitations under the License. */
using namespace mkldnn; // NOLINT using namespace mkldnn; // NOLINT
typedef memory::format format; typedef memory::format format;
typedef inner_product_forward fc_fwd;
typedef inner_product_backward_weights fc_bwdWgt;
typedef inner_product_backward_data fc_bwdData;
namespace paddle { namespace paddle {
...@@ -93,35 +90,88 @@ void MKLDNNFcLayer::reshape( ...@@ -93,35 +90,88 @@ void MKLDNNFcLayer::reshape(
printSizeInfo(); printSizeInfo();
} }
void MKLDNNFcLayer::resetFwd(std::vector<mkldnn::primitive>& pipeline, void MKLDNNFcLayer::resetFwd(std::vector<primitive>& pipeline,
MKLDNNMatrixPtr& in, MKLDNNMatrixPtr& in,
MKLDNNMatrixPtr& wgt, MKLDNNMatrixPtr& wgt,
MKLDNNMatrixPtr& bias, MKLDNNMatrixPtr& bias,
MKLDNNMatrixPtr& out) { MKLDNNMatrixPtr& out) {
pipeline.clear(); resetFwdBuffers(in, wgt, bias, out);
bool hasBias = biases_ && biases_->getW();
const MatrixPtr& wgtVal = weight_->getW(); resetFwdPD(fwdPD_, in, wgt, bias, out);
const MatrixPtr& biasVal = hasBias ? biases_->getW() : nullptr;
const MatrixPtr& outVal = output_.value; resetFwdPipeline(pipeline, fwdPD_, in, wgt, bias, out);
printValueFormatFlow();
}
void MKLDNNFcLayer::resetBwd(std::vector<primitive>& pipeline,
MKLDNNMatrixPtr& in,
MKLDNNMatrixPtr& wgt,
MKLDNNMatrixPtr& bias,
MKLDNNMatrixPtr& out) {
std::shared_ptr<fc_bwdWgt::primitive_desc> bwdWgtPD;
std::shared_ptr<fc_bwdData::primitive_desc> bwdDataPD;
resetBwdBuffers(in, wgt, bias, out);
resetBwdWgtPD(bwdWgtPD, wgt, bias, out);
resetBwdDataPD(bwdDataPD, in, out);
resetBwdPipeline(pipeline, bwdWgtPD, bwdDataPD, in, wgt, bias, out);
printGradFormatFlow();
}
void MKLDNNFcLayer::updateInputData() {
inVal_->setData(getInputValue(0, CPU_DEVICE)->getData());
}
void MKLDNNFcLayer::updateWeights(const UpdateCallback& callback) {
weight_->getParameterPtr()->incUpdate(callback);
if (biases_ && biases_->getWGrad()) {
biases_->getParameterPtr()->incUpdate(callback);
}
}
void MKLDNNFcLayer::resetFwdBuffers(MKLDNNMatrixPtr& in,
MKLDNNMatrixPtr& wgt,
MKLDNNMatrixPtr& bias,
MKLDNNMatrixPtr& out) {
resetInValue(in);
resetWgtBiasValue(wgt, bias);
resetOutValue(out);
}
void MKLDNNFcLayer::resetInValue(MKLDNNMatrixPtr& in) {
if (inputIsOnlyMKLDNN()) { if (inputIsOnlyMKLDNN()) {
const MatrixPtr& inVal = getInputValue(0); const MatrixPtr& dnnIn = getInputValue(0);
in = std::dynamic_pointer_cast<MKLDNNMatrix>(inVal); in = std::dynamic_pointer_cast<MKLDNNMatrix>(dnnIn);
CHECK(in) << "Input should be MKLDNNMatrix"; CHECK(in) << "Input should be MKLDNNMatrix";
} else { } else {
CHECK_EQ(getPrev(0)->getDeviceId(), CPU_DEVICE) << "Only support CPU yet"; CHECK_EQ(getPrev(0)->getDeviceId(), CPU_DEVICE) << "Only support CPU yet";
const MatrixPtr& inVal = getInputValue(0, CPU_DEVICE); const MatrixPtr& cpuIn = getInputValue(0, CPU_DEVICE);
in = MKLDNNMatrix::create( in = MKLDNNMatrix::create(
inVal, memory::dims{bs_, ic_, ih_, iw_}, format::nchw, engine_); cpuIn, {bs_, ic_, ih_, iw_}, format::nchw, engine_);
} }
in->downSpatial(); in->downSpatial();
}
void MKLDNNFcLayer::resetWgtBiasValue(MKLDNNMatrixPtr& wgt,
MKLDNNMatrixPtr& bias) {
wgt = MKLDNNMatrix::create( wgt = MKLDNNMatrix::create(
wgtVal, memory::dims{oc_, ic_, ih_, iw_}, format::oihw, engine_); weight_->getW(), {oc_, ic_, ih_, iw_}, format::oihw, engine_);
wgt->downSpatial(); wgt->downSpatial();
bias = hasBias ? MKLDNNMatrix::create(biasVal, {oc_}, format::x, engine_)
: nullptr;
out = MKLDNNMatrix::create(outVal, {bs_, oc_}, format::nc, engine_);
bias = (biases_ && biases_->getW())
? MKLDNNMatrix::create(biases_->getW(), {oc_}, format::x, engine_)
: nullptr;
}
void MKLDNNFcLayer::resetOutValue(MKLDNNMatrixPtr& out) {
out = MKLDNNMatrix::create(output_.value, {bs_, oc_}, format::nc, engine_);
// change original output value to mkldnn output value // change original output value to mkldnn output value
output_.value = std::dynamic_pointer_cast<Matrix>(out); output_.value = std::dynamic_pointer_cast<Matrix>(out);
if (!outputIsOnlyMKLDNN()) { if (!outputIsOnlyMKLDNN()) {
...@@ -129,46 +179,59 @@ void MKLDNNFcLayer::resetFwd(std::vector<mkldnn::primitive>& pipeline, ...@@ -129,46 +179,59 @@ void MKLDNNFcLayer::resetFwd(std::vector<mkldnn::primitive>& pipeline,
// just share point // just share point
getOutput(CPU_DEVICE).value->setData(output_.value->getData()); getOutput(CPU_DEVICE).value->setData(output_.value->getData());
} }
}
// create forward handle void MKLDNNFcLayer::resetFwdPD(std::shared_ptr<fc_fwd::primitive_desc>& pd,
MKLDNNMatrixPtr in,
MKLDNNMatrixPtr wgt,
MKLDNNMatrixPtr bias,
MKLDNNMatrixPtr out) {
CHECK(in);
CHECK(wgt);
CHECK(out);
prop_kind pk = prop_kind::forward; prop_kind pk = prop_kind::forward;
fc_fwd::desc fwdDesc = hasBias ? fc_fwd::desc(pk, fc_fwd::desc fwdDesc = bias != nullptr ? fc_fwd::desc(pk,
in->getMemoryDesc(), in->getMemoryDesc(),
wgt->getMemoryDesc(), wgt->getMemoryDesc(),
bias->getMemoryDesc(), bias->getMemoryDesc(),
out->getMemoryDesc()) out->getMemoryDesc())
: fc_fwd::desc(pk, : fc_fwd::desc(pk,
in->getMemoryDesc(), in->getMemoryDesc(),
wgt->getMemoryDesc(), wgt->getMemoryDesc(),
out->getMemoryDesc()); out->getMemoryDesc());
fc_fwd::primitive_desc fwdPD = fc_fwd::primitive_desc(fwdDesc, engine_); pd.reset(new fc_fwd::primitive_desc(fwdDesc, engine_));
if (hasBias) { }
fwd_.reset(new fc_fwd(fwdPD, *in, *wgt, *bias, *out));
void MKLDNNFcLayer::resetFwdPipeline(
std::vector<primitive>& pipeline,
std::shared_ptr<fc_fwd::primitive_desc>& pd,
MKLDNNMatrixPtr& in,
MKLDNNMatrixPtr& wgt,
MKLDNNMatrixPtr& bias,
MKLDNNMatrixPtr& out) {
pipeline.clear();
if (bias) {
fwd_.reset(new fc_fwd(*pd, *in, *wgt, *bias, *out));
} else { } else {
fwd_.reset(new fc_fwd(fwdPD, *in, *wgt, *out)); fwd_.reset(new fc_fwd(*pd, *in, *wgt, *out));
} }
printValueFormatFlow();
pipeline.push_back(*fwd_); pipeline.push_back(*fwd_);
} }
void MKLDNNFcLayer::resetBwd(std::vector<mkldnn::primitive>& pipeline, void MKLDNNFcLayer::resetBwdBuffers(MKLDNNMatrixPtr& in,
MKLDNNMatrixPtr& in, MKLDNNMatrixPtr& wgt,
MKLDNNMatrixPtr& wgt, MKLDNNMatrixPtr& bias,
MKLDNNMatrixPtr& bias, MKLDNNMatrixPtr& out) {
MKLDNNMatrixPtr& out) { resetOutGrad(out);
pipeline.clear();
if (!needResetBwd_) { resetWgtBiasGrad(wgt, bias);
return;
}
needResetBwd_ = false;
bool hasBias = biases_ && biases_->getWGrad();
/// backward weight resetInGrad(in);
CHECK(inVal_) << "Should have input value"; }
const MatrixPtr& wgtGrad = weight_->getWGrad();
const MatrixPtr& biasGrad = hasBias ? biases_->getWGrad() : nullptr;
void MKLDNNFcLayer::resetOutGrad(MKLDNNMatrixPtr& out) {
// TODO(TJ): merge outgrad // TODO(TJ): merge outgrad
int device = outputIsOnlyMKLDNN() ? MKLDNN_DEVICE : CPU_DEVICE; int device = outputIsOnlyMKLDNN() ? MKLDNN_DEVICE : CPU_DEVICE;
// for MKLDNN device: // for MKLDNN device:
...@@ -178,66 +241,88 @@ void MKLDNNFcLayer::resetBwd(std::vector<mkldnn::primitive>& pipeline, ...@@ -178,66 +241,88 @@ void MKLDNNFcLayer::resetBwd(std::vector<mkldnn::primitive>& pipeline,
// for CPU device: // for CPU device:
// fc do not need to convert from cpu device since output is always nc format // fc do not need to convert from cpu device since output is always nc format
// only need create from cpu device // only need create from cpu device
const MatrixPtr& outGrad = getOutput(device).grad; CHECK(outVal_);
out = MKLDNNMatrix::create(outGrad, outVal_->getPrimitiveDesc()); out =
wgt = MKLDNNMatrix::create(wgtGrad, wgtVal_->getPrimitiveDesc()); MKLDNNMatrix::create(getOutput(device).grad, outVal_->getPrimitiveDesc());
bias = hasBias ? MKLDNNMatrix::create(biasGrad, biasVal_->getPrimitiveDesc()) }
: nullptr;
void MKLDNNFcLayer::resetWgtBiasGrad(MKLDNNMatrixPtr& wgt,
// create memory primitive desc MKLDNNMatrixPtr& bias) {
fc_fwd::desc fwdDesc = fc_fwd::desc(prop_kind::forward, CHECK(wgtVal_);
inVal_->getMemoryDesc(), wgt = MKLDNNMatrix::create(weight_->getWGrad(), wgtVal_->getPrimitiveDesc());
wgt->getMemoryDesc(),
out->getMemoryDesc()); bias = nullptr;
fc_fwd::primitive_desc fwdPD = fc_fwd::primitive_desc(fwdDesc, engine_); if (biasVal_ == nullptr) {
fc_bwdWgt::desc bwdWgtDesc = hasBias return;
? fc_bwdWgt::desc(inVal_->getMemoryDesc(),
wgt->getMemoryDesc(),
bias->getMemoryDesc(),
out->getMemoryDesc())
: fc_bwdWgt::desc(inVal_->getMemoryDesc(),
wgt->getMemoryDesc(),
out->getMemoryDesc());
fc_bwdWgt::primitive_desc bwdWgtPD =
fc_bwdWgt::primitive_desc(bwdWgtDesc, engine_, fwdPD);
if (hasBias) {
bwdWgt_.reset(new fc_bwdWgt(bwdWgtPD, *inVal_, *out, *wgt, *bias));
} else {
bwdWgt_.reset(new fc_bwdWgt(bwdWgtPD, *inVal_, *out, *wgt));
} }
pipeline.push_back(*bwdWgt_); bias =
MKLDNNMatrix::create(biases_->getWGrad(), biasVal_->getPrimitiveDesc());
}
/// backward data void MKLDNNFcLayer::resetInGrad(MKLDNNMatrixPtr& in) {
in = nullptr;
const MatrixPtr& inGrad = inputLayers_[0]->getOutput().grad; const MatrixPtr& inGrad = inputLayers_[0]->getOutput().grad;
if (inGrad == nullptr) { if (inGrad == nullptr) {
return; return;
} }
if (getInput(0, MKLDNN_DEVICE).getAllCount() > 1) { // TODO(TJ): use outputMaps_ ways to get the inGrad_ when merge outgrad done
// TODO(TJ): use outputMaps_ ways to get the inGrad_ when merge outgrad done CHECK(inVal_);
} else { in = MKLDNNMatrix::create(inGrad, inVal_->getPrimitiveDesc());
in = MKLDNNMatrix::create(inGrad, inVal_->getPrimitiveDesc()); }
}
fc_bwdData::desc bwdDataDesc = fc_bwdData::desc(
inVal_->getMemoryDesc(), wgt->getMemoryDesc(), out->getMemoryDesc());
fc_bwdData::primitive_desc bwdDataPD =
fc_bwdData::primitive_desc(bwdDataDesc, engine_, fwdPD);
CHECK(wgtVal_) << "Should have weight memory"; void MKLDNNFcLayer::resetBwdWgtPD(
bwdData_.reset(new fc_bwdData(bwdDataPD, *out, *wgtVal_, *in)); std::shared_ptr<fc_bwdWgt::primitive_desc>& pd,
printGradFormatFlow(); MKLDNNMatrixPtr& wgt,
pipeline.push_back(*bwdData_); MKLDNNMatrixPtr& bias,
MKLDNNMatrixPtr& out) {
CHECK(inVal_);
fc_bwdWgt::desc bwdWgtDesc = bias ? fc_bwdWgt::desc(inVal_->getMemoryDesc(),
wgt->getMemoryDesc(),
bias->getMemoryDesc(),
out->getMemoryDesc())
: fc_bwdWgt::desc(inVal_->getMemoryDesc(),
wgt->getMemoryDesc(),
out->getMemoryDesc());
pd.reset(new fc_bwdWgt::primitive_desc(bwdWgtDesc, engine_, *fwdPD_));
} }
void MKLDNNFcLayer::updateInputData() { void MKLDNNFcLayer::resetBwdDataPD(
inVal_->setData(getInputValue(0, CPU_DEVICE)->getData()); std::shared_ptr<fc_bwdData::primitive_desc>& pd,
MKLDNNMatrixPtr& in,
MKLDNNMatrixPtr& out) {
pd = nullptr;
if (in == nullptr) {
return;
}
CHECK(wgtVal_);
fc_bwdData::desc bwdDataDesc = fc_bwdData::desc(
in->getMemoryDesc(), wgtVal_->getMemoryDesc(), out->getMemoryDesc());
pd.reset(new fc_bwdData::primitive_desc(bwdDataDesc, engine_, *fwdPD_));
} }
void MKLDNNFcLayer::updateWeights(const UpdateCallback& callback) { void MKLDNNFcLayer::resetBwdPipeline(
weight_->getParameterPtr()->incUpdate(callback); std::vector<primitive>& pipeline,
if (biases_ && biases_->getWGrad()) { std::shared_ptr<fc_bwdWgt::primitive_desc>& bwdWgtPD,
biases_->getParameterPtr()->incUpdate(callback); std::shared_ptr<fc_bwdData::primitive_desc>& bwdDataPD,
MKLDNNMatrixPtr& in,
MKLDNNMatrixPtr& wgt,
MKLDNNMatrixPtr& bias,
MKLDNNMatrixPtr& out) {
pipeline.clear();
CHECK(inVal_);
if (bias) {
bwdWgt_.reset(new fc_bwdWgt(*bwdWgtPD, *inVal_, *out, *wgt, *bias));
} else {
bwdWgt_.reset(new fc_bwdWgt(*bwdWgtPD, *inVal_, *out, *wgt));
}
pipeline.push_back(*bwdWgt_);
if (bwdDataPD == nullptr) {
return;
} }
CHECK(wgtVal_) << "Should have weight memory";
bwdData_.reset(new fc_bwdData(*bwdDataPD, *out, *wgtVal_, *in));
pipeline.push_back(*bwdData_);
} }
} // namespace paddle } // namespace paddle
...@@ -18,6 +18,9 @@ limitations under the License. */ ...@@ -18,6 +18,9 @@ limitations under the License. */
#include "mkldnn.hpp" #include "mkldnn.hpp"
namespace paddle { namespace paddle {
typedef mkldnn::inner_product_forward fc_fwd;
typedef mkldnn::inner_product_backward_weights fc_bwdWgt;
typedef mkldnn::inner_product_backward_data fc_bwdData;
/** /**
* @brief A subclass of MKLDNNLayer fc layer. * @brief A subclass of MKLDNNLayer fc layer.
...@@ -32,6 +35,9 @@ protected: ...@@ -32,6 +35,9 @@ protected:
// if has already init the weight // if has already init the weight
bool hasInitedWgt_; bool hasInitedWgt_;
// save forward primitive_desc, which can be used backward
std::shared_ptr<fc_fwd::primitive_desc> fwdPD_;
// fc weight and bias // fc weight and bias
std::unique_ptr<Weight> weight_; std::unique_ptr<Weight> weight_;
std::unique_ptr<Weight> biases_; std::unique_ptr<Weight> biases_;
...@@ -67,6 +73,59 @@ public: ...@@ -67,6 +73,59 @@ public:
void convertWeightsFromPaddle() override; void convertWeightsFromPaddle() override;
void convertWeightsToPaddle() override; void convertWeightsToPaddle() override;
protected:
/**
* Forward functions: reset buffers(input, output, weight and bias),
* reset primitive descriptor,
* reset pipeline.
*/
void resetFwdBuffers(MKLDNNMatrixPtr& in,
MKLDNNMatrixPtr& wgt,
MKLDNNMatrixPtr& bias,
MKLDNNMatrixPtr& out);
void resetInValue(MKLDNNMatrixPtr& in);
void resetWgtBiasValue(MKLDNNMatrixPtr& wgt, MKLDNNMatrixPtr& bias);
void resetOutValue(MKLDNNMatrixPtr& out);
void resetFwdPD(std::shared_ptr<fc_fwd::primitive_desc>& pd,
MKLDNNMatrixPtr in,
MKLDNNMatrixPtr wgt,
MKLDNNMatrixPtr bias,
MKLDNNMatrixPtr out);
void resetFwdPipeline(std::vector<mkldnn::primitive>& pipeline,
std::shared_ptr<fc_fwd::primitive_desc>& pd,
MKLDNNMatrixPtr& in,
MKLDNNMatrixPtr& wgt,
MKLDNNMatrixPtr& bias,
MKLDNNMatrixPtr& out);
/**
* Backward functions: reset buffers(input, output, weight and bias),
* reset primitive descriptor for backward weight,
* reset primitive descriptor for backward data,
* reset pipeline.
*/
void resetBwdBuffers(MKLDNNMatrixPtr& in,
MKLDNNMatrixPtr& wgt,
MKLDNNMatrixPtr& bias,
MKLDNNMatrixPtr& out);
void resetOutGrad(MKLDNNMatrixPtr& out);
void resetWgtBiasGrad(MKLDNNMatrixPtr& wgt, MKLDNNMatrixPtr& bias);
void resetInGrad(MKLDNNMatrixPtr& in);
void resetBwdWgtPD(std::shared_ptr<fc_bwdWgt::primitive_desc>& pd,
MKLDNNMatrixPtr& wgt,
MKLDNNMatrixPtr& bias,
MKLDNNMatrixPtr& out);
void resetBwdDataPD(std::shared_ptr<fc_bwdData::primitive_desc>& pd,
MKLDNNMatrixPtr& in,
MKLDNNMatrixPtr& out);
void resetBwdPipeline(std::vector<mkldnn::primitive>& pipeline,
std::shared_ptr<fc_bwdWgt::primitive_desc>& bwdWgtPD,
std::shared_ptr<fc_bwdData::primitive_desc>& bwdDataPD,
MKLDNNMatrixPtr& in,
MKLDNNMatrixPtr& wgt,
MKLDNNMatrixPtr& bias,
MKLDNNMatrixPtr& out);
}; };
} // namespace paddle } // namespace paddle
...@@ -66,11 +66,12 @@ public: ...@@ -66,11 +66,12 @@ public:
/** /**
* Create reorder primitive. * Create reorder primitive.
* Create a mkldnn::reorder handle for converting src MKLDNNMatrix to dst. * Create a mkldnn::reorder handle for converting src MKLDNNMatrix to dst.
* checkData: for whether to check the data handle of src and dst is the same. * checkData: whether to check the data handle of src and dst.
* if true, means check it and do not want support inplace reorder; * if true, it will check the data and do not allow them equal;
* otherwise do not check data which means the created reorder * otherwise, it will not check them, then the reorder created
* maybe inplace buffer and do not guarantee the logical is correct * may have inplace buffer.
* since not all format or conversion support inplace. * Do not set false, if you can not guarantee the inplace logical
* would work with your reorder.
*/ */
static std::shared_ptr<mkldnn::reorder> createReorder( static std::shared_ptr<mkldnn::reorder> createReorder(
const MKLDNNMatrixPtr& src, const MKLDNNMatrixPtr& src,
......
...@@ -23,10 +23,15 @@ class AccuracyOp : public framework::OperatorWithKernel { ...@@ -23,10 +23,15 @@ class AccuracyOp : public framework::OperatorWithKernel {
protected: protected:
void InferShape(const framework::InferShapeContext &ctx) const override { void InferShape(const framework::InferShapeContext &ctx) const override {
PADDLE_ENFORCE_NOT_NULL(ctx.InputVar("Inference"), PADDLE_ENFORCE_NOT_NULL(
"Input of Inference must be initialized."); ctx.InputVar("Inference"),
"Input(Inference) of AccuracyOp should not be null.");
PADDLE_ENFORCE_NOT_NULL(ctx.InputVar("Label"), PADDLE_ENFORCE_NOT_NULL(ctx.InputVar("Label"),
"Input of Inference must be initialized."); "Input(Label) of AccuracyOp should not be null.");
PADDLE_ENFORCE_NOT_NULL(
ctx.OutputVar("Accuracy"),
"Output(Accuracy) of AccuracyOp should not be null.");
auto *inference = ctx.Input<framework::Tensor>("Inference"); auto *inference = ctx.Input<framework::Tensor>("Inference");
auto *label = ctx.Input<framework::Tensor>("Label"); auto *label = ctx.Input<framework::Tensor>("Label");
......
...@@ -23,6 +23,13 @@ class AddOp : public framework::OperatorWithKernel { ...@@ -23,6 +23,13 @@ class AddOp : public framework::OperatorWithKernel {
protected: protected:
void InferShape(const framework::InferShapeContext &ctx) const override { void InferShape(const framework::InferShapeContext &ctx) const override {
PADDLE_ENFORCE_NOT_NULL(ctx.InputVar("X"),
"Input(X) of AddOp should not be null.");
PADDLE_ENFORCE_NOT_NULL(ctx.InputVar("Y"),
"Input(Y) of AddOp should not be null.");
PADDLE_ENFORCE_NOT_NULL(ctx.OutputVar("Out"),
"Output(Out) of AddOp should not be null.");
PADDLE_ENFORCE_EQ(ctx.Input<Tensor>("X")->dims(), PADDLE_ENFORCE_EQ(ctx.Input<Tensor>("X")->dims(),
ctx.Input<Tensor>("Y")->dims(), ctx.Input<Tensor>("Y")->dims(),
"Two input of Add Op's dimension must be same."); "Two input of Add Op's dimension must be same.");
......
...@@ -25,6 +25,9 @@ class ConcatOp : public framework::OperatorWithKernel { ...@@ -25,6 +25,9 @@ class ConcatOp : public framework::OperatorWithKernel {
protected: protected:
void InferShape(const framework::InferShapeContext &ctx) const override { void InferShape(const framework::InferShapeContext &ctx) const override {
PADDLE_ENFORCE_NOT_NULL(ctx.OutputVar("Out"),
"Output(Out) of ConcatOp should not be null.");
auto ins = ctx.MultiInput<framework::Tensor>("X"); auto ins = ctx.MultiInput<framework::Tensor>("X");
auto *out = ctx.Output<framework::LoDTensor>("Out"); auto *out = ctx.Output<framework::LoDTensor>("Out");
size_t axis = static_cast<size_t>(ctx.Attr<int>("axis")); size_t axis = static_cast<size_t>(ctx.Attr<int>("axis"));
......
...@@ -33,7 +33,8 @@ using DDim = framework::DDim; ...@@ -33,7 +33,8 @@ using DDim = framework::DDim;
void CondOp::CreateScope(const Scope& scope) const { void CondOp::CreateScope(const Scope& scope) const {
auto sub_scopes_var = scope.FindVar("SubScopes"); auto sub_scopes_var = scope.FindVar("SubScopes");
PADDLE_ENFORCE(sub_scopes_var != nullptr, ""); PADDLE_ENFORCE_NOT_NULL(sub_scopes_var,
"Output(SubScopes) of CondOp should not be null.");
auto sub_scopes = sub_scopes_var->GetMutable<std::vector<Scope*>>(); auto sub_scopes = sub_scopes_var->GetMutable<std::vector<Scope*>>();
auto& sub_scope = scope.NewScope(); auto& sub_scope = scope.NewScope();
sub_scopes->push_back(&sub_scope); sub_scopes->push_back(&sub_scope);
...@@ -41,7 +42,8 @@ void CondOp::CreateScope(const Scope& scope) const { ...@@ -41,7 +42,8 @@ void CondOp::CreateScope(const Scope& scope) const {
void CondOp::CreateIndexTensor(const Scope& scope) const { void CondOp::CreateIndexTensor(const Scope& scope) const {
auto index_tensors_var = scope.FindVar("IndexTensors"); auto index_tensors_var = scope.FindVar("IndexTensors");
PADDLE_ENFORCE(index_tensors_var != nullptr, ""); PADDLE_ENFORCE_NOT_NULL(index_tensors_var,
"Output(IndexTensors) of CondOp should not be null.");
auto& index_tensors = auto& index_tensors =
*index_tensors_var->GetMutable<std::vector<LoDTensor>>(); *index_tensors_var->GetMutable<std::vector<LoDTensor>>();
index_tensors.push_back(LoDTensor()); index_tensors.push_back(LoDTensor());
...@@ -49,7 +51,8 @@ void CondOp::CreateIndexTensor(const Scope& scope) const { ...@@ -49,7 +51,8 @@ void CondOp::CreateIndexTensor(const Scope& scope) const {
void CondOp::InferShape(const Scope& scope) const { void CondOp::InferShape(const Scope& scope) const {
auto sub_scopes_var = scope.FindVar("SubScopes"); auto sub_scopes_var = scope.FindVar("SubScopes");
PADDLE_ENFORCE_NOT_NULL(sub_scopes_var); PADDLE_ENFORCE_NOT_NULL(sub_scopes_var,
"Output(SubScopes) of CondOp should not be null.");
auto& sub_scopes = *sub_scopes_var->GetMutable<std::vector<Scope*>>(); auto& sub_scopes = *sub_scopes_var->GetMutable<std::vector<Scope*>>();
for (int i = 0; i < 2; ++i) { for (int i = 0; i < 2; ++i) {
...@@ -63,7 +66,8 @@ void CondOp::InferShape(const Scope& scope) const { ...@@ -63,7 +66,8 @@ void CondOp::InferShape(const Scope& scope) const {
// branch // branch
CreateIndexTensor(scope); CreateIndexTensor(scope);
PADDLE_ENFORCE(!Inputs("Xs").empty(), "Inputs can't be empty"); PADDLE_ENFORCE(!Inputs("Xs").empty(),
"Inputs(Xs) of CondOp can't be empty.");
for (auto& input : Inputs("Xs")) { for (auto& input : Inputs("Xs")) {
// Create a new tensor in sub-scope for input-type tensor // Create a new tensor in sub-scope for input-type tensor
Variable* v = sub_scopes[i]->NewVar(input); Variable* v = sub_scopes[i]->NewVar(input);
...@@ -108,13 +112,18 @@ void CondOp::InferShape(const Scope& scope) const { ...@@ -108,13 +112,18 @@ void CondOp::InferShape(const Scope& scope) const {
void CondOp::Run(const Scope& scope, void CondOp::Run(const Scope& scope,
const platform::DeviceContext& dev_ctx) const { const platform::DeviceContext& dev_ctx) const {
auto* sub_scopes_var = scope.FindVar("SubScopes"); auto* sub_scopes_var = scope.FindVar("SubScopes");
PADDLE_ENFORCE_NOT_NULL(sub_scopes_var,
"Output(SubScopes) of CondOp should not be null.");
auto sub_scopes = sub_scopes_var->Get<std::vector<Scope*>>(); auto sub_scopes = sub_scopes_var->Get<std::vector<Scope*>>();
auto* index_tensors_var = scope.FindVar("IndexTensors"); auto* index_tensors_var = scope.FindVar("IndexTensors");
PADDLE_ENFORCE_NOT_NULL(index_tensors_var,
"Output(IndexTensors) of CondOp should not be null.");
auto index_tensors = index_tensors_var->Get<std::vector<LoDTensor>>(); auto index_tensors = index_tensors_var->Get<std::vector<LoDTensor>>();
std::string cond_name = Input("Cond"); std::string cond_name = Input("Cond");
Variable* cond_var = scope.FindVar(cond_name); Variable* cond_var = scope.FindVar(cond_name);
PADDLE_ENFORCE_NOT_NULL(cond_var); PADDLE_ENFORCE_NOT_NULL(cond_var,
"Input(Cond) of CondOp should not be null.");
const LoDTensor* cond = cond_var->GetMutable<LoDTensor>(); const LoDTensor* cond = cond_var->GetMutable<LoDTensor>();
// Step 1: get the true/false index at runtime // Step 1: get the true/false index at runtime
...@@ -171,6 +180,8 @@ void CondOp::Run(const Scope& scope, ...@@ -171,6 +180,8 @@ void CondOp::Run(const Scope& scope,
} }
// Step 4: merge output results // Step 4: merge output results
PADDLE_ENFORCE(!Outputs("Outs").empty(),
"Outputs(Outs) of CondOp can't be empty.");
for (int i = 0; i < 2; ++i) { for (int i = 0; i < 2; ++i) {
// i= 0/i for True and False branches respectively // i= 0/i for True and False branches respectively
for (auto& output : Outputs("Outs")) { for (auto& output : Outputs("Outs")) {
......
...@@ -26,8 +26,16 @@ class CosSimOp : public framework::OperatorWithKernel { ...@@ -26,8 +26,16 @@ class CosSimOp : public framework::OperatorWithKernel {
protected: protected:
void InferShape(const framework::InferShapeContext &ctx) const override { void InferShape(const framework::InferShapeContext &ctx) const override {
// notnull check // notnull check
PADDLE_ENFORCE_NOT_NULL(ctx.InputVar("X"), "Input(X) must not be null."); PADDLE_ENFORCE_NOT_NULL(ctx.InputVar("X"),
PADDLE_ENFORCE_NOT_NULL(ctx.InputVar("Y"), "Input(Y) must not be null."); "Input(X) of CosSimOp should not be null.");
PADDLE_ENFORCE_NOT_NULL(ctx.InputVar("Y"),
"Input(Y) of CosSimOp should not be null.");
PADDLE_ENFORCE_NOT_NULL(ctx.OutputVar("Out"),
"Output(Out) of CosSimOp should not be null.");
PADDLE_ENFORCE_NOT_NULL(ctx.OutputVar("XNorm"),
"Output(XNorm) of CosSimOp should not be null.");
PADDLE_ENFORCE_NOT_NULL(ctx.OutputVar("YNorm"),
"Output(YNorm) of CosSimOp should not be null.");
// shape check // shape check
auto x_dims = ctx.Input<Tensor>("X")->dims(); auto x_dims = ctx.Input<Tensor>("X")->dims();
......
...@@ -25,8 +25,14 @@ class ElementWiseMulOp : public framework::OperatorWithKernel { ...@@ -25,8 +25,14 @@ class ElementWiseMulOp : public framework::OperatorWithKernel {
protected: protected:
void InferShape(const framework::InferShapeContext &ctx) const override { void InferShape(const framework::InferShapeContext &ctx) const override {
PADDLE_ENFORCE_NOT_NULL(ctx.InputVar("X"), "Input(X) should not be null"); PADDLE_ENFORCE_NOT_NULL(ctx.InputVar("X"),
PADDLE_ENFORCE_NOT_NULL(ctx.InputVar("Y"), "Input(Y) should not be null"); "Input(X) of ElementWiseMulOp should not be null.");
PADDLE_ENFORCE_NOT_NULL(ctx.InputVar("Y"),
"Input(Y) of ElementWiseMulOp should not be null.");
PADDLE_ENFORCE_NOT_NULL(
ctx.OutputVar("Out"),
"Output(Out) of ElementWiseMulOp should not be null.");
auto x_dim = ctx.Input<Tensor>("X")->dims(); auto x_dim = ctx.Input<Tensor>("X")->dims();
auto y_dim = ctx.Input<Tensor>("Y")->dims(); auto y_dim = ctx.Input<Tensor>("Y")->dims();
PADDLE_ENFORCE_GE(x_dim.size(), y_dim.size(), PADDLE_ENFORCE_GE(x_dim.size(), y_dim.size(),
......
...@@ -13,10 +13,8 @@ ...@@ -13,10 +13,8 @@
limitations under the License. */ limitations under the License. */
#pragma once #pragma once
#include <iostream>
#include "paddle/framework/eigen.h" #include "paddle/framework/eigen.h"
#include "paddle/framework/op_registry.h" #include "paddle/framework/op_registry.h"
#include "paddle/operators/math/math_function.h"
namespace paddle { namespace paddle {
namespace operators { namespace operators {
......
...@@ -23,6 +23,13 @@ class FillZerosLikeOp : public framework::OperatorWithKernel { ...@@ -23,6 +23,13 @@ class FillZerosLikeOp : public framework::OperatorWithKernel {
protected: protected:
void InferShape(const framework::InferShapeContext &ctx) const override { void InferShape(const framework::InferShapeContext &ctx) const override {
PADDLE_ENFORCE_NOT_NULL(
ctx.InputVar("Src"),
"Input(Src) of FillZerosLikeOp should not be null.");
PADDLE_ENFORCE_NOT_NULL(
ctx.OutputVar("Dst"),
"Output(Dst) of FillZerosLikeOp should not be null.");
ctx.Output<framework::LoDTensor>("Dst")->Resize( ctx.Output<framework::LoDTensor>("Dst")->Resize(
ctx.Input<framework::Tensor>("Src")->dims()); ctx.Input<framework::Tensor>("Src")->dims());
} }
......
...@@ -24,6 +24,13 @@ class GatherOp : public framework::OperatorWithKernel { ...@@ -24,6 +24,13 @@ class GatherOp : public framework::OperatorWithKernel {
protected: protected:
void InferShape(const framework::InferShapeContext &ctx) const override { void InferShape(const framework::InferShapeContext &ctx) const override {
PADDLE_ENFORCE_NOT_NULL(ctx.InputVar("X"),
"Input(X) of GatherOp should not be null.");
PADDLE_ENFORCE_NOT_NULL(ctx.InputVar("Index"),
"Input(Index) of GatherOp should not be null.");
PADDLE_ENFORCE_NOT_NULL(ctx.OutputVar("Out"),
"Output(Out) of GatherOp should not be null.");
int batch_size = ctx.Input<Tensor>("Index")->dims()[0]; int batch_size = ctx.Input<Tensor>("Index")->dims()[0];
PADDLE_ENFORCE_GE(batch_size, 0, "Batch size must be >0"); PADDLE_ENFORCE_GE(batch_size, 0, "Batch size must be >0");
framework::DDim output_dims(ctx.Input<Tensor>("X")->dims()); framework::DDim output_dims(ctx.Input<Tensor>("X")->dims());
......
...@@ -43,8 +43,12 @@ class GaussianRandomOp : public framework::OperatorWithKernel { ...@@ -43,8 +43,12 @@ class GaussianRandomOp : public framework::OperatorWithKernel {
using framework::OperatorWithKernel::OperatorWithKernel; using framework::OperatorWithKernel::OperatorWithKernel;
protected: protected:
void InferShape(const framework::InferShapeContext& context) const override { void InferShape(const framework::InferShapeContext& ctx) const override {
auto* tensor = context.Output<framework::LoDTensor>("Out"); PADDLE_ENFORCE_NOT_NULL(
ctx.OutputVar("Out"),
"Output(Out) of GaussianRandomOp should not be null.");
auto* tensor = ctx.Output<framework::LoDTensor>("Out");
auto dims = Attr<std::vector<int>>("dims"); auto dims = Attr<std::vector<int>>("dims");
std::vector<int64_t> temp; std::vector<int64_t> temp;
temp.reserve(dims.size()); temp.reserve(dims.size());
......
...@@ -42,6 +42,11 @@ class IdentityOp : public NetOp { ...@@ -42,6 +42,11 @@ class IdentityOp : public NetOp {
const framework::VariableNameMap &outputs, const framework::VariableNameMap &outputs,
const framework::AttributeMap &attrs) const framework::AttributeMap &attrs)
: NetOp(type, inputs, outputs, attrs) { : NetOp(type, inputs, outputs, attrs) {
PADDLE_ENFORCE_NE(Input("X"), framework::kEmptyVarName,
"Input(X) of IdentityOp should not be null.");
PADDLE_ENFORCE_NE(Output("Out"), framework::kEmptyVarName,
"Output(Out) of IdentityOp should not be null.");
AppendOp(framework::OpRegistry::CreateOp( AppendOp(framework::OpRegistry::CreateOp(
"scale", {{"X", {Input("X")}}}, {{"Out", {Output("Out")}}}, "scale", {{"X", {Input("X")}}}, {{"Out", {Output("Out")}}},
{{"scale", static_cast<AttrType>(1)}})); {{"scale", static_cast<AttrType>(1)}}));
......
...@@ -22,10 +22,17 @@ class LookupTableOp : public framework::OperatorWithKernel { ...@@ -22,10 +22,17 @@ class LookupTableOp : public framework::OperatorWithKernel {
using framework::OperatorWithKernel::OperatorWithKernel; using framework::OperatorWithKernel::OperatorWithKernel;
protected: protected:
void InferShape(const framework::InferShapeContext &context) const override { void InferShape(const framework::InferShapeContext &ctx) const override {
auto table_t = context.Input<Tensor>("W"); PADDLE_ENFORCE_NOT_NULL(ctx.InputVar("W"),
auto ids_t = context.Input<Tensor>("Ids"); "Input(W) of LookupTableOp should not be null.");
auto output_t = context.Output<framework::LoDTensor>("Out"); PADDLE_ENFORCE_NOT_NULL(ctx.InputVar("Ids"),
"Input(Ids) of LookupTableOp should not be null.");
PADDLE_ENFORCE_NOT_NULL(ctx.OutputVar("Out"),
"Output(Out) of LookupTableOp should not be null.");
auto table_t = ctx.Input<Tensor>("W");
auto ids_t = ctx.Input<Tensor>("Ids");
auto output_t = ctx.Output<framework::LoDTensor>("Out");
output_t->Resize({ids_t->dims()[0], table_t->dims()[1]}); output_t->Resize({ids_t->dims()[0], table_t->dims()[1]});
} }
......
...@@ -24,7 +24,9 @@ class MeanOp : public framework::OperatorWithKernel { ...@@ -24,7 +24,9 @@ class MeanOp : public framework::OperatorWithKernel {
protected: protected:
void InferShape(const framework::InferShapeContext &ctx) const override { void InferShape(const framework::InferShapeContext &ctx) const override {
PADDLE_ENFORCE_NOT_NULL(ctx.InputVar("X"), PADDLE_ENFORCE_NOT_NULL(ctx.InputVar("X"),
"Input of MeanOp must be initialized."); "Input(X) of MeanOp should not be null.");
PADDLE_ENFORCE_NOT_NULL(ctx.OutputVar("Out"),
"Output(Out) of MeanOp should not be null.");
ctx.Output<framework::LoDTensor>("Out")->Resize({1}); ctx.Output<framework::LoDTensor>("Out")->Resize({1});
} }
}; };
......
...@@ -27,6 +27,13 @@ class MinusOp : public framework::OperatorWithKernel { ...@@ -27,6 +27,13 @@ class MinusOp : public framework::OperatorWithKernel {
protected: protected:
void InferShape(const framework::InferShapeContext &ctx) const override { void InferShape(const framework::InferShapeContext &ctx) const override {
PADDLE_ENFORCE_NOT_NULL(ctx.InputVar("X"),
"Input(X) of MinusOp should not be null.");
PADDLE_ENFORCE_NOT_NULL(ctx.InputVar("Y"),
"Input(Y) of MinusOp should not be null.");
PADDLE_ENFORCE_NOT_NULL(ctx.OutputVar("Out"),
"Output(Out) of MinusOp should not be null.");
auto *left_tensor = ctx.Input<framework::Tensor>("X"); auto *left_tensor = ctx.Input<framework::Tensor>("X");
auto *right_tensor = ctx.Input<framework::Tensor>("Y"); auto *right_tensor = ctx.Input<framework::Tensor>("Y");
...@@ -77,8 +84,6 @@ class MinusGradOp : public NetOp { ...@@ -77,8 +84,6 @@ class MinusGradOp : public NetOp {
} // namespace operators } // namespace operators
} // namespace paddle } // namespace paddle
USE_OP(scale);
USE_NO_KERNEL_OP(identity);
namespace ops = paddle::operators; namespace ops = paddle::operators;
REGISTER_OP(minus, ops::MinusOp, ops::MinusOpMaker, minus_grad, REGISTER_OP(minus, ops::MinusOp, ops::MinusOpMaker, minus_grad,
ops::MinusGradOp<float>); ops::MinusGradOp<float>);
......
...@@ -26,6 +26,13 @@ class MulOp : public framework::OperatorWithKernel { ...@@ -26,6 +26,13 @@ class MulOp : public framework::OperatorWithKernel {
protected: protected:
void InferShape(const framework::InferShapeContext &ctx) const override { void InferShape(const framework::InferShapeContext &ctx) const override {
PADDLE_ENFORCE_NOT_NULL(ctx.InputVar("X"),
"Input(X) of MulOp should not be null.");
PADDLE_ENFORCE_NOT_NULL(ctx.InputVar("Y"),
"Input(Y) of MulOp should not be null.");
PADDLE_ENFORCE_NOT_NULL(ctx.OutputVar("Out"),
"Output(Out) of MulOp should not be null.");
auto x_dims = ctx.Input<Tensor>("X")->dims(); auto x_dims = ctx.Input<Tensor>("X")->dims();
auto y_dims = ctx.Input<Tensor>("Y")->dims(); auto y_dims = ctx.Input<Tensor>("Y")->dims();
int x_num_col_dims = Attr<int>("x_num_col_dims"); int x_num_col_dims = Attr<int>("x_num_col_dims");
......
...@@ -23,6 +23,16 @@ class OnehotCrossEntropyOp : public framework::OperatorWithKernel { ...@@ -23,6 +23,16 @@ class OnehotCrossEntropyOp : public framework::OperatorWithKernel {
protected: protected:
void InferShape(const framework::InferShapeContext &ctx) const override { void InferShape(const framework::InferShapeContext &ctx) const override {
PADDLE_ENFORCE_NOT_NULL(
ctx.InputVar("X"),
"Input(X) of OnehotCrossEntropyOp should not be null.");
PADDLE_ENFORCE_NOT_NULL(
ctx.InputVar("label"),
"Input(label) of OnehotCrossEntropyOp should not be null.");
PADDLE_ENFORCE_NOT_NULL(
ctx.OutputVar("Y"),
"Output(Y) of OnehotCrossEntropyOp should not be null.");
auto *X = ctx.Input<Tensor>("X"); auto *X = ctx.Input<Tensor>("X");
auto *label = ctx.Input<Tensor>("label"); auto *label = ctx.Input<Tensor>("label");
......
...@@ -25,6 +25,11 @@ class PadOp : public framework::OperatorWithKernel { ...@@ -25,6 +25,11 @@ class PadOp : public framework::OperatorWithKernel {
protected: protected:
void InferShape(const framework::InferShapeContext &ctx) const override { void InferShape(const framework::InferShapeContext &ctx) const override {
PADDLE_ENFORCE_NOT_NULL(ctx.InputVar("X"),
"Input(X) of PadOp should not be null.");
PADDLE_ENFORCE_NOT_NULL(ctx.OutputVar("Out"),
"Output(Out) of PadOp should not be null.");
auto x_dim = ctx.Input<Tensor>("X")->dims(); auto x_dim = ctx.Input<Tensor>("X")->dims();
auto paddings = Attr<std::vector<int>>("paddings"); auto paddings = Attr<std::vector<int>>("paddings");
PADDLE_ENFORCE_EQ(x_dim.size() * 2, int64_t(paddings.size()), PADDLE_ENFORCE_EQ(x_dim.size() * 2, int64_t(paddings.size()),
......
...@@ -28,7 +28,11 @@ class ReshapeOp : public framework::OperatorWithKernel { ...@@ -28,7 +28,11 @@ class ReshapeOp : public framework::OperatorWithKernel {
protected: protected:
void InferShape(const framework::InferShapeContext &ctx) const override { void InferShape(const framework::InferShapeContext &ctx) const override {
// input check // input check
PADDLE_ENFORCE_NOT_NULL(ctx.InputVar("X"), "Input(X) shouldn't be null"); PADDLE_ENFORCE_NOT_NULL(ctx.InputVar("X"),
"Input(X) of ReshapeOp should not be null.");
PADDLE_ENFORCE_NOT_NULL(ctx.OutputVar("Out"),
"Output(Out) of ReshapeOp should not be null.");
auto shape = ctx.Attr<std::vector<int>>("shape"); auto shape = ctx.Attr<std::vector<int>>("shape");
PADDLE_ENFORCE(shape.size() > 0, "Attr(shape) shouldn't be empty."); PADDLE_ENFORCE(shape.size() > 0, "Attr(shape) shouldn't be empty.");
for (auto dim : shape) { for (auto dim : shape) {
......
...@@ -25,6 +25,13 @@ class RowwiseAddOp : public framework::OperatorWithKernel { ...@@ -25,6 +25,13 @@ class RowwiseAddOp : public framework::OperatorWithKernel {
protected: protected:
void InferShape(const framework::InferShapeContext &ctx) const override { void InferShape(const framework::InferShapeContext &ctx) const override {
PADDLE_ENFORCE_NOT_NULL(ctx.InputVar("X"),
"Input(X) of RowwiseAddOp should not be null.");
PADDLE_ENFORCE_NOT_NULL(ctx.InputVar("b"),
"Input(b) of RowwiseAddOp should not be null.");
PADDLE_ENFORCE_NOT_NULL(ctx.OutputVar("Out"),
"Output(Out) of RowwiseAddOp should not be null.");
auto x_dims = ctx.Input<Tensor>("X")->dims(); auto x_dims = ctx.Input<Tensor>("X")->dims();
auto b_dims = ctx.Input<Tensor>("b")->dims(); auto b_dims = ctx.Input<Tensor>("b")->dims();
PADDLE_ENFORCE_GT( PADDLE_ENFORCE_GT(
......
...@@ -27,6 +27,11 @@ class ScaleOp : public framework::OperatorWithKernel { ...@@ -27,6 +27,11 @@ class ScaleOp : public framework::OperatorWithKernel {
protected: protected:
void InferShape(const framework::InferShapeContext &ctx) const override { void InferShape(const framework::InferShapeContext &ctx) const override {
PADDLE_ENFORCE_NOT_NULL(ctx.InputVar("X"),
"Input(X) of ScaleOp should not be null.");
PADDLE_ENFORCE_NOT_NULL(ctx.OutputVar("Out"),
"Output(Out) of ScaleOp should not be null.");
auto *in = ctx.Input<framework::Tensor>("X"); auto *in = ctx.Input<framework::Tensor>("X");
auto *out = ctx.Output<framework::LoDTensor>("Out"); auto *out = ctx.Output<framework::LoDTensor>("Out");
out->Resize(in->dims()); out->Resize(in->dims());
......
...@@ -24,6 +24,15 @@ class ScatterOp : public framework::OperatorWithKernel { ...@@ -24,6 +24,15 @@ class ScatterOp : public framework::OperatorWithKernel {
protected: protected:
void InferShape(const framework::InferShapeContext &ctx) const override { void InferShape(const framework::InferShapeContext &ctx) const override {
PADDLE_ENFORCE_NOT_NULL(ctx.InputVar("Ref"),
"Input(Ref) of ScatterOp should not be null.");
PADDLE_ENFORCE_NOT_NULL(ctx.InputVar("Index"),
"Input(Index) of ScatterOp should not be null.");
PADDLE_ENFORCE_NOT_NULL(ctx.InputVar("Updates"),
"Input(Updates) of ScatterOp should not be null.");
PADDLE_ENFORCE_NOT_NULL(ctx.OutputVar("Out"),
"Output(Out) of ScatterOp should not be null.");
PADDLE_ENFORCE_EQ(ctx.Input<Tensor>("Index")->dims().size(), 1, PADDLE_ENFORCE_EQ(ctx.Input<Tensor>("Index")->dims().size(), 1,
"Update Index should be 1-D."); "Update Index should be 1-D.");
PADDLE_ENFORCE_EQ(ctx.Input<Tensor>("Ref")->dims().size(), PADDLE_ENFORCE_EQ(ctx.Input<Tensor>("Ref")->dims().size(),
......
...@@ -23,9 +23,12 @@ class SequenceAvgPoolOp : public framework::OperatorWithKernel { ...@@ -23,9 +23,12 @@ class SequenceAvgPoolOp : public framework::OperatorWithKernel {
protected: protected:
void InferShape(const framework::InferShapeContext& ctx) const override { void InferShape(const framework::InferShapeContext& ctx) const override {
PADDLE_ENFORCE_NOT_NULL(ctx.InputVar("X"), PADDLE_ENFORCE_NOT_NULL(
"Input of SequenceAvgPoolOp" ctx.InputVar("X"), "Input(X) of SequenceAvgPoolOp should not be null.");
"must be initialized."); PADDLE_ENFORCE_NOT_NULL(
ctx.OutputVar("Out"),
"Output(Out) of SequenceAvgPoolOp should not be null.");
auto* x = ctx.Input<framework::LoDTensor>("X"); auto* x = ctx.Input<framework::LoDTensor>("X");
auto dims = x->dims(); auto dims = x->dims();
auto lod = x->lod(); auto lod = x->lod();
...@@ -60,7 +63,9 @@ class SequenceAvgPoolGradOp : public framework::OperatorWithKernel { ...@@ -60,7 +63,9 @@ class SequenceAvgPoolGradOp : public framework::OperatorWithKernel {
protected: protected:
void InferShape(const framework::InferShapeContext& ctx) const override { void InferShape(const framework::InferShapeContext& ctx) const override {
PADDLE_ENFORCE_NOT_NULL(ctx.InputVar(framework::GradVarName("Out")), PADDLE_ENFORCE_NOT_NULL(ctx.InputVar(framework::GradVarName("Out")),
"Gradient of Out should not be null"); "Gradient of Out should not be null.");
PADDLE_ENFORCE_NOT_NULL(ctx.InputVar("X"),
"The input X should not be null.");
auto og_dims = auto og_dims =
ctx.Input<framework::LoDTensor>(framework::GradVarName("Out"))->dims(); ctx.Input<framework::LoDTensor>(framework::GradVarName("Out"))->dims();
auto x_dims = ctx.Input<framework::LoDTensor>("X")->dims(); auto x_dims = ctx.Input<framework::LoDTensor>("X")->dims();
......
...@@ -21,6 +21,9 @@ namespace operators { ...@@ -21,6 +21,9 @@ namespace operators {
using Tensor = framework::Tensor; using Tensor = framework::Tensor;
using LoDTensor = framework::LoDTensor; using LoDTensor = framework::LoDTensor;
template <typename T, int MajorType = Eigen::RowMajor,
typename IndexType = Eigen::DenseIndex>
using EigenVector = framework::EigenVector<T, MajorType, IndexType>;
template <typename T, int MajorType = Eigen::RowMajor, template <typename T, int MajorType = Eigen::RowMajor,
typename IndexType = Eigen::DenseIndex> typename IndexType = Eigen::DenseIndex>
using EigenMatrix = framework::EigenMatrix<T, MajorType, IndexType>; using EigenMatrix = framework::EigenMatrix<T, MajorType, IndexType>;
...@@ -43,8 +46,8 @@ class SequenceAvgPoolKernel : public framework::OpKernel { ...@@ -43,8 +46,8 @@ class SequenceAvgPoolKernel : public framework::OpKernel {
static_cast<int>(lod[0][i + 1])); static_cast<int>(lod[0][i + 1]));
Tensor out_t = out->Slice<T>(i, i + 1); Tensor out_t = out->Slice<T>(i, i + 1);
int64_t h = static_cast<int64_t>(lod[0][i + 1] - lod[0][i]); int64_t h = static_cast<int64_t>(lod[0][i + 1] - lod[0][i]);
auto in_e = EigenMatrix<T>::From(in_t, {h, w}); auto in_e = EigenMatrix<T>::From(in_t, framework::make_ddim({h, w}));
auto out_e = EigenMatrix<T>::From(out_t, {h, w}); auto out_e = EigenVector<T>::Flatten(out_t);
out_e.device(place) = in_e.mean(Eigen::array<int, 1>({{0}})); out_e.device(place) = in_e.mean(Eigen::array<int, 1>({{0}}));
} }
} }
...@@ -54,9 +57,9 @@ template <typename Place, typename T> ...@@ -54,9 +57,9 @@ template <typename Place, typename T>
class SequenceAvgPoolGradKernel : public framework::OpKernel { class SequenceAvgPoolGradKernel : public framework::OpKernel {
public: public:
void Compute(const framework::ExecutionContext& context) const override { void Compute(const framework::ExecutionContext& context) const override {
auto* in = context.Output<LoDTensor>("X"); auto* in = context.Input<LoDTensor>("X");
auto* in_g = context.Output<LoDTensor>(framework::GradVarName("X"));
auto* out_g = context.Input<LoDTensor>(framework::GradVarName("Out")); auto* out_g = context.Input<LoDTensor>(framework::GradVarName("Out"));
auto* in_g = context.Output<LoDTensor>(framework::GradVarName("X"));
auto dims = in->dims(); auto dims = in->dims();
auto lod = in->lod(); auto lod = in->lod();
...@@ -71,7 +74,7 @@ class SequenceAvgPoolGradKernel : public framework::OpKernel { ...@@ -71,7 +74,7 @@ class SequenceAvgPoolGradKernel : public framework::OpKernel {
int64_t h = static_cast<int64_t>(lod[0][i + 1] - lod[0][i]); int64_t h = static_cast<int64_t>(lod[0][i + 1] - lod[0][i]);
auto in_g_e = EigenMatrix<T>::From(in_g_t, {h, w}); auto in_g_e = EigenMatrix<T>::From(in_g_t, {h, w});
auto out_g_e = EigenMatrix<T>::From(out_g_t, {1, w}); auto out_g_e = EigenMatrix<T>::From(out_g_t, {1, w});
Eigen::DSizes<int, 2> bcast(h, w); Eigen::DSizes<int, 2> bcast(h, 1);
in_g_e.device(place) = (out_g_e / static_cast<T>(h)).broadcast(bcast); in_g_e.device(place) = (out_g_e / static_cast<T>(h)).broadcast(bcast);
} }
} }
......
...@@ -23,6 +23,13 @@ class SGDOp : public framework::OperatorWithKernel { ...@@ -23,6 +23,13 @@ class SGDOp : public framework::OperatorWithKernel {
protected: protected:
void InferShape(const framework::InferShapeContext &ctx) const override { void InferShape(const framework::InferShapeContext &ctx) const override {
PADDLE_ENFORCE_NOT_NULL(ctx.InputVar("param"),
"Input(param) of SGDOp should not be null.");
PADDLE_ENFORCE_NOT_NULL(ctx.InputVar("grad"),
"Input(grad) of SGDOp should not be null.");
PADDLE_ENFORCE_NOT_NULL(ctx.OutputVar("param_out"),
"Output(param_out) of SGDOp should not be null.");
PADDLE_ENFORCE_EQ(ctx.Input<Tensor>("param")->dims(), PADDLE_ENFORCE_EQ(ctx.Input<Tensor>("param")->dims(),
ctx.Input<Tensor>("grad")->dims(), ctx.Input<Tensor>("grad")->dims(),
"Two input of SGD Op's dimension must be same."); "Two input of SGD Op's dimension must be same.");
......
...@@ -23,6 +23,11 @@ class SoftmaxOp : public framework::OperatorWithKernel { ...@@ -23,6 +23,11 @@ class SoftmaxOp : public framework::OperatorWithKernel {
protected: protected:
void InferShape(const framework::InferShapeContext &ctx) const override { void InferShape(const framework::InferShapeContext &ctx) const override {
PADDLE_ENFORCE_NOT_NULL(ctx.InputVar("X"),
"Input(X) of SoftmaxOp should not be null.");
PADDLE_ENFORCE_NOT_NULL(ctx.OutputVar("Y"),
"Output(Y) of SoftmaxOp should not be null.");
PADDLE_ENFORCE(ctx.Input<Tensor>("X")->dims().size() == 2UL, PADDLE_ENFORCE(ctx.Input<Tensor>("X")->dims().size() == 2UL,
"The input of softmax op must be a matrix."); "The input of softmax op must be a matrix.");
ctx.Output<framework::LoDTensor>("Y")->Resize( ctx.Output<framework::LoDTensor>("Y")->Resize(
......
...@@ -23,12 +23,18 @@ class SquaredL2DistanceOp : public framework::OperatorWithKernel { ...@@ -23,12 +23,18 @@ class SquaredL2DistanceOp : public framework::OperatorWithKernel {
protected: protected:
void InferShape(const framework::InferShapeContext& ctx) const override { void InferShape(const framework::InferShapeContext& ctx) const override {
PADDLE_ENFORCE_NOT_NULL(ctx.InputVar("X"), PADDLE_ENFORCE_NOT_NULL(
"Input of SquaredL2DistanceOp " ctx.InputVar("X"),
"must be initialized."); "Input(X) of SquaredL2DistanceOp should not be null.");
PADDLE_ENFORCE_NOT_NULL(ctx.InputVar("Y"), PADDLE_ENFORCE_NOT_NULL(
"Target of SquaredL2DistanceOp " ctx.InputVar("Y"),
"must be initialized."); "Input(Y) of SquaredL2DistanceOp should not be null.");
PADDLE_ENFORCE_NOT_NULL(
ctx.OutputVar("sub_result"),
"Output(sub_result) of SquaredL2DistanceOp should not be null.");
PADDLE_ENFORCE_NOT_NULL(
ctx.OutputVar("Out"),
"Output(Out) of SquaredL2DistanceOp should not be null.");
auto* x = ctx.Input<Tensor>("X"); auto* x = ctx.Input<Tensor>("X");
auto x_dims = x->dims(); auto x_dims = x->dims();
......
...@@ -22,6 +22,11 @@ class SumOp : public framework::OperatorWithKernel { ...@@ -22,6 +22,11 @@ class SumOp : public framework::OperatorWithKernel {
protected: protected:
void InferShape(const framework::InferShapeContext &ctx) const override { void InferShape(const framework::InferShapeContext &ctx) const override {
PADDLE_ENFORCE(!ctx.MultiInputVar("X").empty(),
"Input(X) of SumOp should not be null.");
PADDLE_ENFORCE_NOT_NULL(ctx.OutputVar("Out"),
"Output(Out) of SumOp should not be null.");
auto ins = ctx.MultiInput<framework::Tensor>("X"); auto ins = ctx.MultiInput<framework::Tensor>("X");
auto *out = ctx.Output<framework::LoDTensor>("Out"); auto *out = ctx.Output<framework::LoDTensor>("Out");
int N = ins.size(); int N = ins.size();
......
...@@ -24,7 +24,12 @@ class TopkOp : public framework::OperatorWithKernel { ...@@ -24,7 +24,12 @@ class TopkOp : public framework::OperatorWithKernel {
protected: protected:
void InferShape(const framework::InferShapeContext &ctx) const override { void InferShape(const framework::InferShapeContext &ctx) const override {
PADDLE_ENFORCE_NOT_NULL(ctx.InputVar("X"), PADDLE_ENFORCE_NOT_NULL(ctx.InputVar("X"),
"Input of TopkOP must be initialized."); "Input(X) of TopkOp should not be null.");
PADDLE_ENFORCE_NOT_NULL(ctx.OutputVar("Out"),
"Output(Out) of TopkOp should not be null.");
PADDLE_ENFORCE_NOT_NULL(ctx.OutputVar("Indices"),
"Output(Indices) of TopkOp should not be null.");
auto *input = ctx.Input<framework::Tensor>("X"); auto *input = ctx.Input<framework::Tensor>("X");
const int k = static_cast<int>(ctx.Attr<int>("k")); const int k = static_cast<int>(ctx.Attr<int>("k"));
......
...@@ -48,6 +48,10 @@ class UniformRandomOp : public framework::OperatorWithKernel { ...@@ -48,6 +48,10 @@ class UniformRandomOp : public framework::OperatorWithKernel {
protected: protected:
void InferShape(const framework::InferShapeContext& ctx) const override { void InferShape(const framework::InferShapeContext& ctx) const override {
PADDLE_ENFORCE_NOT_NULL(
ctx.OutputVar("Out"),
"Output(Out) of UniformRandomOp should not be null.");
PADDLE_ENFORCE(Attr<float>("min") < Attr<float>("max"), PADDLE_ENFORCE(Attr<float>("min") < Attr<float>("max"),
"uniform_random's min must less then max"); "uniform_random's min must less then max");
auto* tensor = ctx.Output<framework::LoDTensor>("Out"); auto* tensor = ctx.Output<framework::LoDTensor>("Out");
......
...@@ -11,10 +11,8 @@ ...@@ -11,10 +11,8 @@
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and # See the License for the specific language governing permissions and
# limitations under the License. # limitations under the License.
"""
"""
# from activations import *
from activations import LinearActivation, ReluActivation, SoftmaxActivation, \ from activations import LinearActivation, ReluActivation, SoftmaxActivation, \
IdentityActivation, TanhActivation, SequenceSoftmaxActivation IdentityActivation, TanhActivation, SequenceSoftmaxActivation
from attrs import ExtraAttr from attrs import ExtraAttr
...@@ -55,49 +53,49 @@ def sequence_conv_pool(input, ...@@ -55,49 +53,49 @@ def sequence_conv_pool(input,
context_attr=None, context_attr=None,
pool_attr=None): pool_attr=None):
""" """
Text convolution pooling layers helper. Text convolution pooling group.
Text input => Context Projection => FC Layer => Pooling => Output. Text input => Context Projection => FC Layer => Pooling => Output.
:param name: name of output layer(pooling layer name) :param name: group name.
:type name: basestring :type name: basestring
:param input: name of input layer :param input: input layer.
:type input: LayerOutput :type input: LayerOutput
:param context_len: context projection length. See :param context_len: context projection length. See
context_projection's document. context_projection's document.
:type context_len: int :type context_len: int
:param hidden_size: FC Layer size. :param hidden_size: FC Layer size.
:type hidden_size: int :type hidden_size: int
:param context_start: context projection length. See :param context_start: context start position. See
context_projection's context_start. context_projection's context_start.
:type context_start: int or None :type context_start: int|None
:param pool_type: pooling layer type. See pooling_layer's document. :param pool_type: pooling layer type. See pooling_layer's document.
:type pool_type: BasePoolingType. :type pool_type: BasePoolingType
:param context_proj_layer_name: context projection layer name. :param context_proj_layer_name: context projection layer name.
None if user don't care. None if user don't care.
:type context_proj_layer_name: basestring :type context_proj_layer_name: basestring
:param context_proj_param_attr: context projection parameter attribute. :param context_proj_param_attr: padding parameter attribute of context projection layer.
None if user don't care. If false, it means padding always be zero.
:type context_proj_param_attr: ParameterAttribute or None. :type context_proj_param_attr: ParameterAttribute|None
:param fc_layer_name: fc layer name. None if user don't care. :param fc_layer_name: fc layer name. None if user don't care.
:type fc_layer_name: basestring :type fc_layer_name: basestring
:param fc_param_attr: fc layer parameter attribute. None if user don't care. :param fc_param_attr: fc layer parameter attribute. None if user don't care.
:type fc_param_attr: ParameterAttribute or None :type fc_param_attr: ParameterAttribute|None
:param fc_bias_attr: fc bias parameter attribute. False if no bias, :param fc_bias_attr: fc bias parameter attribute. False if no bias,
None if user don't care. None if user don't care.
:type fc_bias_attr: ParameterAttribute or None :type fc_bias_attr: ParameterAttribute|False|None
:param fc_act: fc layer activation type. None means tanh :param fc_act: fc layer activation type. None means tanh.
:type fc_act: BaseActivation :type fc_act: BaseActivation
:param pool_bias_attr: pooling layer bias attr. None if don't care. :param pool_bias_attr: pooling layer bias attr. False if no bias.
False if no bias. None if user don't care.
:type pool_bias_attr: ParameterAttribute or None. :type pool_bias_attr: ParameterAttribute|False|None
:param fc_attr: fc layer extra attribute. :param fc_attr: fc layer extra attribute.
:type fc_attr: ExtraLayerAttribute :type fc_attr: ExtraLayerAttribute
:param context_attr: context projection layer extra attribute. :param context_attr: context projection layer extra attribute.
:type context_attr: ExtraLayerAttribute :type context_attr: ExtraLayerAttribute
:param pool_attr: pooling layer extra attribute. :param pool_attr: pooling layer extra attribute.
:type pool_attr: ExtraLayerAttribute :type pool_attr: ExtraLayerAttribute
:return: output layer name. :return: layer's output.
:rtype: LayerOutput :rtype: LayerOutput
""" """
# Set Default Value to param # Set Default Value to param
...@@ -163,45 +161,45 @@ def simple_img_conv_pool(input, ...@@ -163,45 +161,45 @@ def simple_img_conv_pool(input,
""" """
Simple image convolution and pooling group. Simple image convolution and pooling group.
Input => conv => pooling Img input => Conv => Pooling => Output.
:param name: group name :param name: group name.
:type name: basestring :type name: basestring
:param input: input layer name. :param input: input layer.
:type input: LayerOutput :type input: LayerOutput
:param filter_size: see img_conv_layer for details :param filter_size: see img_conv_layer for details.
:type filter_size: int :type filter_size: int
:param num_filters: see img_conv_layer for details :param num_filters: see img_conv_layer for details.
:type num_filters: int :type num_filters: int
:param pool_size: see img_pool_layer for details :param pool_size: see img_pool_layer for details.
:type pool_size: int :type pool_size: int
:param pool_type: see img_pool_layer for details :param pool_type: see img_pool_layer for details.
:type pool_type: BasePoolingType :type pool_type: BasePoolingType
:param act: see img_conv_layer for details :param act: see img_conv_layer for details.
:type act: BaseActivation :type act: BaseActivation
:param groups: see img_conv_layer for details :param groups: see img_conv_layer for details.
:type groups: int :type groups: int
:param conv_stride: see img_conv_layer for details :param conv_stride: see img_conv_layer for details.
:type conv_stride: int :type conv_stride: int
:param conv_padding: see img_conv_layer for details :param conv_padding: see img_conv_layer for details.
:type conv_padding: int :type conv_padding: int
:param bias_attr: see img_conv_layer for details :param bias_attr: see img_conv_layer for details.
:type bias_attr: ParameterAttribute :type bias_attr: ParameterAttribute
:param num_channel: see img_conv_layer for details :param num_channel: see img_conv_layer for details.
:type num_channel: int :type num_channel: int
:param param_attr: see img_conv_layer for details :param param_attr: see img_conv_layer for details.
:type param_attr: ParameterAttribute :type param_attr: ParameterAttribute
:param shared_bias: see img_conv_layer for details :param shared_bias: see img_conv_layer for details.
:type shared_bias: bool :type shared_bias: bool
:param conv_layer_attr: see img_conv_layer for details :param conv_layer_attr: see img_conv_layer for details.
:type conv_layer_attr: ExtraLayerAttribute :type conv_layer_attr: ExtraLayerAttribute
:param pool_stride: see img_pool_layer for details :param pool_stride: see img_pool_layer for details.
:type pool_stride: int :type pool_stride: int
:param pool_padding: see img_pool_layer for details :param pool_padding: see img_pool_layer for details.
:type pool_padding: int :type pool_padding: int
:param pool_layer_attr: see img_pool_layer for details :param pool_layer_attr: see img_pool_layer for details.
:type pool_layer_attr: ExtraLayerAttribute :type pool_layer_attr: ExtraLayerAttribute
:return: Layer's output :return: layer's output
:rtype: LayerOutput :rtype: LayerOutput
""" """
_conv_ = img_conv_layer( _conv_ = img_conv_layer(
...@@ -252,48 +250,52 @@ def img_conv_bn_pool(input, ...@@ -252,48 +250,52 @@ def img_conv_bn_pool(input,
pool_layer_attr=None): pool_layer_attr=None):
""" """
Convolution, batch normalization, pooling group. Convolution, batch normalization, pooling group.
Img input => Conv => BN => Pooling => Output.
:param name: group name :param name: group name.
:type name: basestring :type name: basestring
:param input: layer's input :param input: input layer.
:type input: LayerOutput :type input: LayerOutput
:param filter_size: see img_conv_layer's document :param filter_size: see img_conv_layer for details.
:type filter_size: int :type filter_size: int
:param num_filters: see img_conv_layer's document :param num_filters: see img_conv_layer for details.
:type num_filters: int :type num_filters: int
:param pool_size: see img_pool_layer's document. :param pool_size: see img_pool_layer for details.
:type pool_size: int :type pool_size: int
:param pool_type: see img_pool_layer's document. :param pool_type: see img_pool_layer for details.
:type pool_type: BasePoolingType :type pool_type: BasePoolingType
:param act: see batch_norm_layer's document. :param act: see batch_norm_layer for details.
:type act: BaseActivation :type act: BaseActivation
:param groups: see img_conv_layer's document :param groups: see img_conv_layer for details.
:type groups: int :type groups: int
:param conv_stride: see img_conv_layer's document. :param conv_stride: see img_conv_layer for details.
:type conv_stride: int :type conv_stride: int
:param conv_padding: see img_conv_layer's document. :param conv_padding: see img_conv_layer for details.
:type conv_padding: int :type conv_padding: int
:param conv_bias_attr: see img_conv_layer's document. :param conv_bias_attr: see img_conv_layer for details.
:type conv_bias_attr: ParameterAttribute :type conv_bias_attr: ParameterAttribute
:param num_channel: see img_conv_layer's document. :param num_channel: see img_conv_layer for details.
:type num_channel: int :type num_channel: int
:param conv_param_attr: see img_conv_layer's document. :param conv_param_attr: see img_conv_layer for details.
:type conv_param_attr: ParameterAttribute :type conv_param_attr: ParameterAttribute
:param shared_bias: see img_conv_layer's document. :param shared_bias: see img_conv_layer for details.
:type shared_bias: bool :type shared_bias: bool
:param conv_layer_attr: see img_conv_layer's document. :param conv_layer_attr: see img_conv_layer for details.
:type conv_layer_attr: ExtraLayerOutput :type conv_layer_attr: ExtraLayerOutput
:param bn_param_attr: see batch_norm_layer's document. :param bn_param_attr: see batch_norm_layer for details.
:type bn_param_attr: ParameterAttribute. :type bn_param_attr: ParameterAttribute
:param bn_bias_attr: see batch_norm_layer's document. :param bn_bias_attr: see batch_norm_layer for details.
:param bn_layer_attr: ParameterAttribute. :type bn_bias_attr: ParameterAttribute
:param pool_stride: see img_pool_layer's document. :param bn_layer_attr: see batch_norm_layer for details.
:type bn_layer_attr: ExtraLayerAttribute
:param pool_stride: see img_pool_layer for details.
:type pool_stride: int :type pool_stride: int
:param pool_padding: see img_pool_layer's document. :param pool_padding: see img_pool_layer for details.
:type pool_padding: int :type pool_padding: int
:param pool_layer_attr: see img_pool_layer's document. :param pool_layer_attr: see img_pool_layer for details.
:type pool_layer_attr: ExtraLayerAttribute :type pool_layer_attr: ExtraLayerAttribute
:return: Layer groups output :return: layer's output
:rtype: LayerOutput :rtype: LayerOutput
""" """
__conv__ = img_conv_layer( __conv__ = img_conv_layer(
...@@ -348,10 +350,10 @@ def img_conv_group(input, ...@@ -348,10 +350,10 @@ def img_conv_group(input,
:param conv_batchnorm_drop_rate: if conv_with_batchnorm[i] is true, :param conv_batchnorm_drop_rate: if conv_with_batchnorm[i] is true,
conv_batchnorm_drop_rate[i] represents the drop rate of each batch norm. conv_batchnorm_drop_rate[i] represents the drop rate of each batch norm.
:type conv_batchnorm_drop_rate: list :type conv_batchnorm_drop_rate: list
:param input: layer's input. :param input: input layer.
:type input: LayerOutput :type input: LayerOutput
:param conv_num_filter: output channels num. :param conv_num_filter: list of output channels num.
:type conv_num_filter: int :type conv_num_filter: list|tuple
:param pool_size: pooling filter size. :param pool_size: pooling filter size.
:type pool_size: int :type pool_size: int
:param num_channels: input channels num. :param num_channels: input channels num.
...@@ -362,18 +364,18 @@ def img_conv_group(input, ...@@ -362,18 +364,18 @@ def img_conv_group(input,
:type conv_filter_size: int :type conv_filter_size: int
:param conv_act: activation funciton after convolution. :param conv_act: activation funciton after convolution.
:type conv_act: BaseActivation :type conv_act: BaseActivation
:param conv_with_batchnorm: conv_with_batchnorm[i] represents :param conv_with_batchnorm: if conv_with_batchnorm[i] is true,
if there is a batch normalization after each convolution. there is a batch normalization operation after each convolution.
:type conv_with_batchnorm: list :type conv_with_batchnorm: list
:param pool_stride: pooling stride size. :param pool_stride: pooling stride size.
:type pool_stride: int :type pool_stride: int
:param pool_type: pooling type. :param pool_type: pooling type.
:type pool_type: BasePoolingType :type pool_type: BasePoolingType
:param param_attr: Convolution param attribute. :param param_attr: param attribute of convolution layer,
None means default attribute. None means default attribute.
:type param_attr: ParameterAttribute :type param_attr: ParameterAttribute
:return: Layer's output :return: layer's output
:type: LayerOutput :rtype: LayerOutput
""" """
tmp = input tmp = input
...@@ -466,12 +468,14 @@ def vgg_16_network(input_image, num_channels, num_classes=1000): ...@@ -466,12 +468,14 @@ def vgg_16_network(input_image, num_channels, num_classes=1000):
""" """
Same model from https://gist.github.com/ksimonyan/211839e770f7b538e2d8 Same model from https://gist.github.com/ksimonyan/211839e770f7b538e2d8
:param num_classes: :param num_classes: number of class.
:param input_image: :type num_classes: int
:param input_image: input layer.
:type input_image: LayerOutput :type input_image: LayerOutput
:param num_channels: :param num_channels: input channels num.
:type num_channels: int :type num_channels: int
:return: :return: layer's output
:rtype: LayerOutput
""" """
tmp = img_conv_group( tmp = img_conv_group(
...@@ -560,8 +564,8 @@ def simple_lstm(input, ...@@ -560,8 +564,8 @@ def simple_lstm(input,
""" """
Simple LSTM Cell. Simple LSTM Cell.
It just combine a mixed layer with fully_matrix_projection and a lstmemory It just combines a mixed layer with fully_matrix_projection and a lstmemory
layer. The simple lstm cell was implemented as follow equations. layer. The simple lstm cell was implemented with follow equations.
.. math:: .. math::
...@@ -575,37 +579,37 @@ def simple_lstm(input, ...@@ -575,37 +579,37 @@ def simple_lstm(input,
h_t & = o_t tanh(c_t) h_t & = o_t tanh(c_t)
Please refer **Generating Sequences With Recurrent Neural Networks** if you Please refer to **Generating Sequences With Recurrent Neural Networks** for more
want to know what lstm is. Link_ is here. details about lstm. Link_ is here.
.. _Link: http://arxiv.org/abs/1308.0850 .. _Link: http://arxiv.org/abs/1308.0850
:param name: lstm layer name. :param name: lstm layer name.
:type name: basestring :type name: basestring
:param input: input layer name. :param input: layer's input.
:type input: LayerOutput :type input: LayerOutput
:param size: lstm layer size. :param size: lstm layer size.
:type size: int :type size: int
:param reverse: whether to process the input data in a reverse order :param reverse: process the input in a reverse order or not.
:type reverse: bool :type reverse: bool
:param mat_param_attr: mixed layer's matrix projection parameter attribute. :param mat_param_attr: parameter attribute of matrix projection in mixed layer.
:type mat_param_attr: ParameterAttribute :type mat_param_attr: ParameterAttribute
:param bias_param_attr: bias parameter attribute. False means no bias, None :param bias_param_attr: bias parameter attribute. False means no bias, None
means default bias. means default bias.
:type bias_param_attr: ParameterAttribute|False :type bias_param_attr: ParameterAttribute|False
:param inner_param_attr: lstm cell parameter attribute. :param inner_param_attr: parameter attribute of lstm cell.
:type inner_param_attr: ParameterAttribute :type inner_param_attr: ParameterAttribute
:param act: lstm final activiation type :param act: last activiation type of lstm.
:type act: BaseActivation :type act: BaseActivation
:param gate_act: lstm gate activiation type :param gate_act: gate activiation type of lstm.
:type gate_act: BaseActivation :type gate_act: BaseActivation
:param state_act: lstm state activiation type. :param state_act: state activiation type of lstm.
:type state_act: BaseActivation :type state_act: BaseActivation
:param mixed_layer_attr: mixed layer's extra attribute. :param mixed_layer_attr: extra attribute of mixed layer.
:type mixed_layer_attr: ExtraLayerAttribute :type mixed_layer_attr: ExtraLayerAttribute
:param lstm_cell_attr: lstm layer's extra attribute. :param lstm_cell_attr: extra attribute of lstm.
:type lstm_cell_attr: ExtraLayerAttribute :type lstm_cell_attr: ExtraLayerAttribute
:return: lstm layer name. :return: layer's output.
:rtype: LayerOutput :rtype: LayerOutput
""" """
fc_name = 'lstm_transform_%s' % name fc_name = 'lstm_transform_%s' % name
...@@ -643,9 +647,9 @@ def lstmemory_unit(input, ...@@ -643,9 +647,9 @@ def lstmemory_unit(input,
lstm_bias_attr=None, lstm_bias_attr=None,
lstm_layer_attr=None): lstm_layer_attr=None):
""" """
Define calculations that a LSTM unit performs during a single time step. lstmemory_unit defines the caculation process of a LSTM unit during a
This function itself is not a recurrent layer, so it can not be single time step. This function is not a recurrent layer, so it can not be
directly used to process sequence inputs. This function is always used in directly used to process sequence input. This function is always used in
recurrent_group (see layers.py for more details) to implement attention recurrent_group (see layers.py for more details) to implement attention
mechanism. mechanism.
...@@ -676,7 +680,7 @@ def lstmemory_unit(input, ...@@ -676,7 +680,7 @@ def lstmemory_unit(input,
state_act=TanhActivation()) state_act=TanhActivation())
:param input: input layer name. :param input: input layer.
:type input: LayerOutput :type input: LayerOutput
:param out_memory: output of previous time step :param out_memory: output of previous time step
:type out_memory: LayerOutput | None :type out_memory: LayerOutput | None
...@@ -684,15 +688,15 @@ def lstmemory_unit(input, ...@@ -684,15 +688,15 @@ def lstmemory_unit(input,
:type name: basestring :type name: basestring
:param size: lstmemory unit size. :param size: lstmemory unit size.
:type size: int :type size: int
:param param_attr: Parameter config, None if use default. :param param_attr: parameter attribute, None means default attribute.
:type param_attr: ParameterAttribute :type param_attr: ParameterAttribute
:param act: lstm final activiation type :param act: last activiation type of lstm.
:type act: BaseActivation :type act: BaseActivation
:param gate_act: lstm gate activiation type :param gate_act: gate activiation type of lstm.
:type gate_act: BaseActivation :type gate_act: BaseActivation
:param state_act: lstm state activiation type. :param state_act: state activiation type of lstm.
:type state_act: BaseActivation :type state_act: BaseActivation
:param input_proj_bias_attr: bias attribute for input-to-hidden projection. :param input_proj_bias_attr: bias attribute for input to hidden projection.
False means no bias, None means default bias. False means no bias, None means default bias.
:type input_proj_bias_attr: ParameterAttribute|False|None :type input_proj_bias_attr: ParameterAttribute|False|None
:param input_proj_layer_attr: extra layer attribute for input to hidden :param input_proj_layer_attr: extra layer attribute for input to hidden
...@@ -700,8 +704,8 @@ def lstmemory_unit(input, ...@@ -700,8 +704,8 @@ def lstmemory_unit(input,
:type input_proj_layer_attr: ExtraLayerAttribute :type input_proj_layer_attr: ExtraLayerAttribute
:param lstm_bias_attr: bias parameter attribute of lstm layer. :param lstm_bias_attr: bias parameter attribute of lstm layer.
False means no bias, None means default bias. False means no bias, None means default bias.
:type lstm_bias_attr: ParameterAttribute|False :type lstm_bias_attr: ParameterAttribute|False|None
:param lstm_layer_attr: lstm layer's extra attribute. :param lstm_layer_attr: extra attribute of lstm layer.
:type lstm_layer_attr: ExtraLayerAttribute :type lstm_layer_attr: ExtraLayerAttribute
:return: lstmemory unit name. :return: lstmemory unit name.
:rtype: LayerOutput :rtype: LayerOutput
...@@ -758,9 +762,9 @@ def lstmemory_group(input, ...@@ -758,9 +762,9 @@ def lstmemory_group(input,
lstm_group is a recurrent_group version of Long Short Term Memory. It lstm_group is a recurrent_group version of Long Short Term Memory. It
does exactly the same calculation as the lstmemory layer (see lstmemory in does exactly the same calculation as the lstmemory layer (see lstmemory in
layers.py for the maths) does. A promising benefit is that LSTM memory layers.py for the maths) does. A promising benefit is that LSTM memory
cell states, or hidden states in every time step are accessible to the cell states(or hidden states) in every time step are accessible to the
user. This is especially useful in attention model. If you do not need to user. This is especially useful in attention model. If you do not need to
access the internal states of the lstm, but merely use its outputs, access the internal states of the lstm and merely use its outputs,
it is recommended to use the lstmemory, which is relatively faster than it is recommended to use the lstmemory, which is relatively faster than
lstmemory_group. lstmemory_group.
...@@ -781,28 +785,28 @@ def lstmemory_group(input, ...@@ -781,28 +785,28 @@ def lstmemory_group(input,
gate_act=SigmoidActivation(), gate_act=SigmoidActivation(),
state_act=TanhActivation()) state_act=TanhActivation())
:param input: input layer name. :param input: input layer.
:type input: LayerOutput :type input: LayerOutput
:param size: lstmemory group size. :param size: lstmemory group size.
:type size: int :type size: int
:param name: name of the lstmemory group. :param name: name of lstmemory group.
:type name: basestring :type name: basestring
:param out_memory: output of previous time step :param out_memory: output of previous time step.
:type out_memory: LayerOutput | None :type out_memory: LayerOutput | None
:param reverse: is lstm reversed :param reverse: process the input in a reverse order or not.
:type reverse: bool :type reverse: bool
:param param_attr: Parameter config, None if use default. :param param_attr: parameter attribute, None means default attribute.
:type param_attr: ParameterAttribute :type param_attr: ParameterAttribute
:param act: lstm final activiation type :param act: last activiation type of lstm.
:type act: BaseActivation :type act: BaseActivation
:param gate_act: lstm gate activiation type :param gate_act: gate activiation type of lstm.
:type gate_act: BaseActivation :type gate_act: BaseActivation
:param state_act: lstm state activiation type. :param state_act: state activiation type of lstm.
:type state_act: BaseActivation :type state_act: BaseActivation
:param lstm_bias_attr: bias parameter attribute of lstm layer. :param lstm_bias_attr: bias parameter attribute of lstm layer.
False means no bias, None means default bias. False means no bias, None means default bias.
:type lstm_bias_attr: ParameterAttribute|False :type lstm_bias_attr: ParameterAttribute|False|None
:param input_proj_bias_attr: bias attribute for input-to-hidden projection. :param input_proj_bias_attr: bias attribute for input to hidden projection.
False means no bias, None means default bias. False means no bias, None means default bias.
:type input_proj_bias_attr: ParameterAttribute|False|None :type input_proj_bias_attr: ParameterAttribute|False|None
:param input_proj_layer_attr: extra layer attribute for input to hidden :param input_proj_layer_attr: extra layer attribute for input to hidden
...@@ -848,15 +852,15 @@ def gru_unit(input, ...@@ -848,15 +852,15 @@ def gru_unit(input,
gru_layer_attr=None, gru_layer_attr=None,
naive=False): naive=False):
""" """
Define calculations that a gated recurrent unit performs in a single time gru_unit defines the calculation process of a gated recurrent unit during a single
step. This function itself is not a recurrent layer, so it can not be time step. This function is not a recurrent layer, so it can not be
directly used to process sequence inputs. This function is always used in directly used to process sequence input. This function is always used in
the recurrent_group (see layers.py for more details) to implement attention the recurrent_group (see layers.py for more details) to implement attention
mechanism. mechanism.
Please see grumemory in layers.py for the details about the maths. Please see grumemory in layers.py for the details about the maths.
:param input: input layer name. :param input: input layer.
:type input: LayerOutput :type input: LayerOutput
:param memory_boot: the initialization state of the LSTM cell. :param memory_boot: the initialization state of the LSTM cell.
:type memory_boot: LayerOutput | None :type memory_boot: LayerOutput | None
...@@ -864,12 +868,12 @@ def gru_unit(input, ...@@ -864,12 +868,12 @@ def gru_unit(input,
:type name: basestring :type name: basestring
:param size: hidden size of the gru. :param size: hidden size of the gru.
:type size: int :type size: int
:param act: type of the activation :param act: activation type of gru
:type act: BaseActivation :type act: BaseActivation
:param gate_act: type of the gate activation :param gate_act: gate activation type or gru
:type gate_act: BaseActivation :type gate_act: BaseActivation
:param gru_layer_attr: Extra parameter attribute of the gru layer. :param gru_layer_attr: Extra attribute of the gru layer.
:type gru_layer_attr: ParameterAttribute|False :type gru_layer_attr: ExtraLayerAttribute
:return: the gru output layer. :return: the gru output layer.
:rtype: LayerOutput :rtype: LayerOutput
""" """
...@@ -915,7 +919,7 @@ def gru_group(input, ...@@ -915,7 +919,7 @@ def gru_group(input,
does exactly the same calculation as the grumemory layer does. A promising does exactly the same calculation as the grumemory layer does. A promising
benefit is that gru hidden states are accessible to the user. This is benefit is that gru hidden states are accessible to the user. This is
especially useful in attention model. If you do not need to access especially useful in attention model. If you do not need to access
any internal state, but merely use the outputs of a GRU, it is recommended any internal state and merely use the outputs of a GRU, it is recommended
to use the grumemory, which is relatively faster. to use the grumemory, which is relatively faster.
Please see grumemory in layers.py for more detail about the maths. Please see grumemory in layers.py for more detail about the maths.
...@@ -924,12 +928,12 @@ def gru_group(input, ...@@ -924,12 +928,12 @@ def gru_group(input,
.. code-block:: python .. code-block:: python
gru = gur_group(input=[layer1], gru = gru_group(input=[layer1],
size=256, size=256,
act=TanhActivation(), act=TanhActivation(),
gate_act=SigmoidActivation()) gate_act=SigmoidActivation())
:param input: input layer name. :param input: input layer.
:type input: LayerOutput :type input: LayerOutput
:param memory_boot: the initialization state of the LSTM cell. :param memory_boot: the initialization state of the LSTM cell.
:type memory_boot: LayerOutput | None :type memory_boot: LayerOutput | None
...@@ -937,16 +941,17 @@ def gru_group(input, ...@@ -937,16 +941,17 @@ def gru_group(input,
:type name: basestring :type name: basestring
:param size: hidden size of the gru. :param size: hidden size of the gru.
:type size: int :type size: int
:param reverse: whether to process the input data in a reverse order :param reverse: process the input in a reverse order or not.
:type reverse: bool :type reverse: bool
:param act: type of the activiation :param act: activiation type of gru
:type act: BaseActivation :type act: BaseActivation
:param gate_act: type of the gate activiation :param gate_act: gate activiation type of gru
:type gate_act: BaseActivation :type gate_act: BaseActivation
:param gru_bias_attr: bias. False means no bias, None means default bias. :param gru_bias_attr: bias parameter attribute of gru layer,
:type gru_bias_attr: ParameterAttribute|False False means no bias, None means default bias.
:param gru_layer_attr: Extra parameter attribute of the gru layer. :type gru_bias_attr: ParameterAttribute|False|None
:type gru_layer_attr: ParameterAttribute|False :param gru_layer_attr: Extra attribute of the gru layer.
:type gru_layer_attr: ExtraLayerAttribute
:return: the gru group. :return: the gru group.
:rtype: LayerOutput :rtype: LayerOutput
""" """
...@@ -986,11 +991,11 @@ def simple_gru(input, ...@@ -986,11 +991,11 @@ def simple_gru(input,
gru_layer_attr=None, gru_layer_attr=None,
naive=False): naive=False):
""" """
You maybe see gru_step_layer, grumemory in layers.py, gru_unit, gru_group, You may see gru_step_layer, grumemory in layers.py, gru_unit, gru_group,
simple_gru in network.py. The reason why there are so many interfaces is simple_gru in network.py. The reason why there are so many interfaces is
that we have two ways to implement recurrent neural network. One way is to that we have two ways to implement recurrent neural network. One way is to
use one complete layer to implement rnn (including simple rnn, gru and lstm) use one complete layer to implement rnn (including simple rnn, gru and lstm)
with multiple time steps, such as recurrent_layer, lstmemory, grumemory. But, with multiple time steps, such as recurrent_layer, lstmemory, grumemory. But
the multiplication operation :math:`W x_t` is not computed in these layers. the multiplication operation :math:`W x_t` is not computed in these layers.
See details in their interfaces in layers.py. See details in their interfaces in layers.py.
The other implementation is to use an recurrent group which can ensemble a The other implementation is to use an recurrent group which can ensemble a
...@@ -1018,22 +1023,23 @@ def simple_gru(input, ...@@ -1018,22 +1023,23 @@ def simple_gru(input,
gru = simple_gru(input=[layer1], size=256) gru = simple_gru(input=[layer1], size=256)
:param input: input layer name. :param input: input layer.
:type input: LayerOutput :type input: LayerOutput
:param name: name of the gru group. :param name: name of the gru group.
:type name: basestring :type name: basestring
:param size: hidden size of the gru. :param size: hidden size of the gru.
:type size: int :type size: int
:param reverse: whether to process the input data in a reverse order :param reverse: process the input in a reverse order or not.
:type reverse: bool :type reverse: bool
:param act: type of the activiation :param act: activiation type of gru
:type act: BaseActivation :type act: BaseActivation
:param gate_act: type of the gate activiation :param gate_act: gate activiation type of gru
:type gate_act: BaseActivation :type gate_act: BaseActivation
:param gru_bias_attr: bias. False means no bias, None means default bias. :param gru_bias_attr: bias parameter attribute of gru layer,
:type gru_bias_attr: ParameterAttribute|False False means no bias, None means default bias.
:param gru_layer_attr: Extra parameter attribute of the gru layer. :type gru_bias_attr: ParameterAttribute|False|None
:type gru_layer_attr: ParameterAttribute|False :param gru_layer_attr: Extra attribute of the gru layer.
:type gru_layer_attr: ExtraLayerAttribute
:return: the gru group. :return: the gru group.
:rtype: LayerOutput :rtype: LayerOutput
""" """
...@@ -1071,8 +1077,8 @@ def simple_gru2(input, ...@@ -1071,8 +1077,8 @@ def simple_gru2(input,
mixed_layer_attr=None, mixed_layer_attr=None,
gru_cell_attr=None): gru_cell_attr=None):
""" """
simple_gru2 is the same with simple_gru, but using grumemory instead simple_gru2 is the same with simple_gru, but using grumemory instead.
Please see grumemory in layers.py for more detail about the maths. Please refer to grumemory in layers.py for more detail about the math.
simple_gru2 is faster than simple_gru. simple_gru2 is faster than simple_gru.
The example usage is: The example usage is:
...@@ -1081,22 +1087,23 @@ def simple_gru2(input, ...@@ -1081,22 +1087,23 @@ def simple_gru2(input,
gru = simple_gru2(input=[layer1], size=256) gru = simple_gru2(input=[layer1], size=256)
:param input: input layer name. :param input: input layer.
:type input: LayerOutput :type input: LayerOutput
:param name: name of the gru group. :param name: name of the gru group.
:type name: basestring :type name: basestring
:param size: hidden size of the gru. :param size: hidden size of the gru.
:type size: int :type size: int
:param reverse: whether to process the input data in a reverse order :param reverse: process the input in a reverse order or not.
:type reverse: bool :type reverse: bool
:param act: type of the activiation :param act: activiation type of gru
:type act: BaseActivation :type act: BaseActivation
:param gate_act: type of the gate activiation :param gate_act: gate activiation type of gru
:type gate_act: BaseActivation :type gate_act: BaseActivation
:param gru_bias_attr: bias. False means no bias, None means default bias. :param gru_bias_attr: bias parameter attribute of gru layer,
:type gru_bias_attr: ParameterAttribute|False False means no bias, None means default bias.
:param gru_layer_attr: Extra parameter attribute of the gru layer. :type gru_bias_attr: ParameterAttribute|False|None
:type gru_layer_attr: ParameterAttribute|False :param gru_layer_attr: Extra attribute of the gru layer.
:type gru_layer_attr: ExtraLayerAttribute
:return: the gru group. :return: the gru group.
:rtype: LayerOutput :rtype: LayerOutput
""" """
...@@ -1145,7 +1152,7 @@ def bidirectional_gru(input, ...@@ -1145,7 +1152,7 @@ def bidirectional_gru(input,
concat_act=None): concat_act=None):
""" """
A bidirectional_gru is a recurrent unit that iterates over the input A bidirectional_gru is a recurrent unit that iterates over the input
sequence both in forward and bardward orders, and then concatenate two sequence both in forward and backward orders, and then concatenate two
outputs to form a final output. However, concatenation of two outputs outputs to form a final output. However, concatenation of two outputs
is not the only way to form the final output, you can also, for example, is not the only way to form the final output, you can also, for example,
just add them together. just add them together.
...@@ -1162,11 +1169,10 @@ def bidirectional_gru(input, ...@@ -1162,11 +1169,10 @@ def bidirectional_gru(input,
:type input: LayerOutput :type input: LayerOutput
:param size: gru layer size. :param size: gru layer size.
:type size: int :type size: int
:param return_seq: If set False, outputs of the last time step are :param return_seq: If set False, the last time step of output are
concatenated and returned.
If set True, the entire output sequences that are
processed in forward and backward directions are
concatenated and returned. concatenated and returned.
If set True, the entire output sequences in forward
and backward directions are concatenated and returned.
:type return_seq: bool :type return_seq: bool
:return: LayerOutput object. :return: LayerOutput object.
:rtype: LayerOutput :rtype: LayerOutput
...@@ -1230,8 +1236,8 @@ def bidirectional_lstm(input, ...@@ -1230,8 +1236,8 @@ def bidirectional_lstm(input,
concat_act=None): concat_act=None):
""" """
A bidirectional_lstm is a recurrent unit that iterates over the input A bidirectional_lstm is a recurrent unit that iterates over the input
sequence both in forward and bardward orders, and then concatenate two sequence both in forward and backward orders, and then concatenate two
outputs form a final output. However, concatenation of two outputs outputs to form a final output. However, concatenation of two outputs
is not the only way to form the final output, you can also, for example, is not the only way to form the final output, you can also, for example,
just add them together. just add them together.
...@@ -1252,13 +1258,12 @@ def bidirectional_lstm(input, ...@@ -1252,13 +1258,12 @@ def bidirectional_lstm(input,
:type input: LayerOutput :type input: LayerOutput
:param size: lstm layer size. :param size: lstm layer size.
:type size: int :type size: int
:param return_seq: If set False, outputs of the last time step are :param return_seq: If set False, the last time step of output are
concatenated and returned.
If set True, the entire output sequences that are
processed in forward and backward directions are
concatenated and returned. concatenated and returned.
If set True, the entire output sequences in forward
and backward directions are concatenated and returned.
:type return_seq: bool :type return_seq: bool
:return: LayerOutput object accroding to the return_seq. :return: LayerOutput object.
:rtype: LayerOutput :rtype: LayerOutput
""" """
args = locals() args = locals()
...@@ -1303,7 +1308,7 @@ def simple_attention(encoded_sequence, ...@@ -1303,7 +1308,7 @@ def simple_attention(encoded_sequence,
weight_act=None, weight_act=None,
name=None): name=None):
""" """
Calculate and then return a context vector by attention machanism. Calculate and return a context vector with attention mechanism.
Size of the context vector equals to size of the encoded_sequence. Size of the context vector equals to size of the encoded_sequence.
.. math:: .. math::
...@@ -1336,10 +1341,10 @@ def simple_attention(encoded_sequence, ...@@ -1336,10 +1341,10 @@ def simple_attention(encoded_sequence,
:param name: name of the attention model. :param name: name of the attention model.
:type name: basestring :type name: basestring
:param softmax_param_attr: parameter attribute of sequence softmax :param softmax_param_attr: parameter attribute of sequence softmax
that is used to produce attention weight that is used to produce attention weight.
:type softmax_param_attr: ParameterAttribute :type softmax_param_attr: ParameterAttribute
:param weight_act: activation of the attention model :param weight_act: activation of the attention model.
:type weight_act: Activation :type weight_act: BaseActivation
:param encoded_sequence: output of the encoder :param encoded_sequence: output of the encoder
:type encoded_sequence: LayerOutput :type encoded_sequence: LayerOutput
:param encoded_proj: attention weight is computed by a feed forward neural :param encoded_proj: attention weight is computed by a feed forward neural
...@@ -1411,7 +1416,7 @@ def inputs(layers, *args): ...@@ -1411,7 +1416,7 @@ def inputs(layers, *args):
def outputs(layers, *args): def outputs(layers, *args):
""" """
Declare the outputs of network. If user have not defined the inputs of Declare the outputs of network. If user has not defined the inputs of
network, this method will calculate the input order by dfs travel. network, this method will calculate the input order by dfs travel.
:param layers: Output layers. :param layers: Output layers.
......
...@@ -47,17 +47,24 @@ def set_input(scope, op, inputs, place): ...@@ -47,17 +47,24 @@ def set_input(scope, op, inputs, place):
if in_name in inputs: if in_name in inputs:
if in_dup: if in_dup:
sub_in = inputs[in_name] sub_in = inputs[in_name]
for sub_in_name, sub_in_array in sub_in: for sub_in_name, sub_in_val in sub_in:
var = scope.find_var(sub_in_name) var = scope.find_var(sub_in_name)
tensor = var.get_tensor() tensor = var.get_tensor()
sub_in_array = sub_in_val[0] \
if isinstance(sub_in_val, tuple) else sub_in_val
tensor.set_dims(sub_in_array.shape) tensor.set_dims(sub_in_array.shape)
tensor.set(sub_in_array, place) tensor.set(sub_in_array, place)
if isinstance(sub_in_val, tuple):
tensor.set_lod(sub_in_val[1])
else: else:
var = scope.find_var(in_name) var = scope.find_var(in_name)
tensor = var.get_tensor() tensor = var.get_tensor()
arr = inputs[in_name] in_val = inputs[in_name]
tensor.set_dims(arr.shape) in_array = in_val[0] if isinstance(in_val, tuple) else in_val
tensor.set(arr, place) tensor.set_dims(in_array.shape)
tensor.set(in_array, place)
if isinstance(in_val, tuple):
tensor.set_lod(in_val[1])
def set_output_grad(scope, op, outputs, place): def set_output_grad(scope, op, outputs, place):
......
...@@ -106,7 +106,7 @@ class TestBRelu(OpTest): ...@@ -106,7 +106,7 @@ class TestBRelu(OpTest):
t_max = 4 t_max = 4
# The same with TestAbs # The same with TestAbs
x[np.abs(x - t_min) < 0.005] = t_min + 0.02 x[np.abs(x - t_min) < 0.005] = t_min + 0.02
x[np.abs(x - t_max) < 0.005] = t_min + 0.02 x[np.abs(x - t_max) < 0.005] = t_max + 0.02
self.inputs = {'X': x} self.inputs = {'X': x}
self.attrs = {'t_min': t_min, 't_max': t_max} self.attrs = {'t_min': t_min, 't_max': t_max}
......
...@@ -4,7 +4,7 @@ from paddle.v2.framework.op import Operator ...@@ -4,7 +4,7 @@ from paddle.v2.framework.op import Operator
import numpy import numpy
class GaussianRandomTest(unittest.TestCase): class TestGaussianRandomOp(unittest.TestCase):
def test_cpu(self): def test_cpu(self):
self.gaussian_random_test(place=core.CPUPlace()) self.gaussian_random_test(place=core.CPUPlace())
......
import unittest
import numpy as np
from op_test import OpTest
class TestIdentityOp(OpTest):
def setUp(self):
self.op_type = "identity"
self.inputs = {'X': np.random.random((10, 10)).astype("float32")}
self.outputs = {'Out': self.inputs['X']}
def test_check_output(self):
self.check_output()
def test_check_grad(self):
self.check_grad(['X'], 'Out')
if __name__ == "__main__":
unittest.main()
...@@ -3,7 +3,7 @@ import numpy as np ...@@ -3,7 +3,7 @@ import numpy as np
from op_test import OpTest from op_test import OpTest
class MinusOpTest(OpTest): class TestMinusOp(OpTest):
def setUp(self): def setUp(self):
self.op_type = "minus" self.op_type = "minus"
self.inputs = { self.inputs = {
......
...@@ -3,7 +3,7 @@ import numpy ...@@ -3,7 +3,7 @@ import numpy
from op_test import OpTest from op_test import OpTest
class TestCrossEntropy(OpTest): class TestOnehotCrossEntropyOp(OpTest):
def setUp(self): def setUp(self):
self.op_type = "onehot_cross_entropy" self.op_type = "onehot_cross_entropy"
batch_size = 30 batch_size = 30
......
...@@ -3,20 +3,7 @@ import numpy as np ...@@ -3,20 +3,7 @@ import numpy as np
from op_test import OpTest from op_test import OpTest
class IdentityTest(OpTest): class TestScaleOp(OpTest):
def setUp(self):
self.op_type = "identity"
self.inputs = {'X': np.random.random((10, 10)).astype("float32")}
self.outputs = {'Out': self.inputs['X']}
def test_check_output(self):
self.check_output()
def test_check_grad(self):
self.check_grad(['X'], 'Out')
class ScaleTest(OpTest):
def setUp(self): def setUp(self):
self.op_type = "scale" self.op_type = "scale"
self.inputs = {'X': np.random.random((10, 10)).astype("float32")} self.inputs = {'X': np.random.random((10, 10)).astype("float32")}
......
import unittest
import numpy as np
from op_test import OpTest
class TestSeqAvgPool1D(OpTest):
def setUp(self):
self.op_type = 'sequence_avg_pool'
# one level, batch size is 4
x = np.random.uniform(0.1, 1, [11, 23]).astype('float32')
lod = [[0, 4, 5, 8, 11]]
out = np.zeros((4, 23)).astype('float32')
for i in range(4):
sub_x = x[lod[0][i]:lod[0][i + 1], :]
out[i] = sub_x.mean(axis=0)
self.inputs = {'X': (x, lod)}
self.outputs = {'Out': out}
def test_check_output(self):
self.check_output()
def test_check_grad(self):
self.check_grad(["X"], "Out")
class TestSeqAvgPool2D(OpTest):
def setUp(self):
self.op_type = 'sequence_avg_pool'
# one level, batch size is 4
x = np.random.uniform(0.1, 1, [13, 3, 17]).astype('float32')
lod = [[0, 4, 5, 8, 13]]
out = np.zeros((4, 3, 17)).astype('float32')
for i in range(4):
sub_x = np.reshape(x[lod[0][i]:lod[0][i + 1], :], (-1, 3 * 17))
out[i] = np.reshape(sub_x.mean(axis=0), (3, 17))
self.inputs = {'X': (x, lod)}
self.outputs = {'Out': out}
def test_check_output(self):
self.check_output()
def test_check_grad(self):
self.check_grad(["X"], "Out")
if __name__ == '__main__':
unittest.main()
...@@ -3,7 +3,7 @@ import numpy as np ...@@ -3,7 +3,7 @@ import numpy as np
from op_test import OpTest from op_test import OpTest
class TestSGD(OpTest): class TestSGDOp(OpTest):
def setUp(self): def setUp(self):
self.op_type = "sgd" self.op_type = "sgd"
w = np.random.random((102, 105)).astype("float32") w = np.random.random((102, 105)).astype("float32")
......
...@@ -21,6 +21,9 @@ class TestTopkOp(OpTest): ...@@ -21,6 +21,9 @@ class TestTopkOp(OpTest):
self.outputs = {'Out': output, 'Indices': indices} self.outputs = {'Out': output, 'Indices': indices}
def test_check_output(self):
self.check_output()
class TestTopkOp3d(OpTest): class TestTopkOp3d(OpTest):
def setUp(self): def setUp(self):
...@@ -42,6 +45,9 @@ class TestTopkOp3d(OpTest): ...@@ -42,6 +45,9 @@ class TestTopkOp3d(OpTest):
self.outputs = {'Out': output, 'Indices': indices} self.outputs = {'Out': output, 'Indices': indices}
def test_check_output(self):
self.check_output()
if __name__ == "__main__": if __name__ == "__main__":
unittest.main() unittest.main()
...@@ -4,7 +4,7 @@ import paddle.v2.framework.core as core ...@@ -4,7 +4,7 @@ import paddle.v2.framework.core as core
import numpy import numpy
class UniformRandomTest(unittest.TestCase): class TestUniformRandomOp(unittest.TestCase):
def test_uniform_random_cpu(self): def test_uniform_random_cpu(self):
self.uniform_random_test(place=core.CPUPlace()) self.uniform_random_test(place=core.CPUPlace())
......
Markdown is supported
0% .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑!
想要评论请 注册