Skip to content

  • 体验新版
    • 正在加载...
  • 登录
  • PaddlePaddle
  • Paddle
  • Issue
  • #3129

P
Paddle
  • 项目概览

PaddlePaddle / Paddle
大约 2 年 前同步成功

通知 2325
Star 20933
Fork 5424
  • 代码
    • 文件
    • 提交
    • 分支
    • Tags
    • 贡献者
    • 分支图
    • Diff
  • Issue 1423
    • 列表
    • 看板
    • 标记
    • 里程碑
  • 合并请求 543
  • Wiki 0
    • Wiki
  • 分析
    • 仓库
    • DevOps
  • 项目成员
  • Pages
P
Paddle
  • 项目概览
    • 项目概览
    • 详情
    • 发布
  • 仓库
    • 仓库
    • 文件
    • 提交
    • 分支
    • 标签
    • 贡献者
    • 分支图
    • 比较
  • Issue 1,423
    • Issue 1,423
    • 列表
    • 看板
    • 标记
    • 里程碑
  • 合并请求 543
    • 合并请求 543
  • Pages
  • 分析
    • 分析
    • 仓库分析
    • DevOps
  • Wiki 0
    • Wiki
  • 成员
    • 成员
  • 收起侧边栏
  • 动态
  • 分支图
  • 创建新Issue
  • 提交
  • Issue看板
You need to sign in or sign up before continuing.
已关闭
开放中
Opened 8月 01, 2017 by saxon_zh@saxon_zhGuest

Python v2 API to new operator framework

Created by: wangkuiyi

Design Doc: RNNOp

A Plain Network

predict = paddle.layer.fc(
      paddle.layer.data(name="x"),
      output_size = 100)
      
cost = paddle.layer.mse(
    predict,
    paddle.layer.data(name="y"))
    
parameters = paddle.train(cost)
paddle.save_model(cost, paddle.datasets.mnist.train(), parameters, "filename");

p = paddle.load_model(predict, "filename")
paddle.infer(predict, 

Layers, Variables, and Default Scope

# in package paddle.layer
def data(name):
  return paddle.cpp.variable(paddle.cpp.default_scope(), name)

def fc(input, output_size):
  output = paddle.cpp.variable(paddle.cpp.default_scope())
  W = paddle.cpp.variable(paddle.cpp.default_scope(), label="parameter")
  b = paddle.cpp.variable(paddle.cpp.default_scope(), label="parameter")
  paddle.cpp.operator("FC", read={input, W, b}, output_size, write={output})
  return output
  
def mse(input1, input2):
  output = paddle.cpp.variable(paddle.cpp.default_scope())
  paddle.cpp.operator("MSE", read={input1, input2}, write={output})
  return output

where

  • paddle.cpp.variable is a Python binding of C++ method Scope::NewVar().
  • paddle.cpp.operator creates an operator and mark it as a reader of some variable and a writer of some others. We will cover this later in more details.

paddle::operator::Net

paddle.train receives a variable created by paddle.layer.mse and need to trace all related operators and sort them by the topological order.

Please be aware that all operators are derived from class OperatorBase, which refers to Variables by their names:

class OperatorBase:
  vector<string> inputs_;
  vector<string> outputs_;
};

and Variables doesn't have names if they are not in a Scope.

Also, each Varaible maintains:

class Variable {
  list<Operator*> readers_;
  list<Operator*> writers_;
};

Please be aware the trace from an operator to its input variables depends on the default scope. The tracing is done in C++ space, so paddle.cpp.default_scope is a binding to C++ code.

class Net {
 public:
  static Net* TraceAndBuild(Variable* output, Scope* scope) {
    std::list<std::pair<Operaor*, int/*distance to output*/> > dists;
    std::list<std::pair<Variable*, int /*distance to output*/> > frontier;
    frontier.push_back(make_pair<output, 0>);

    while (frontier.size() > 0) {
      Variable* v = frontier.front().first;
      int dist = frontier.front().second;
      frontier.pop_front();
      
      for (Operator* o : v->writers_) {
        dists.push_back(make_pair(v, dist));
        for (const string& s : o->writers_) {
          frontier.push_back(make_pair(scope->FindVar(s), dist+1));
        }
      }
    }
    
    std::sort(dists, /*by the descending order of dist*/);
    
    return new Net(dists); 
 }
};

We can call

Net::TraceAndBuild(output_variable, DefaultScope()).Run(DefaultScope());

to extract the network using the default scope and run it.

Scope Hierarchy

An RNN operator may have kinds of variables:

  1. global variable -- in outer scope
  2. memory variable -- in RNNOp-local scope
  3. local variable -- in step-local scope
   outer scope
      /|\
       |
   RNNOp scope 
(the memory over steps)
  /|\  /|\   /|\
   |    |     |
step-0 step-1 step-2
scope  scope  scope

Just like what a programing language compiler/interpreter would do, for each step, there is a step-local scope, but there is only one copy of compiled code (binary code) or step-net in our case.

Above three tiers can be simplified to two-tier by moving memory variables to the outer scope, but this is not necessary.

outer scope (including all memory variables of an RNNOp)
  /|\  /|\   /|\
   |    |     |
step-0 step-1 step-2
scope  scope  scope

A Recurrent Network

x = paddle.layer.data(name="features")
y = paddle.layer.data(name="labels")

accum = paddle.framework.tensor()

cost = paddle.layer.mse(
  paddle.layer.fc(
    paddle.layer.rnn(
      input = paddle.layer.fc(x),
      step_net = paddle.layer.fc(
                   paddle.layer.add_to(accum, NULL), 
                   output_size=100),
      concat_output=true)),
  y)

paddle.train(cost, ...)

Here we use NULL as the placeholder of the input of the step net.

Please notice that we don't have to consume the output of an RNNOp. For example, we can use the memory as the RNNOp's output:

x = paddle.layer.data(name="features")
y = paddle.layer.data(name="labels")

memory = paddle.framework.tensor()

paddle.layer.rnn(
  input = paddle.layer.fc(x),
  step_net = paddle.layer.fc(
               paddle.layer.add_to(memory, NULL), 
               output_size=100),
  concat_output=true)

cost = paddle.layer.mse(paddle.layer.fc(memory), y)

paddle.train(cost, ...)

Step-Net

Above example shows that the step_net parameter of paddle.layer.rnn accepts a variable returned by paddle.layer.fc. We need to trace the step-net from this variable. This can be done by calling the aforementined paddle::operator::Net::TraceAndBuild

namespace paddle { 
namespace operator {
class RNN {
 public:
  void Run(Scope* scope) {
    RNNInput* whole_input = inputs_[0]->Get<RNNInput>();
    int sequence_len = whole_input->Len(0);
    for (int i = 0; i < sequence_len; ++i) {
      Scope* step_scope = scope->NewScope();
      step_scope->NewVar("step_input")->GetMutable<Tensor>()->Slice(whole_input, i);
      Net* net = Net::TraceAndBuild(GetAttr<Variable*>("step_net"), step_scope);
      net->Run(step_scope)
    }
  }
};
指派人
分配到
无
里程碑
无
分配里程碑
工时统计
无
截止日期
无
标识: paddlepaddle/Paddle#3129
渝ICP备2023009037号

京公网安备11010502055752号

网络110报警服务 Powered by GitLab CE v13.7
开源知识
Git 入门 Pro Git 电子书 在线学 Git
Markdown 基础入门 IT 技术知识开源图谱
帮助
使用手册 反馈建议 博客
《GitCode 隐私声明》 《GitCode 服务条款》 关于GitCode
Powered by GitLab CE v13.7