Skip to content
体验新版
项目
组织
正在加载...
登录
切换导航
打开侧边栏
wzpei9
Paddle
提交
dd229dc7
P
Paddle
项目概览
wzpei9
/
Paddle
与 Fork 源项目一致
Fork自
PaddlePaddle / Paddle
通知
1
Star
1
Fork
0
代码
文件
提交
分支
Tags
贡献者
分支图
Diff
Issue
1
列表
看板
标记
里程碑
合并请求
0
Wiki
0
Wiki
分析
仓库
DevOps
项目成员
Pages
P
Paddle
项目概览
项目概览
详情
发布
仓库
仓库
文件
提交
分支
标签
贡献者
分支图
比较
Issue
1
Issue
1
列表
看板
标记
里程碑
合并请求
0
合并请求
0
Pages
分析
分析
仓库分析
DevOps
Wiki
0
Wiki
成员
成员
收起侧边栏
关闭侧边栏
动态
分支图
创建新Issue
提交
Issue看板
体验新版 GitCode,发现更多精彩内容 >>
提交
dd229dc7
编写于
2月 09, 2017
作者:
Y
Yi Wang
浏览文件
操作
浏览文件
下载
电子邮件补丁
差异文件
Update api.md
上级
36036c0e
变更
1
显示空白变更内容
内联
并排
Showing
1 changed file
with
90 addition
and
38 deletion
+90
-38
doc/design/api.md
doc/design/api.md
+90
-38
未找到文件。
doc/design/api.md
浏览文件 @
dd229dc7
import yi_json
g = 100
def read():
queue q;
# warmup q
for i = 0 : 1000
q.push(read())
yield q.shuffle_get()
input = paddle.layer.data(...)
intermediate = paddle.layers.fc(input)
output = paddle.layer.softmax(intermediate)
model = paddle.model.create(output)
train(model, data_provider=read, cluster="clusterId")
#--------------------------------------------------------------------------------
# 1. package, docker build, docker push
# 2. kubectl, clusterId Kuberentes job, 10 trainer containers, 5 parameter server containers
#--------------------------------------------------------------------------------
def train():
if os.environ["kube_api_server"] == nil:
docker_build()
docker_push()
kube_ctrl_start_job()
else:
rank = kube_mpi_rank()
if rank == 0:
master()
elif rank >= 15:
parameter_server()
else:
_train()
# Design Doc: PaddlePaddle API
## Ingredients
As the first step of our design, we list important concepts in deep
learning and try to figure their relationship, as shown below:
```
Model = {topology, parameters}
Evaluator = {Model*, activations}
- forward
- test
GradientMachine = {Model*, gradients}
- backward
Optimizer = {Model*, Evaluator*, GradientMachine*}
- train
- update
- checkpoint
```
where the pair of curly braces
`{`
and
`}`
indicate
*composition*
,
`*`
indicates a
*reference*
, and
`-`
marks a "class method".
### Model
We used to think that parameters are part of the toplogy (or layers).
But that is not true, because multiple layers could share the same
parameter matrix. An example is a network that compares two text
segments in a semantic space:
```
semantic
text A -> projection ---\
layer A \
cosine
similarity -> output
layer
semantic /
text B -> projection ---/
layer B
```
In this network, the two semantic projection layers (A and B) share
the same parameter matrix.
For more information about our API that specifies topology and
parameter sharing, please refer to [TODO: API].
### Evaluator
Supposed that we have a trained ranking model, we should be able to
use it in our search engine. The search engine's Web server is a
concurrent program so to serve many HTTP requests simultaneously. It
doens't make sense for each of these threads to have its own copy of
model, because that would duplicate topologies and parameters.
However, each thread should be able to record layer outputs, i.e.,
activations, computed from an input, derived from the request. With
*Evaluator*
that saves activations, we can write the over-simplified
server program as:
```
python
m
=
paddle
.
model
.
load
(
"trained.model"
)
http
.
handle
(
"/"
,
lambda
req
:
e
=
paddle
.
evaluator
.
create
(
m
)
e
.
forward
(
req
)
e
.
activation
(
layer
=
"output"
))
# returns activations of layer "output"
```
### GradientMachine
Similar to the evaluation, the training needs to compute gradients so
to update model parameters. Because an
[
optimizer
](
#optimizer
)
might
run multiple simultaneous threads to update the same model, gradients
should be separated from the model. Because gradients are only used
in training, but not serving, they should be separate from Evaluator.
Hence the
`GradientMachine`
.
### Optimizer
None of Model, Evaluator, nor GradientMachine implements the training
loop, hence Optimizer. We can define a concurrent optimizer that runs
multiple simultaneious threads to train a model -- just let each
thread has its own GradientMachine object.
编辑
预览
Markdown
is supported
0%
请重试
或
添加新附件
.
添加附件
取消
You are about to add
0
people
to the discussion. Proceed with caution.
先完成此消息的编辑!
取消
想要评论请
注册
或
登录