Skip to content
体验新版
项目
组织
正在加载...
登录
切换导航
打开侧边栏
机器未来
Paddle
提交
d6c8d267
P
Paddle
项目概览
机器未来
/
Paddle
与 Fork 源项目一致
Fork自
PaddlePaddle / Paddle
通知
1
Star
1
Fork
0
代码
文件
提交
分支
Tags
贡献者
分支图
Diff
Issue
1
列表
看板
标记
里程碑
合并请求
0
Wiki
0
Wiki
分析
仓库
DevOps
项目成员
Pages
P
Paddle
项目概览
项目概览
详情
发布
仓库
仓库
文件
提交
分支
标签
贡献者
分支图
比较
Issue
1
Issue
1
列表
看板
标记
里程碑
合并请求
0
合并请求
0
Pages
分析
分析
仓库分析
DevOps
Wiki
0
Wiki
成员
成员
收起侧边栏
关闭侧边栏
动态
分支图
创建新Issue
提交
Issue看板
提交
d6c8d267
编写于
6月 12, 2018
作者:
Q
qiaolongfei
浏览文件
操作
浏览文件
下载
电子邮件补丁
差异文件
optimize code and comment
上级
f031555c
变更
2
隐藏空白更改
内联
并排
Showing
2 changed file
with
11 addition
and
9 deletion
+11
-9
paddle/fluid/operators/merge_ids_op.cc
paddle/fluid/operators/merge_ids_op.cc
+6
-4
paddle/fluid/operators/merge_ids_op.h
paddle/fluid/operators/merge_ids_op.h
+5
-5
未找到文件。
paddle/fluid/operators/merge_ids_op.cc
浏览文件 @
d6c8d267
...
...
@@ -21,15 +21,17 @@ class MergeIdsOpMaker : public framework::OpProtoAndCheckerMaker {
public:
void
Make
()
override
{
AddInput
(
"Ids"
,
"(LoDTensor) the input ids with shape{batch_num, 1}"
);
AddInput
(
"X"
,
"(LoDTensor) the input tensor with shape{batch_num, N}, N is the "
"size of embedding table"
)
AddInput
(
"X"
,
"(LoDTensors) multi input tensor with shape{batch_num, N}, N is the "
"size of embedding table"
)
.
AsDuplicable
();
AddOutput
(
"Out"
,
"(LoDTensor) The merged outputs of the input tensors."
);
AddComment
(
R"DOC(
Merge multi LoDTensor's into one according to Ids's shard num.
The values in the input LoDTensor are lookuped from the output of splite_ids_op
The values in the input LoDTensor are lookuped from the output of split_ids_op
Example:
Input:
Ids = [1,2,3,4,5,6]
...
...
paddle/fluid/operators/merge_ids_op.h
浏览文件 @
d6c8d267
...
...
@@ -47,10 +47,10 @@ class MergeIdsOpKernel : public framework::OpKernel<T> {
int
batch_size
=
0
;
int
embedding_size
=
0
;
for
(
auto
&
input
:
x_tensors
)
{
if
(
embedding_size
==
0
)
{
embedding_size
=
input
->
dims
()[
1
];
}
if
(
framework
::
product
(
input
->
dims
())
!=
0
)
{
if
(
embedding_size
==
0
)
{
embedding_size
=
input
->
dims
()[
1
];
}
PADDLE_ENFORCE_EQ
(
embedding_size
,
input
->
dims
()[
1
],
"embedding size of all input should be the same"
);
batch_size
+=
input
->
dims
()[
0
];
...
...
@@ -58,7 +58,7 @@ class MergeIdsOpKernel : public framework::OpKernel<T> {
}
PADDLE_ENFORCE_EQ
(
batch_size
,
ids_dims
[
0
],
"the batch size of ids and embedding value should be the same"
);
"the batch size of ids and
merged
embedding value should be the same"
);
const
size_t
shard_num
=
x_tensors
.
size
();
...
...
@@ -80,7 +80,7 @@ class MergeIdsOpKernel : public framework::OpKernel<T> {
in_indexs
[
shard_id
]
+=
1
;
}
for
(
in
t
i
=
0
;
i
<
shard_num
;
++
i
)
{
for
(
size_
t
i
=
0
;
i
<
shard_num
;
++
i
)
{
PADDLE_ENFORCE_EQ
(
in_indexs
[
i
],
x_tensors
[
i
]
->
dims
()[
0
],
"after merge, all data in x_tensor should be used"
);
}
...
...
编辑
预览
Markdown
is supported
0%
请重试
或
添加新附件
.
添加附件
取消
You are about to add
0
people
to the discussion. Proceed with caution.
先完成此消息的编辑!
取消
想要评论请
注册
或
登录