Skip to content
体验新版
项目
组织
正在加载...
登录
切换导航
打开侧边栏
wmsofts
Paddle
提交
08c25995
P
Paddle
项目概览
wmsofts
/
Paddle
与 Fork 源项目一致
Fork自
PaddlePaddle / Paddle
通知
1
Star
0
Fork
0
代码
文件
提交
分支
Tags
贡献者
分支图
Diff
Issue
0
列表
看板
标记
里程碑
合并请求
0
Wiki
0
Wiki
分析
仓库
DevOps
项目成员
Pages
P
Paddle
项目概览
项目概览
详情
发布
仓库
仓库
文件
提交
分支
标签
贡献者
分支图
比较
Issue
0
Issue
0
列表
看板
标记
里程碑
合并请求
0
合并请求
0
Pages
分析
分析
仓库分析
DevOps
Wiki
0
Wiki
成员
成员
收起侧边栏
关闭侧边栏
动态
分支图
创建新Issue
提交
Issue看板
体验新版 GitCode,发现更多精彩内容 >>
提交
08c25995
编写于
3月 06, 2019
作者:
D
dongdaxiang
浏览文件
操作
浏览文件
下载
电子邮件补丁
差异文件
add run from dataset in executor.
上级
c28bbdf8
变更
2
隐藏空白更改
内联
并排
Showing
2 changed file
with
32 addition
and
10 deletion
+32
-10
paddle/fluid/framework/executor.h
paddle/fluid/framework/executor.h
+13
-0
paddle/fluid/framework/multi_trainer.cc
paddle/fluid/framework/multi_trainer.cc
+19
-10
未找到文件。
paddle/fluid/framework/executor.h
浏览文件 @
08c25995
...
...
@@ -54,6 +54,7 @@ class Executor {
explicit
Executor
(
const
platform
::
Place
&
place
);
explicit
Executor
(
Scope
*
scope
,
const
platform
::
Place
&
place
);
/*
* Close this Executor.
* Calling this method will send complete messages to all pserver instances.
...
...
@@ -110,8 +111,20 @@ class Executor {
void
EnableMKLDNN
(
const
ProgramDesc
&
program
);
void
RunFromTrainerDesc
(
const
ProgramDesc
&
main_program
,
const
std
::
string
&
trainer_desc_str
,
const
bool
debug
);
void
RunFromDataset
(
const
ProgramDesc
&
main_program
,
const
Dataset
*
dataset
,
const
std
::
string
&
trainer_desc_str
,
const
bool
debug
);
public:
std
::
shared_ptr
<
paddle
::
framework
::
FleetWrapper
>
fleet_ptr_
;
Scope
*
root_scope_
;
private:
const
platform
::
Place
place_
;
int
actual_thread_num_
;
};
}
// namespace framework
...
...
paddle/fluid/framework/multi_trainer.cc
浏览文件 @
08c25995
...
...
@@ -21,25 +21,34 @@ limitations under the License. */
namespace
paddle
{
namespace
framework
{
void
MultiTrainer
::
Initialize
(
const
TrainerDesc
&
trainer_desc
)
{
void
MultiTrainer
::
Initialize
(
const
TrainerDesc
&
trainer_desc
,
Dataset
*
dataset
)
{
thread_num_
=
trainer_desc
.
thread_num
();
// get filelist from trainer_desc here
workers_
.
resize
(
thread_num_
);
readers_
.
resize
(
thread_num_
);
if
(
NULL
==
dataset
)
{
readers_
.
resize
(
thread_num_
);
for
(
int
i
=
0
;
i
<
thread_num_
;
++
i
)
{
readers_
[
i
]
=
DataFeedFactory
::
CreateDataFeed
(
trainer_desc
.
data_desc
().
name
());
readers_
[
i
]
->
Init
(
trainer_desc
.
data_desc
());
}
std
::
vector
<
std
::
string
>
filelist_vec
;
for
(
unsigned
i
=
0
;
i
<
trainer_desc
.
filelist_size
();
++
i
)
{
filelist_vec
.
push_back
(
trainer_desc
.
filelist
(
i
));
}
readers_
[
0
]
->
SetFileList
(
filelist_vec
);
}
else
{
// readers_ = dataset.get_readers(); ?
}
for
(
int
i
=
0
;
i
<
thread_num_
;
++
i
)
{
workers_
[
i
]
=
DeviceWorkerFactory
::
CreateDeviceWorker
(
trainer_desc
.
device_worker_name
());
readers_
[
i
]
=
DataFeedFactory
::
CreateDataFeed
(
trainer_desc
.
data_desc
().
name
());
workers_
[
i
]
->
SetDeviceIndex
(
i
);
readers_
[
i
]
->
Init
(
trainer_desc
.
data_desc
());
workers_
[
i
]
->
SetDataFeed
(
readers_
[
i
]);
}
std
::
vector
<
std
::
string
>
filelist_vec
;
for
(
unsigned
i
=
0
;
i
<
trainer_desc
.
filelist_size
();
++
i
)
{
filelist_vec
.
push_back
(
trainer_desc
.
filelist
(
i
));
}
readers_
[
0
]
->
SetFileList
(
filelist_vec
);
}
// call only after all resources are set in current trainer
...
...
编辑
预览
Markdown
is supported
0%
请重试
或
添加新附件
.
添加附件
取消
You are about to add
0
people
to the discussion. Proceed with caution.
先完成此消息的编辑!
取消
想要评论请
注册
或
登录