Skip to content
体验新版
项目
组织
正在加载...
登录
切换导航
打开侧边栏
xiaopch
PaddleRec
提交
8334c08d
P
PaddleRec
项目概览
xiaopch
/
PaddleRec
与 Fork 源项目一致
Fork自
PaddlePaddle / PaddleRec
通知
1
Star
0
Fork
0
代码
文件
提交
分支
Tags
贡献者
分支图
Diff
Issue
0
列表
看板
标记
里程碑
合并请求
0
Wiki
0
Wiki
分析
仓库
DevOps
项目成员
Pages
P
PaddleRec
项目概览
项目概览
详情
发布
仓库
仓库
文件
提交
分支
标签
贡献者
分支图
比较
Issue
0
Issue
0
列表
看板
标记
里程碑
合并请求
0
合并请求
0
Pages
分析
分析
仓库分析
DevOps
Wiki
0
Wiki
成员
成员
收起侧边栏
关闭侧边栏
动态
分支图
创建新Issue
提交
Issue看板
体验新版 GitCode,发现更多精彩内容 >>
提交
8334c08d
编写于
4月 15, 2020
作者:
T
tangwei
浏览文件
操作
浏览文件
下载
电子邮件补丁
差异文件
structure rebuild
上级
baf8765b
变更
10
显示空白变更内容
内联
并排
Showing
10 changed file
with
34 addition
and
120 deletion
+34
-120
fleetrec/core/trainer/details/local_engine.py
fleetrec/core/trainer/details/local_engine.py
+3
-3
fleetrec/examples/built_in/cluster_training_local.yaml
fleetrec/examples/built_in/cluster_training_local.yaml
+10
-0
fleetrec/examples/built_in/cluster_training_mpi.yaml
fleetrec/examples/built_in/cluster_training_mpi.yaml
+10
-0
fleetrec/examples/built_in/cluster_training_user_define.yaml
fleetrec/examples/built_in/cluster_training_user_define.yaml
+2
-0
fleetrec/examples/built_in/ctr-dnn_train.yaml
fleetrec/examples/built_in/ctr-dnn_train.yaml
+5
-2
fleetrec/examples/built_in/run.py
fleetrec/examples/built_in/run.py
+1
-1
fleetrec/examples/built_in/single_training.yaml
fleetrec/examples/built_in/single_training.yaml
+2
-0
fleetrec/examples/built_in/user_define_trainer.py
fleetrec/examples/built_in/user_define_trainer.py
+1
-0
fleetrec/examples/ctr-dnn_train_cluster.yaml
fleetrec/examples/ctr-dnn_train_cluster.yaml
+0
-62
fleetrec/examples/ctr-dnn_train_userdefine.yaml
fleetrec/examples/ctr-dnn_train_userdefine.yaml
+0
-52
未找到文件。
fleetrec/core/trainer/details/local_engine.py
浏览文件 @
8334c08d
...
...
@@ -38,7 +38,7 @@ def start_procs(args, yaml):
user_endpoints_ips
=
[
x
.
split
(
":"
)[
0
]
for
x
in
user_endpoints
.
split
(
","
)]
user_endpoints_port
=
[
x
.
split
(
":"
)[
1
]
for
x
in
user_endpoints
.
split
(
","
)]
factory
=
"fleetrec.
trainer
.factory"
factory
=
"fleetrec.
core
.factory"
cmd
=
[
sys
.
executable
,
"-u"
,
"-m"
,
factory
,
yaml
]
for
i
in
range
(
server_num
):
...
...
@@ -91,11 +91,11 @@ def start_procs(args, yaml):
procs
[
i
].
terminate
()
print
(
"all parameter server are killed"
,
file
=
sys
.
stderr
)
class
Launch
():
class
Launch
:
def
__init__
(
self
,
envs
,
trainer
):
self
.
envs
=
envs
self
.
trainer
=
trainer
def
run
(
self
):
start_procs
(
self
.
envs
,
self
.
trainer
)
fleetrec/examples/built_in/cluster_training_local.yaml
0 → 100644
浏览文件 @
8334c08d
trainer
:
"
LocalClusterTraining"
pserver_num
:
2
trainer_num
:
2
start_port
:
36001
log_dirname
:
"
logs"
strategy
:
mode
:
"
async"
fleetrec/examples/built_in/cluster_training_mpi.yaml
0 → 100644
浏览文件 @
8334c08d
trainer
:
"
MPIClusterTraining"
pserver_num
:
2
trainer_num
:
2
start_port
:
36001
log_dirname
:
"
logs"
strategy
:
mode
:
"
async"
fleetrec/examples/built_in/cluster_training_user_define.yaml
0 → 100644
浏览文件 @
8334c08d
trainer
:
"
UserDefineTrainer"
location
:
"
/root/FleetRec/fleetrec/examples/user_define_trainer.py"
fleetrec/examples/
ctr-dnn_train_single
.yaml
→
fleetrec/examples/
built_in/ctr-dnn_train
.yaml
浏览文件 @
8334c08d
...
...
@@ -15,9 +15,10 @@
train
:
threads
:
12
epochs
:
10
trainer
:
"
SingleTraining
"
trainer
:
"
single_training.yaml
"
reader
:
mode
:
"
dataset"
batch_size
:
2
class
:
"
fleetrec.models.ctr_dnn.data_generator"
train_data_path
:
"
/root/FleetRec/fleetrec/models/ctr_dnn/data/train/"
...
...
@@ -40,6 +41,8 @@ train:
inference
:
dirname
:
"
models_for_inference"
epoch_interval
:
4
feed_varnames
:
[
"
C1"
,
"
C2"
,
"
C3"
]
fetch_varnames
:
"
predict"
save_last
:
True
evaluate
:
...
...
fleetrec/examples/
trai
n.py
→
fleetrec/examples/
built_in/ru
n.py
浏览文件 @
8334c08d
...
...
@@ -14,7 +14,7 @@
import
os
from
fleetrec.
trainer
.factory
import
TrainerFactory
from
fleetrec.
core
.factory
import
TrainerFactory
if
__name__
==
"__main__"
:
...
...
fleetrec/examples/built_in/single_training.yaml
0 → 100644
浏览文件 @
8334c08d
trainer
:
"
SingleTraining"
\ No newline at end of file
fleetrec/examples/user_define_trainer.py
→
fleetrec/examples/
built_in/
user_define_trainer.py
浏览文件 @
8334c08d
...
...
@@ -21,6 +21,7 @@ from fleetrec.utils import envs
class
UserDefineTrainer
(
TranspileTrainer
):
def
__init__
(
self
,
config
=
None
):
TranspileTrainer
.
__init__
(
self
,
config
)
print
(
"this is a demo about how to use user define trainer in fleet-rec"
)
def
processor_register
(
self
):
self
.
regist_context_processor
(
'uninit'
,
self
.
instance
)
...
...
fleetrec/examples/ctr-dnn_train_cluster.yaml
已删除
100644 → 0
浏览文件 @
baf8765b
# Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
train
:
threads
:
12
epochs
:
10
trainer
:
"
ClusterTraining"
container
:
"
local"
pserver_num
:
2
trainer_num
:
2
start_port
:
36001
log_dirname
:
"
logs"
strategy
:
mode
:
"
async"
reader
:
mode
:
"
dataset"
batch_size
:
2
pipe_command
:
"
python
/paddle/eleps/fleetrec/models/ctr_dnn/dataset.py"
train_data_path
:
"
/paddle/eleps/fleetrec/models/ctr_dnn/data/train"
model
:
models
:
"
fleetrec.models.ctr_dnn.model"
hyper_parameters
:
sparse_inputs_slots
:
27
sparse_feature_number
:
1000001
sparse_feature_dim
:
8
dense_input_dim
:
13
fc_sizes
:
[
512
,
256
,
128
,
32
]
learning_rate
:
0.001
save
:
increment
:
dirname
:
"
models_for_increment"
epoch_interval
:
2
save_last
:
True
inference
:
dirname
:
"
models_for_inference"
epoch_interval
:
4
feed_varnames
:
[
"
C1"
,
"
C2"
,
"
C3"
]
fetch_varnames
:
"
predict"
save_last
:
True
evaluate
:
batch_size
:
32
train_thread_num
:
12
reader
:
"
reader.py"
fleetrec/examples/ctr-dnn_train_userdefine.yaml
已删除
100644 → 0
浏览文件 @
baf8765b
# Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
train
:
threads
:
12
epochs
:
10
trainer
:
"
UserDefineTrainer"
location
:
"
/root/FleetRec/fleetrec/examples/user_define_trainer.py"
reader
:
batch_size
:
2
class
:
"
fleetrec.models.ctr_dnn.data_generator"
train_data_path
:
"
/root/FleetRec/fleetrec/models/ctr_dnn/data/train/"
model
:
models
:
"
fleetrec.models.ctr_dnn.model"
hyper_parameters
:
sparse_inputs_slots
:
27
sparse_feature_number
:
1000001
sparse_feature_dim
:
8
dense_input_dim
:
13
fc_sizes
:
[
512
,
256
,
128
,
32
]
learning_rate
:
0.001
save
:
increment
:
dirname
:
"
models_for_increment"
epoch_interval
:
2
save_last
:
True
inference
:
dirname
:
"
models_for_inference"
epoch_interval
:
4
save_last
:
True
evaluate
:
batch_size
:
32
train_thread_num
:
12
reader
:
"
reader.py"
编辑
预览
Markdown
is supported
0%
请重试
或
添加新附件
.
添加附件
取消
You are about to add
0
people
to the discussion. Proceed with caution.
先完成此消息的编辑!
取消
想要评论请
注册
或
登录