Skip to content
体验新版
项目
组织
正在加载...
登录
切换导航
打开侧边栏
PaddlePaddle
PaddleRec
提交
0bfba555
P
PaddleRec
项目概览
PaddlePaddle
/
PaddleRec
通知
68
Star
12
Fork
5
代码
文件
提交
分支
Tags
贡献者
分支图
Diff
Issue
27
列表
看板
标记
里程碑
合并请求
10
Wiki
1
Wiki
分析
仓库
DevOps
项目成员
Pages
P
PaddleRec
项目概览
项目概览
详情
发布
仓库
仓库
文件
提交
分支
标签
贡献者
分支图
比较
Issue
27
Issue
27
列表
看板
标记
里程碑
合并请求
10
合并请求
10
Pages
分析
分析
仓库分析
DevOps
Wiki
1
Wiki
成员
成员
收起侧边栏
关闭侧边栏
动态
分支图
创建新Issue
提交
Issue看板
提交
0bfba555
编写于
4月 09, 2020
作者:
T
tangwei
浏览文件
操作
浏览文件
下载
电子邮件补丁
差异文件
update to fleetrec
上级
d14987ed
变更
39
隐藏空白更改
内联
并排
Showing
39 changed file
with
23 addition
and
23 deletion
+23
-23
fleetrec/__init__.py
fleetrec/__init__.py
+0
-0
fleetrec/examples/__init__.py
fleetrec/examples/__init__.py
+0
-0
fleetrec/examples/ctr-dnn_train_cluster.yaml
fleetrec/examples/ctr-dnn_train_cluster.yaml
+3
-3
fleetrec/examples/ctr-dnn_train_single.yaml
fleetrec/examples/ctr-dnn_train_single.yaml
+3
-3
fleetrec/examples/train.py
fleetrec/examples/train.py
+1
-1
fleetrec/metrics/__init__.py
fleetrec/metrics/__init__.py
+0
-0
fleetrec/metrics/auc_metrics.py
fleetrec/metrics/auc_metrics.py
+0
-0
fleetrec/metrics/base.py
fleetrec/metrics/base.py
+0
-0
fleetrec/models/__init__.py
fleetrec/models/__init__.py
+0
-0
fleetrec/models/base.py
fleetrec/models/base.py
+0
-0
fleetrec/models/ctr_dnn/__init__.py
fleetrec/models/ctr_dnn/__init__.py
+0
-0
fleetrec/models/ctr_dnn/data/test/sample_test.txt
fleetrec/models/ctr_dnn/data/test/sample_test.txt
+0
-0
fleetrec/models/ctr_dnn/data/train/sample_train.txt
fleetrec/models/ctr_dnn/data/train/sample_train.txt
+0
-0
fleetrec/models/ctr_dnn/dataloader.py
fleetrec/models/ctr_dnn/dataloader.py
+0
-0
fleetrec/models/ctr_dnn/dataset.py
fleetrec/models/ctr_dnn/dataset.py
+0
-0
fleetrec/models/ctr_dnn/hyper_parameters.yaml
fleetrec/models/ctr_dnn/hyper_parameters.yaml
+0
-0
fleetrec/models/ctr_dnn/model.py
fleetrec/models/ctr_dnn/model.py
+1
-1
fleetrec/models/layers.py
fleetrec/models/layers.py
+0
-0
fleetrec/models/word2vec/hyper_parameters.yaml
fleetrec/models/word2vec/hyper_parameters.yaml
+0
-0
fleetrec/models/word2vec/model.py
fleetrec/models/word2vec/model.py
+0
-0
fleetrec/models/word2vec/reader.py
fleetrec/models/word2vec/reader.py
+0
-0
fleetrec/models/word2vec/sample_test.txt
fleetrec/models/word2vec/sample_test.txt
+0
-0
fleetrec/models/word2vec/sample_train.txt
fleetrec/models/word2vec/sample_train.txt
+0
-0
fleetrec/reader/__init__.py
fleetrec/reader/__init__.py
+0
-0
fleetrec/reader/data_loader.py
fleetrec/reader/data_loader.py
+0
-0
fleetrec/reader/dataset.py
fleetrec/reader/dataset.py
+0
-0
fleetrec/trainer/__init__.py
fleetrec/trainer/__init__.py
+0
-0
fleetrec/trainer/cluster_trainer.py
fleetrec/trainer/cluster_trainer.py
+0
-0
fleetrec/trainer/ctr_trainer.py
fleetrec/trainer/ctr_trainer.py
+5
-5
fleetrec/trainer/factory.py
fleetrec/trainer/factory.py
+8
-8
fleetrec/trainer/local_engine.py
fleetrec/trainer/local_engine.py
+1
-1
fleetrec/trainer/single_trainer.py
fleetrec/trainer/single_trainer.py
+0
-0
fleetrec/trainer/trainer.py
fleetrec/trainer/trainer.py
+0
-0
fleetrec/trainer/transpiler_trainer.py
fleetrec/trainer/transpiler_trainer.py
+0
-0
fleetrec/utils/__init__.py
fleetrec/utils/__init__.py
+0
-0
fleetrec/utils/envs.py
fleetrec/utils/envs.py
+1
-1
fleetrec/utils/fs.py
fleetrec/utils/fs.py
+0
-0
fleetrec/utils/table.py
fleetrec/utils/table.py
+0
-0
fleetrec/utils/util.py
fleetrec/utils/util.py
+0
-0
未找到文件。
__init__.py
→
fleetrec/
__init__.py
浏览文件 @
0bfba555
文件已移动
examples/__init__.py
→
fleetrec/
examples/__init__.py
浏览文件 @
0bfba555
文件已移动
examples/ctr-dnn_train_cluster.yaml
→
fleetrec/
examples/ctr-dnn_train_cluster.yaml
浏览文件 @
0bfba555
...
@@ -41,11 +41,11 @@ train:
...
@@ -41,11 +41,11 @@ train:
reader
:
reader
:
mode
:
"
dataset"
mode
:
"
dataset"
batch_size
:
2
batch_size
:
2
pipe_command
:
"
python
/paddle/fleet
_
rec/models/ctr_dnn/dataset.py"
pipe_command
:
"
python
/paddle/fleetrec/models/ctr_dnn/dataset.py"
train_data_path
:
"
/paddle/fleet
_
rec/models/ctr_dnn/data/train"
train_data_path
:
"
/paddle/fleetrec/models/ctr_dnn/data/train"
model
:
model
:
models
:
"
fleet
_
rec.models.ctr_dnn.model"
models
:
"
fleetrec.models.ctr_dnn.model"
hyper_parameters
:
hyper_parameters
:
sparse_inputs_slots
:
27
sparse_inputs_slots
:
27
sparse_feature_number
:
1000001
sparse_feature_number
:
1000001
...
...
examples/ctr-dnn_train_single.yaml
→
fleetrec/
examples/ctr-dnn_train_single.yaml
浏览文件 @
0bfba555
...
@@ -35,11 +35,11 @@ train:
...
@@ -35,11 +35,11 @@ train:
reader
:
reader
:
mode
:
"
dataset"
mode
:
"
dataset"
batch_size
:
2
batch_size
:
2
pipe_command
:
"
python
/paddle/fleet
_
rec/models/ctr_dnn/dataset.py"
pipe_command
:
"
python
/paddle/fleetrec/models/ctr_dnn/dataset.py"
train_data_path
:
"
/paddle/fleet
_
rec/models/ctr_dnn/data/train"
train_data_path
:
"
/paddle/fleetrec/models/ctr_dnn/data/train"
model
:
model
:
models
:
"
fleet
_
rec.models.ctr_dnn.model"
models
:
"
fleetrec.models.ctr_dnn.model"
hyper_parameters
:
hyper_parameters
:
sparse_inputs_slots
:
27
sparse_inputs_slots
:
27
sparse_feature_number
:
1000001
sparse_feature_number
:
1000001
...
...
examples/train.py
→
fleetrec/
examples/train.py
浏览文件 @
0bfba555
...
@@ -26,7 +26,7 @@
...
@@ -26,7 +26,7 @@
import
os
import
os
from
fleet
_
rec.trainer.factory
import
TrainerFactory
from
fleetrec.trainer.factory
import
TrainerFactory
if
__name__
==
"__main__"
:
if
__name__
==
"__main__"
:
...
...
metrics/__init__.py
→
fleetrec/
metrics/__init__.py
浏览文件 @
0bfba555
文件已移动
metrics/auc_metrics.py
→
fleetrec/
metrics/auc_metrics.py
浏览文件 @
0bfba555
文件已移动
metrics/base.py
→
fleetrec/
metrics/base.py
浏览文件 @
0bfba555
文件已移动
models/__init__.py
→
fleetrec/
models/__init__.py
浏览文件 @
0bfba555
文件已移动
models/base.py
→
fleetrec/
models/base.py
浏览文件 @
0bfba555
文件已移动
models/ctr_dnn/__init__.py
→
fleetrec/
models/ctr_dnn/__init__.py
浏览文件 @
0bfba555
文件已移动
models/ctr_dnn/data/test/sample_test.txt
→
fleetrec/
models/ctr_dnn/data/test/sample_test.txt
浏览文件 @
0bfba555
文件已移动
models/ctr_dnn/data/train/sample_train.txt
→
fleetrec/
models/ctr_dnn/data/train/sample_train.txt
浏览文件 @
0bfba555
文件已移动
models/ctr_dnn/dataloader.py
→
fleetrec/
models/ctr_dnn/dataloader.py
浏览文件 @
0bfba555
文件已移动
models/ctr_dnn/dataset.py
→
fleetrec/
models/ctr_dnn/dataset.py
浏览文件 @
0bfba555
文件已移动
models/ctr_dnn/hyper_parameters.yaml
→
fleetrec/
models/ctr_dnn/hyper_parameters.yaml
浏览文件 @
0bfba555
文件已移动
models/ctr_dnn/model.py
→
fleetrec/
models/ctr_dnn/model.py
浏览文件 @
0bfba555
...
@@ -15,7 +15,7 @@
...
@@ -15,7 +15,7 @@
import
math
import
math
import
paddle.fluid
as
fluid
import
paddle.fluid
as
fluid
from
fleet
_
rec.utils
import
envs
from
fleetrec.utils
import
envs
class
Train
(
object
):
class
Train
(
object
):
...
...
models/layers.py
→
fleetrec/
models/layers.py
浏览文件 @
0bfba555
文件已移动
models/word2vec/hyper_parameters.yaml
→
fleetrec/
models/word2vec/hyper_parameters.yaml
浏览文件 @
0bfba555
文件已移动
models/word2vec/model.py
→
fleetrec/
models/word2vec/model.py
浏览文件 @
0bfba555
文件已移动
models/word2vec/reader.py
→
fleetrec/
models/word2vec/reader.py
浏览文件 @
0bfba555
文件已移动
models/word2vec/sample_test.txt
→
fleetrec/
models/word2vec/sample_test.txt
浏览文件 @
0bfba555
文件已移动
models/word2vec/sample_train.txt
→
fleetrec/
models/word2vec/sample_train.txt
浏览文件 @
0bfba555
文件已移动
reader/__init__.py
→
fleetrec/
reader/__init__.py
浏览文件 @
0bfba555
文件已移动
reader/data_loader.py
→
fleetrec/
reader/data_loader.py
浏览文件 @
0bfba555
文件已移动
reader/dataset.py
→
fleetrec/
reader/dataset.py
浏览文件 @
0bfba555
文件已移动
trainer/__init__.py
→
fleetrec/
trainer/__init__.py
浏览文件 @
0bfba555
文件已移动
trainer/cluster_trainer.py
→
fleetrec/
trainer/cluster_trainer.py
浏览文件 @
0bfba555
文件已移动
trainer/ctr_trainer.py
→
fleetrec/
trainer/ctr_trainer.py
浏览文件 @
0bfba555
...
@@ -23,11 +23,11 @@ import paddle.fluid as fluid
...
@@ -23,11 +23,11 @@ import paddle.fluid as fluid
from
paddle.fluid.incubate.fleet.parameter_server.pslib
import
fleet
from
paddle.fluid.incubate.fleet.parameter_server.pslib
import
fleet
from
paddle.fluid.incubate.fleet.base.role_maker
import
GeneralRoleMaker
from
paddle.fluid.incubate.fleet.base.role_maker
import
GeneralRoleMaker
from
fleet
_
rec.utils
import
fs
as
fs
from
fleetrec.utils
import
fs
as
fs
from
fleet
_
rec.utils
import
util
as
util
from
fleetrec.utils
import
util
as
util
from
fleet
_
rec.metrics.auc_metrics
import
AUCMetric
from
fleetrec.metrics.auc_metrics
import
AUCMetric
from
fleet
_
rec.models
import
base
as
model_basic
from
fleetrec.models
import
base
as
model_basic
from
fleet
_
rec.reader
import
dataset
from
fleetrec.reader
import
dataset
from
.trainer
import
Trainer
from
.trainer
import
Trainer
...
...
trainer/factory.py
→
fleetrec/
trainer/factory.py
浏览文件 @
0bfba555
...
@@ -29,16 +29,16 @@ import sys
...
@@ -29,16 +29,16 @@ import sys
import
yaml
import
yaml
from
fleet
_
rec.trainer.single_trainer
import
SingleTrainerWithDataloader
from
fleetrec.trainer.single_trainer
import
SingleTrainerWithDataloader
from
fleet
_
rec.trainer.single_trainer
import
SingleTrainerWithDataset
from
fleetrec.trainer.single_trainer
import
SingleTrainerWithDataset
from
fleet
_
rec.trainer.cluster_trainer
import
ClusterTrainerWithDataloader
from
fleetrec.trainer.cluster_trainer
import
ClusterTrainerWithDataloader
from
fleet
_
rec.trainer.cluster_trainer
import
ClusterTrainerWithDataset
from
fleetrec.trainer.cluster_trainer
import
ClusterTrainerWithDataset
from
fleet
_
rec.trainer.local_engine
import
Launch
from
fleetrec.trainer.local_engine
import
Launch
from
fleet
_
rec.trainer.ctr_trainer
import
CtrPaddleTrainer
from
fleetrec.trainer.ctr_trainer
import
CtrPaddleTrainer
from
fleet
_
rec.utils
import
envs
from
fleetrec.utils
import
envs
def
str2bool
(
v
):
def
str2bool
(
v
):
...
@@ -103,7 +103,7 @@ class TrainerFactory(object):
...
@@ -103,7 +103,7 @@ class TrainerFactory(object):
with
open
(
config
,
'r'
)
as
rb
:
with
open
(
config
,
'r'
)
as
rb
:
_config
=
yaml
.
load
(
rb
.
read
(),
Loader
=
yaml
.
FullLoader
)
_config
=
yaml
.
load
(
rb
.
read
(),
Loader
=
yaml
.
FullLoader
)
else
:
else
:
raise
ValueError
(
"fleet
_
rec's config only support yaml"
)
raise
ValueError
(
"fleetrec's config only support yaml"
)
envs
.
set_global_envs
(
_config
)
envs
.
set_global_envs
(
_config
)
mode
=
envs
.
get_global_env
(
"train.trainer"
)
mode
=
envs
.
get_global_env
(
"train.trainer"
)
...
...
trainer/local_engine.py
→
fleetrec/
trainer/local_engine.py
浏览文件 @
0bfba555
...
@@ -38,7 +38,7 @@ def start_procs(args, yaml):
...
@@ -38,7 +38,7 @@ def start_procs(args, yaml):
user_endpoints_ips
=
[
x
.
split
(
":"
)[
0
]
for
x
in
user_endpoints
.
split
(
","
)]
user_endpoints_ips
=
[
x
.
split
(
":"
)[
0
]
for
x
in
user_endpoints
.
split
(
","
)]
user_endpoints_port
=
[
x
.
split
(
":"
)[
1
]
for
x
in
user_endpoints
.
split
(
","
)]
user_endpoints_port
=
[
x
.
split
(
":"
)[
1
]
for
x
in
user_endpoints
.
split
(
","
)]
factory
=
"fleet
_
rec.trainer.factory"
factory
=
"fleetrec.trainer.factory"
cmd
=
[
sys
.
executable
,
"-u"
,
"-m"
,
factory
,
yaml
]
cmd
=
[
sys
.
executable
,
"-u"
,
"-m"
,
factory
,
yaml
]
for
i
in
range
(
server_num
):
for
i
in
range
(
server_num
):
...
...
trainer/single_trainer.py
→
fleetrec/
trainer/single_trainer.py
浏览文件 @
0bfba555
文件已移动
trainer/trainer.py
→
fleetrec/
trainer/trainer.py
浏览文件 @
0bfba555
文件已移动
trainer/transpiler_trainer.py
→
fleetrec/
trainer/transpiler_trainer.py
浏览文件 @
0bfba555
文件已移动
utils/__init__.py
→
fleetrec/
utils/__init__.py
浏览文件 @
0bfba555
文件已移动
utils/envs.py
→
fleetrec/
utils/envs.py
浏览文件 @
0bfba555
...
@@ -70,7 +70,7 @@ def pretty_print_envs(envs, header=None):
...
@@ -70,7 +70,7 @@ def pretty_print_envs(envs, header=None):
if
header
:
if
header
:
draws
+=
h_format
.
format
(
header
[
0
],
header
[
1
])
draws
+=
h_format
.
format
(
header
[
0
],
header
[
1
])
else
:
else
:
draws
+=
h_format
.
format
(
"fleet
_
rec Global Envs"
,
"Value"
)
draws
+=
h_format
.
format
(
"fleetrec Global Envs"
,
"Value"
)
draws
+=
line
+
"
\n
"
draws
+=
line
+
"
\n
"
...
...
utils/fs.py
→
fleetrec/
utils/fs.py
浏览文件 @
0bfba555
文件已移动
utils/table.py
→
fleetrec/
utils/table.py
浏览文件 @
0bfba555
文件已移动
utils/util.py
→
fleetrec/
utils/util.py
浏览文件 @
0bfba555
文件已移动
编辑
预览
Markdown
is supported
0%
请重试
或
添加新附件
.
添加附件
取消
You are about to add
0
people
to the discussion. Proceed with caution.
先完成此消息的编辑!
取消
想要评论请
注册
或
登录