Skip to content
体验新版
项目
组织
正在加载...
登录
切换导航
打开侧边栏
BaiXuePrincess
PaddleRec
提交
0bfba555
P
PaddleRec
项目概览
BaiXuePrincess
/
PaddleRec
与 Fork 源项目一致
Fork自
PaddlePaddle / PaddleRec
通知
1
Star
0
Fork
0
代码
文件
提交
分支
Tags
贡献者
分支图
Diff
Issue
0
列表
看板
标记
里程碑
合并请求
0
Wiki
0
Wiki
分析
仓库
DevOps
项目成员
Pages
P
PaddleRec
项目概览
项目概览
详情
发布
仓库
仓库
文件
提交
分支
标签
贡献者
分支图
比较
Issue
0
Issue
0
列表
看板
标记
里程碑
合并请求
0
合并请求
0
Pages
分析
分析
仓库分析
DevOps
Wiki
0
Wiki
成员
成员
收起侧边栏
关闭侧边栏
动态
分支图
创建新Issue
提交
Issue看板
提交
0bfba555
编写于
4月 09, 2020
作者:
T
tangwei
浏览文件
操作
浏览文件
下载
电子邮件补丁
差异文件
update to fleetrec
上级
d14987ed
变更
39
显示空白变更内容
内联
并排
Showing
39 changed file
with
23 addition
and
23 deletion
+23
-23
fleetrec/__init__.py
fleetrec/__init__.py
+0
-0
fleetrec/examples/__init__.py
fleetrec/examples/__init__.py
+0
-0
fleetrec/examples/ctr-dnn_train_cluster.yaml
fleetrec/examples/ctr-dnn_train_cluster.yaml
+3
-3
fleetrec/examples/ctr-dnn_train_single.yaml
fleetrec/examples/ctr-dnn_train_single.yaml
+3
-3
fleetrec/examples/train.py
fleetrec/examples/train.py
+1
-1
fleetrec/metrics/__init__.py
fleetrec/metrics/__init__.py
+0
-0
fleetrec/metrics/auc_metrics.py
fleetrec/metrics/auc_metrics.py
+0
-0
fleetrec/metrics/base.py
fleetrec/metrics/base.py
+0
-0
fleetrec/models/__init__.py
fleetrec/models/__init__.py
+0
-0
fleetrec/models/base.py
fleetrec/models/base.py
+0
-0
fleetrec/models/ctr_dnn/__init__.py
fleetrec/models/ctr_dnn/__init__.py
+0
-0
fleetrec/models/ctr_dnn/data/test/sample_test.txt
fleetrec/models/ctr_dnn/data/test/sample_test.txt
+0
-0
fleetrec/models/ctr_dnn/data/train/sample_train.txt
fleetrec/models/ctr_dnn/data/train/sample_train.txt
+0
-0
fleetrec/models/ctr_dnn/dataloader.py
fleetrec/models/ctr_dnn/dataloader.py
+0
-0
fleetrec/models/ctr_dnn/dataset.py
fleetrec/models/ctr_dnn/dataset.py
+0
-0
fleetrec/models/ctr_dnn/hyper_parameters.yaml
fleetrec/models/ctr_dnn/hyper_parameters.yaml
+0
-0
fleetrec/models/ctr_dnn/model.py
fleetrec/models/ctr_dnn/model.py
+1
-1
fleetrec/models/layers.py
fleetrec/models/layers.py
+0
-0
fleetrec/models/word2vec/hyper_parameters.yaml
fleetrec/models/word2vec/hyper_parameters.yaml
+0
-0
fleetrec/models/word2vec/model.py
fleetrec/models/word2vec/model.py
+0
-0
fleetrec/models/word2vec/reader.py
fleetrec/models/word2vec/reader.py
+0
-0
fleetrec/models/word2vec/sample_test.txt
fleetrec/models/word2vec/sample_test.txt
+0
-0
fleetrec/models/word2vec/sample_train.txt
fleetrec/models/word2vec/sample_train.txt
+0
-0
fleetrec/reader/__init__.py
fleetrec/reader/__init__.py
+0
-0
fleetrec/reader/data_loader.py
fleetrec/reader/data_loader.py
+0
-0
fleetrec/reader/dataset.py
fleetrec/reader/dataset.py
+0
-0
fleetrec/trainer/__init__.py
fleetrec/trainer/__init__.py
+0
-0
fleetrec/trainer/cluster_trainer.py
fleetrec/trainer/cluster_trainer.py
+0
-0
fleetrec/trainer/ctr_trainer.py
fleetrec/trainer/ctr_trainer.py
+5
-5
fleetrec/trainer/factory.py
fleetrec/trainer/factory.py
+8
-8
fleetrec/trainer/local_engine.py
fleetrec/trainer/local_engine.py
+1
-1
fleetrec/trainer/single_trainer.py
fleetrec/trainer/single_trainer.py
+0
-0
fleetrec/trainer/trainer.py
fleetrec/trainer/trainer.py
+0
-0
fleetrec/trainer/transpiler_trainer.py
fleetrec/trainer/transpiler_trainer.py
+0
-0
fleetrec/utils/__init__.py
fleetrec/utils/__init__.py
+0
-0
fleetrec/utils/envs.py
fleetrec/utils/envs.py
+1
-1
fleetrec/utils/fs.py
fleetrec/utils/fs.py
+0
-0
fleetrec/utils/table.py
fleetrec/utils/table.py
+0
-0
fleetrec/utils/util.py
fleetrec/utils/util.py
+0
-0
未找到文件。
__init__.py
→
fleetrec/
__init__.py
浏览文件 @
0bfba555
文件已移动
examples/__init__.py
→
fleetrec/
examples/__init__.py
浏览文件 @
0bfba555
文件已移动
examples/ctr-dnn_train_cluster.yaml
→
fleetrec/
examples/ctr-dnn_train_cluster.yaml
浏览文件 @
0bfba555
...
@@ -41,11 +41,11 @@ train:
...
@@ -41,11 +41,11 @@ train:
reader
:
reader
:
mode
:
"
dataset"
mode
:
"
dataset"
batch_size
:
2
batch_size
:
2
pipe_command
:
"
python
/paddle/fleet
_
rec/models/ctr_dnn/dataset.py"
pipe_command
:
"
python
/paddle/fleetrec/models/ctr_dnn/dataset.py"
train_data_path
:
"
/paddle/fleet
_
rec/models/ctr_dnn/data/train"
train_data_path
:
"
/paddle/fleetrec/models/ctr_dnn/data/train"
model
:
model
:
models
:
"
fleet
_
rec.models.ctr_dnn.model"
models
:
"
fleetrec.models.ctr_dnn.model"
hyper_parameters
:
hyper_parameters
:
sparse_inputs_slots
:
27
sparse_inputs_slots
:
27
sparse_feature_number
:
1000001
sparse_feature_number
:
1000001
...
...
examples/ctr-dnn_train_single.yaml
→
fleetrec/
examples/ctr-dnn_train_single.yaml
浏览文件 @
0bfba555
...
@@ -35,11 +35,11 @@ train:
...
@@ -35,11 +35,11 @@ train:
reader
:
reader
:
mode
:
"
dataset"
mode
:
"
dataset"
batch_size
:
2
batch_size
:
2
pipe_command
:
"
python
/paddle/fleet
_
rec/models/ctr_dnn/dataset.py"
pipe_command
:
"
python
/paddle/fleetrec/models/ctr_dnn/dataset.py"
train_data_path
:
"
/paddle/fleet
_
rec/models/ctr_dnn/data/train"
train_data_path
:
"
/paddle/fleetrec/models/ctr_dnn/data/train"
model
:
model
:
models
:
"
fleet
_
rec.models.ctr_dnn.model"
models
:
"
fleetrec.models.ctr_dnn.model"
hyper_parameters
:
hyper_parameters
:
sparse_inputs_slots
:
27
sparse_inputs_slots
:
27
sparse_feature_number
:
1000001
sparse_feature_number
:
1000001
...
...
examples/train.py
→
fleetrec/
examples/train.py
浏览文件 @
0bfba555
...
@@ -26,7 +26,7 @@
...
@@ -26,7 +26,7 @@
import
os
import
os
from
fleet
_
rec.trainer.factory
import
TrainerFactory
from
fleetrec.trainer.factory
import
TrainerFactory
if
__name__
==
"__main__"
:
if
__name__
==
"__main__"
:
...
...
metrics/__init__.py
→
fleetrec/
metrics/__init__.py
浏览文件 @
0bfba555
文件已移动
metrics/auc_metrics.py
→
fleetrec/
metrics/auc_metrics.py
浏览文件 @
0bfba555
文件已移动
metrics/base.py
→
fleetrec/
metrics/base.py
浏览文件 @
0bfba555
文件已移动
models/__init__.py
→
fleetrec/
models/__init__.py
浏览文件 @
0bfba555
文件已移动
models/base.py
→
fleetrec/
models/base.py
浏览文件 @
0bfba555
文件已移动
models/ctr_dnn/__init__.py
→
fleetrec/
models/ctr_dnn/__init__.py
浏览文件 @
0bfba555
文件已移动
models/ctr_dnn/data/test/sample_test.txt
→
fleetrec/
models/ctr_dnn/data/test/sample_test.txt
浏览文件 @
0bfba555
文件已移动
models/ctr_dnn/data/train/sample_train.txt
→
fleetrec/
models/ctr_dnn/data/train/sample_train.txt
浏览文件 @
0bfba555
文件已移动
models/ctr_dnn/dataloader.py
→
fleetrec/
models/ctr_dnn/dataloader.py
浏览文件 @
0bfba555
文件已移动
models/ctr_dnn/dataset.py
→
fleetrec/
models/ctr_dnn/dataset.py
浏览文件 @
0bfba555
文件已移动
models/ctr_dnn/hyper_parameters.yaml
→
fleetrec/
models/ctr_dnn/hyper_parameters.yaml
浏览文件 @
0bfba555
文件已移动
models/ctr_dnn/model.py
→
fleetrec/
models/ctr_dnn/model.py
浏览文件 @
0bfba555
...
@@ -15,7 +15,7 @@
...
@@ -15,7 +15,7 @@
import
math
import
math
import
paddle.fluid
as
fluid
import
paddle.fluid
as
fluid
from
fleet
_
rec.utils
import
envs
from
fleetrec.utils
import
envs
class
Train
(
object
):
class
Train
(
object
):
...
...
models/layers.py
→
fleetrec/
models/layers.py
浏览文件 @
0bfba555
文件已移动
models/word2vec/hyper_parameters.yaml
→
fleetrec/
models/word2vec/hyper_parameters.yaml
浏览文件 @
0bfba555
文件已移动
models/word2vec/model.py
→
fleetrec/
models/word2vec/model.py
浏览文件 @
0bfba555
文件已移动
models/word2vec/reader.py
→
fleetrec/
models/word2vec/reader.py
浏览文件 @
0bfba555
文件已移动
models/word2vec/sample_test.txt
→
fleetrec/
models/word2vec/sample_test.txt
浏览文件 @
0bfba555
文件已移动
models/word2vec/sample_train.txt
→
fleetrec/
models/word2vec/sample_train.txt
浏览文件 @
0bfba555
文件已移动
reader/__init__.py
→
fleetrec/
reader/__init__.py
浏览文件 @
0bfba555
文件已移动
reader/data_loader.py
→
fleetrec/
reader/data_loader.py
浏览文件 @
0bfba555
文件已移动
reader/dataset.py
→
fleetrec/
reader/dataset.py
浏览文件 @
0bfba555
文件已移动
trainer/__init__.py
→
fleetrec/
trainer/__init__.py
浏览文件 @
0bfba555
文件已移动
trainer/cluster_trainer.py
→
fleetrec/
trainer/cluster_trainer.py
浏览文件 @
0bfba555
文件已移动
trainer/ctr_trainer.py
→
fleetrec/
trainer/ctr_trainer.py
浏览文件 @
0bfba555
...
@@ -23,11 +23,11 @@ import paddle.fluid as fluid
...
@@ -23,11 +23,11 @@ import paddle.fluid as fluid
from
paddle.fluid.incubate.fleet.parameter_server.pslib
import
fleet
from
paddle.fluid.incubate.fleet.parameter_server.pslib
import
fleet
from
paddle.fluid.incubate.fleet.base.role_maker
import
GeneralRoleMaker
from
paddle.fluid.incubate.fleet.base.role_maker
import
GeneralRoleMaker
from
fleet
_
rec.utils
import
fs
as
fs
from
fleetrec.utils
import
fs
as
fs
from
fleet
_
rec.utils
import
util
as
util
from
fleetrec.utils
import
util
as
util
from
fleet
_
rec.metrics.auc_metrics
import
AUCMetric
from
fleetrec.metrics.auc_metrics
import
AUCMetric
from
fleet
_
rec.models
import
base
as
model_basic
from
fleetrec.models
import
base
as
model_basic
from
fleet
_
rec.reader
import
dataset
from
fleetrec.reader
import
dataset
from
.trainer
import
Trainer
from
.trainer
import
Trainer
...
...
trainer/factory.py
→
fleetrec/
trainer/factory.py
浏览文件 @
0bfba555
...
@@ -29,16 +29,16 @@ import sys
...
@@ -29,16 +29,16 @@ import sys
import
yaml
import
yaml
from
fleet
_
rec.trainer.single_trainer
import
SingleTrainerWithDataloader
from
fleetrec.trainer.single_trainer
import
SingleTrainerWithDataloader
from
fleet
_
rec.trainer.single_trainer
import
SingleTrainerWithDataset
from
fleetrec.trainer.single_trainer
import
SingleTrainerWithDataset
from
fleet
_
rec.trainer.cluster_trainer
import
ClusterTrainerWithDataloader
from
fleetrec.trainer.cluster_trainer
import
ClusterTrainerWithDataloader
from
fleet
_
rec.trainer.cluster_trainer
import
ClusterTrainerWithDataset
from
fleetrec.trainer.cluster_trainer
import
ClusterTrainerWithDataset
from
fleet
_
rec.trainer.local_engine
import
Launch
from
fleetrec.trainer.local_engine
import
Launch
from
fleet
_
rec.trainer.ctr_trainer
import
CtrPaddleTrainer
from
fleetrec.trainer.ctr_trainer
import
CtrPaddleTrainer
from
fleet
_
rec.utils
import
envs
from
fleetrec.utils
import
envs
def
str2bool
(
v
):
def
str2bool
(
v
):
...
@@ -103,7 +103,7 @@ class TrainerFactory(object):
...
@@ -103,7 +103,7 @@ class TrainerFactory(object):
with
open
(
config
,
'r'
)
as
rb
:
with
open
(
config
,
'r'
)
as
rb
:
_config
=
yaml
.
load
(
rb
.
read
(),
Loader
=
yaml
.
FullLoader
)
_config
=
yaml
.
load
(
rb
.
read
(),
Loader
=
yaml
.
FullLoader
)
else
:
else
:
raise
ValueError
(
"fleet
_
rec's config only support yaml"
)
raise
ValueError
(
"fleetrec's config only support yaml"
)
envs
.
set_global_envs
(
_config
)
envs
.
set_global_envs
(
_config
)
mode
=
envs
.
get_global_env
(
"train.trainer"
)
mode
=
envs
.
get_global_env
(
"train.trainer"
)
...
...
trainer/local_engine.py
→
fleetrec/
trainer/local_engine.py
浏览文件 @
0bfba555
...
@@ -38,7 +38,7 @@ def start_procs(args, yaml):
...
@@ -38,7 +38,7 @@ def start_procs(args, yaml):
user_endpoints_ips
=
[
x
.
split
(
":"
)[
0
]
for
x
in
user_endpoints
.
split
(
","
)]
user_endpoints_ips
=
[
x
.
split
(
":"
)[
0
]
for
x
in
user_endpoints
.
split
(
","
)]
user_endpoints_port
=
[
x
.
split
(
":"
)[
1
]
for
x
in
user_endpoints
.
split
(
","
)]
user_endpoints_port
=
[
x
.
split
(
":"
)[
1
]
for
x
in
user_endpoints
.
split
(
","
)]
factory
=
"fleet
_
rec.trainer.factory"
factory
=
"fleetrec.trainer.factory"
cmd
=
[
sys
.
executable
,
"-u"
,
"-m"
,
factory
,
yaml
]
cmd
=
[
sys
.
executable
,
"-u"
,
"-m"
,
factory
,
yaml
]
for
i
in
range
(
server_num
):
for
i
in
range
(
server_num
):
...
...
trainer/single_trainer.py
→
fleetrec/
trainer/single_trainer.py
浏览文件 @
0bfba555
文件已移动
trainer/trainer.py
→
fleetrec/
trainer/trainer.py
浏览文件 @
0bfba555
文件已移动
trainer/transpiler_trainer.py
→
fleetrec/
trainer/transpiler_trainer.py
浏览文件 @
0bfba555
文件已移动
utils/__init__.py
→
fleetrec/
utils/__init__.py
浏览文件 @
0bfba555
文件已移动
utils/envs.py
→
fleetrec/
utils/envs.py
浏览文件 @
0bfba555
...
@@ -70,7 +70,7 @@ def pretty_print_envs(envs, header=None):
...
@@ -70,7 +70,7 @@ def pretty_print_envs(envs, header=None):
if
header
:
if
header
:
draws
+=
h_format
.
format
(
header
[
0
],
header
[
1
])
draws
+=
h_format
.
format
(
header
[
0
],
header
[
1
])
else
:
else
:
draws
+=
h_format
.
format
(
"fleet
_
rec Global Envs"
,
"Value"
)
draws
+=
h_format
.
format
(
"fleetrec Global Envs"
,
"Value"
)
draws
+=
line
+
"
\n
"
draws
+=
line
+
"
\n
"
...
...
utils/fs.py
→
fleetrec/
utils/fs.py
浏览文件 @
0bfba555
文件已移动
utils/table.py
→
fleetrec/
utils/table.py
浏览文件 @
0bfba555
文件已移动
utils/util.py
→
fleetrec/
utils/util.py
浏览文件 @
0bfba555
文件已移动
编辑
预览
Markdown
is supported
0%
请重试
或
添加新附件
.
添加附件
取消
You are about to add
0
people
to the discussion. Proceed with caution.
先完成此消息的编辑!
取消
想要评论请
注册
或
登录