提交 0ec8775b 编写于 作者: T tangwei12

add cloud

上级 b58041de
...@@ -74,3 +74,6 @@ python infer.py \ ...@@ -74,3 +74,6 @@ python infer.py \
1. 用preprocess.py处理训练数据生成train.txt。 1. 用preprocess.py处理训练数据生成train.txt。
1. 将train.txt切分成集群机器份,放到每台机器上。 1. 将train.txt切分成集群机器份,放到每台机器上。
1. 用上面的 `分布式训练` 中的命令行启动分布式训练任务. 1. 用上面的 `分布式训练` 中的命令行启动分布式训练任务.
## 在PaddleCloud上运行集群训练
如果你正在使用PaddleCloud做集群训练,你可以使用```cloud.py```这个文件来帮助你提交任务,```trian.py```中所需要的参数可以通过PaddleCloud的环境变量来提交。
\ No newline at end of file
...@@ -91,3 +91,6 @@ Note: The AUC value in the last log info is the total AUC for all test dataset. ...@@ -91,3 +91,6 @@ Note: The AUC value in the last log info is the total AUC for all test dataset.
1. Prepare dataset using preprocess.py. 1. Prepare dataset using preprocess.py.
1. Split the train.txt to trainer_num parts and put them on the machines. 1. Split the train.txt to trainer_num parts and put them on the machines.
1. Run training with the cluster train using the command in `Distributed Train` above. 1. Run training with the cluster train using the command in `Distributed Train` above.
## Train on Paddle Cloud
If you want to run this training on PaddleCloud, you can use the script ```cloud.py```, you can change the arguments in ```trian.py``` through environments in PaddleCloud.
\ No newline at end of file
...@@ -5,12 +5,13 @@ ...@@ -5,12 +5,13 @@
# Copyright (c) 2017 Baidu.com, Inc. All Rights Reserved # Copyright (c) 2017 Baidu.com, Inc. All Rights Reserved
# #
# ====================================================================== # ======================================================================
"""this file is only for PaddleCloud"""
import os import os
import logging import logging
import hdfs_utils import paddle.fluid.contrib.utils.hdfs_utils as hdfs_utils
logging.basicConfig( logging.basicConfig(
format='%(asctime)s - %(levelname)s - %(message)s') format='%(asctime)s - %(levelname)s - %(message)s')
......
Markdown is supported
0% .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑!
想要评论请 注册