提交 0ec8775b 编写于 作者: T tangwei12

add cloud

上级 b58041de
...@@ -74,3 +74,6 @@ python infer.py \ ...@@ -74,3 +74,6 @@ python infer.py \
1. 用preprocess.py处理训练数据生成train.txt。 1. 用preprocess.py处理训练数据生成train.txt。
1. 将train.txt切分成集群机器份,放到每台机器上。 1. 将train.txt切分成集群机器份,放到每台机器上。
1. 用上面的 `分布式训练` 中的命令行启动分布式训练任务. 1. 用上面的 `分布式训练` 中的命令行启动分布式训练任务.
## 在PaddleCloud上运行集群训练
如果你正在使用PaddleCloud做集群训练,你可以使用```cloud.py```这个文件来帮助你提交任务,```trian.py```中所需要的参数可以通过PaddleCloud的环境变量来提交。
\ No newline at end of file
...@@ -90,4 +90,7 @@ Note: The AUC value in the last log info is the total AUC for all test dataset. ...@@ -90,4 +90,7 @@ Note: The AUC value in the last log info is the total AUC for all test dataset.
1. Please prepare some CPU machines on Baidu Cloud following the steps in [train_on_baidu_cloud](https://github.com/PaddlePaddle/FluidDoc/blob/develop/doc/fluid/user_guides/howto/training/train_on_baidu_cloud_cn.rst) 1. Please prepare some CPU machines on Baidu Cloud following the steps in [train_on_baidu_cloud](https://github.com/PaddlePaddle/FluidDoc/blob/develop/doc/fluid/user_guides/howto/training/train_on_baidu_cloud_cn.rst)
1. Prepare dataset using preprocess.py. 1. Prepare dataset using preprocess.py.
1. Split the train.txt to trainer_num parts and put them on the machines. 1. Split the train.txt to trainer_num parts and put them on the machines.
1. Run training with the cluster train using the command in `Distributed Train` above. 1. Run training with the cluster train using the command in `Distributed Train` above.
\ No newline at end of file
## Train on Paddle Cloud
If you want to run this training on PaddleCloud, you can use the script ```cloud.py```, you can change the arguments in ```trian.py``` through environments in PaddleCloud.
\ No newline at end of file
...@@ -5,12 +5,13 @@ ...@@ -5,12 +5,13 @@
# Copyright (c) 2017 Baidu.com, Inc. All Rights Reserved # Copyright (c) 2017 Baidu.com, Inc. All Rights Reserved
# #
# ====================================================================== # ======================================================================
"""this file is only for PaddleCloud"""
import os import os
import logging import logging
import hdfs_utils import paddle.fluid.contrib.utils.hdfs_utils as hdfs_utils
logging.basicConfig( logging.basicConfig(
format='%(asctime)s - %(levelname)s - %(message)s') format='%(asctime)s - %(levelname)s - %(message)s')
......
Markdown is supported
0% .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑!
想要评论请 注册