未验证 提交 db2c1529 编写于 作者: D Daniel Yang 提交者: GitHub

this is a pull-request to revise the docs, test=develop (#1790)

上级 e2841c89
.. _addon_development:
.. _addon_development:
########
二次开发
########
.. toctree::
:hidden:
:maxdepth: 1
design_idea/fluid_design_idea.md
new_op/index_cn.rst
contribute_code/index_cn.rst
contribute-docs/write_docs_cn.md
......@@ -6,11 +6,11 @@ Addon Development
################
.. toctree::
:hidden:
:maxdepth: 1
design_idea/fluid_design_idea_en.md
new_op/index_en.rst
contribute_code/index_en.rst
contribute-docs/write_docs_en.md
......@@ -2,6 +2,8 @@
Write New Operators
###################
This section will guide you how to add an operator, and it also includes some necessary notes.
- `How to write new operator <new_op_en.html>`_ :guides to write new operators
- `op notes <op_notes_en.html>`_ :notes on developing new operators
......
.. _user_guide_distribute:
##########
分布式训练
##########
......
......@@ -9,5 +9,4 @@ Distributed Training
cluster_quick_start_en.rst
cluster_howto_en.rst
train_on_baidu_cloud_en.rst
############
模型评估和调试
############
PaddlePaddle Fluid提供了常用的模型评估指标,并提供了VisualDL工具可视化模型效果。
.. toctree::
:maxdepth: 2
metrics
......@@ -4,9 +4,9 @@
本部分包括两篇文档:
- `模型评估 <../evaluation_and_debugging/evaluation/metrics.html>`_:介绍常用模型评估指标的构造方法
- `模型评估 <evaluation/metrics.html>`_:介绍常用模型评估指标的构造方法
- `Visual DL 工具 <../evaluation_and_debugging/debug/visualdl.html>`_:介绍如何利用 Visual DL 工具可视化训练过程
- `Visual DL 工具 <debug/visualdl.html>`_:介绍如何利用 Visual DL 工具可视化训练过程
.. toctree::
:hidden:
......
......@@ -6,9 +6,9 @@ Model Evaluation and Debugging
There are two articles in this section:
- `Model Evaluation <../evaluation_and_debugging/evaluation/metrics_en.html>`_:This section will introduce the construction of common metrics.
- `Model Evaluation <evaluation/metrics_en.html>`_:This section will introduce the construction of common metrics.
- `Visual DL <../evaluation_and_debugging/debug/visualdl_en.html>`_:How to use Visual DL to visualise training process.
- `VisualDL Tools <debug/index_en.html>`_:How to use Visual DL to visualise training process.
.. toctree::
:hidden:
......
.. _user_guide_:
########
进阶指南
########
......@@ -14,9 +12,9 @@
- `性能调优 <../advanced_guide/performance_improving/index_cn.html>`_ :介绍飞桨使用过程中的调优方法
- `模型评估调试 <../advanced_guide/evaluation_debugging/index_cn.html>`_ :介绍模型评估与调试的典型方法
- `模型评估/调试 <../advanced_guide/evaluation_debugging/index_cn.html>`_ :介绍模型评估与调试的典型方法
- `二次开发 <../advanced_guide/addon_development/index_cn.html>`_ :介绍如何新增Operator和如何向飞桨社区贡献代码和文档
- `二次开发 <../advanced_guide/addon_development/index_cn.html>`_ :介绍如何新增Operator和如何向飞桨开源社区贡献代码
- `环境变量FLAGS <../advanced_guide/flags/index_cn.html>`_
......
......@@ -8,19 +8,19 @@ Advanced User Guides
So far you have already been familiar with PaddlePaddle. And the next expectation, read more on:
- `Data Preparing <../advanced_guide/data_preparing/index_cn.html>`_:How to prepare the data efficently.
- `Prepare Data <data_preparing/index_en.html>`_:How to prepare the data efficiently.
- `Distributed Training <../advanced_guide/distributed_training/index_cn.html>`_ :How to apply the distributed training in your projects.
- `Distributed Training <distributed_training/index_en.html>`_ :How to apply the distributed training in your projects.
- `Deploy Inference Model <../advanced_guide/inference_deployment/index_cn.html>`_ :How to deploy the trained network to perform practical inference
- `Deploy Inference Model <inference_deployment/index_en.html>`_ :How to deploy the trained network to perform practical inference
- `Performance Profiling <../advanced_guide/performance_improving/index_cn.html>`_ :How to do profiling for Fluid programs
- `Practice Improving <performance_improving/index_en.html>`_ :How to do profiling for Fluid programs
- `Model Evalution <../advanced_guide/evaluation_debugging/index_cn.html>`_ :How to evalute your program.
- `Model Evaluation and Debugging <evaluation_debugging/index_en.html>`_ :How to evaluate your program.
- `Add on development <../advanced_guide/addon_development/index_cn.html>`_ :How to contribute codes and documentation to our communities
- `Addon Development <addon_development/index_en.html>`_ :How to contribute codes and documentation to our communities
- `Env FLAGS <../advanced_guide/flags/index_cn.html>`_
- `FLAGS <flags_en.html>`_
.. toctree::
......
......@@ -2,9 +2,9 @@
预测部署
########
- `服务器端部署 <inference/index_cn.html>`_ :介绍了支持模型部署上线的Fluid C++ API
- `服务器端部署 <inference/index_cn.html>`_ :介绍了如何在服务器端将模型部署上线
- `移动端部署 <mobile/index_cn.html>`_:介绍了 PaddlePaddle组织下的嵌入式平台深度学习框架Paddle-Lite
- `移动端部署 <mobile/index_cn.html>`_:介绍了 PaddlePaddle 组织下的嵌入式平台深度学习框架Paddle-Lite
.. toctree::
:hidden:
......
......@@ -2,11 +2,10 @@
Deploy Inference Model
#######################
- `Server side Deployment <inference/index_en.html>`_ : This section illustrates Fluid C++ API to support deployment and release of trained models.
- `Server side Deployment <inference/index_en.html>`_ : This section illustrates the method how to deploy and release the trained models on the servers
- `Paddle Lite <mobile/index_en.html>`_ : Embedded deep learning framework Paddle-Lite organized by PaddlePaddle.
.. toctree::
:hidden:
inference/index_en.rst
inference/index_en.rst
\ No newline at end of file
.. _user_guide_inference:
############
服务器端部署
############
......
......@@ -2,7 +2,7 @@
Server-side Deployment
######################
PaddlePaddle Fluid provides C++ API to support deployment and release of trained models.
PaddlePaddle provides various methods to support deployment and release of trained models.
.. toctree::
:titlesonly:
......@@ -10,5 +10,4 @@ PaddlePaddle Fluid provides C++ API to support deployment and release of trained
build_and_install_lib_en.rst
windows_cpp_inference_en.md
native_infer_en.md
paddle_tensorrt_infer_en.md
paddle_gpu_benchmark_en.md
.. _user_guide_mobile:
##########
移动端部署
##########
本模块介绍了飞桨的端侧推理引擎Paddle-Lite以及在模型压缩工具PaddleSlim,包括:
* `项目简介 <mobile_index.html>`_:简要介绍了 Paddle-Lite 特点以及使用说明。
* `Paddle Lite <mobile_index.html>`_:简要介绍了 Paddle-Lite 特点以及使用说明。
* `项目简介 <paddle_slim.html>`_:简要介绍了PaddleSlim 特点以及使用说明。
* `PaddleSlim <paddle_slim.html>`_:简要介绍了PaddleSlim 特点以及使用说明。
.. toctree::
:hidden:
......
##########
性能调优
##########
###############
性能优化分析及工具
###############
.. toctree::
:hidden:
......
.. _performance_improving_:
########
性能调优
########
......
......@@ -3,7 +3,7 @@ Practice Improving
###############
.. toctree::
:hidden:
:maxdepth: 1
multinode_training_improving/cpu_train_best_practice_en.rst
......
......@@ -2,16 +2,16 @@
基本概念
############
本文介绍 Paddle 中的基本概念:
本文介绍飞桨核心框架中的基本概念:
- `编程指南 <./programming_guide/programming_guide.html>`_ : 介绍 Paddle 的基本概念和使用方法。
- `Variable <variable.html>`_ : Variable表示变量,在Paddle中可以包含任何类型的值,在大多数情况下是一个Lod-Tensor。
- `编程指南 <./programming_guide/programming_guide.html>`_ : 介绍飞桨的基本概念和使用方法。
- `Variable <variable.html>`_ : Variable表示变量,在飞桨中可以包含任何类型的值,在大多数情况下是一个Lod-Tensor。
- `Tensor <tensor.html>`_ : Tensor表示数据。
- `LoD-Tensor <lod_tensor.html>`_ : LoD-Tensor是Paddle的高级特性,它在Tensor基础上附加了序列信息,支持处理变长数据。
- `LoD-Tensor <lod_tensor.html>`_ : LoD-Tensor是飞桨的高级特性,它在Tensor基础上附加了序列信息,支持处理变长数据。
- `Operator <operator.html>`_ : Operator表示对数据的操作。
- `Program <program.html>`_ : Program表示对计算过程的描述。
- `Executor <executor.html>`_ : Executor表示执行引擎。
- `DyGraph模式 <./dygraph/DyGraph.html>`_ : Executor表示执行引擎
- `动态图机制-DyGraph <./dygraph/DyGraph.html>`_ : 介绍飞桨动态图执行机制
.. toctree::
:hidden:
......
......@@ -322,6 +322,6 @@ with fluid.layers.control_flow.Switch() as switch:
完成网络搭建后,可以开始在单机上训练您的网络了,详细步骤请参考[单机训练](../../coding_practice/single_node.html)
除此之外,使用文档模块根据开发者的不同背景划分了三个学习阶段:[快速入门](../../index_cn.html)[典型案例](../../../user_guides/index_cn.html)[进阶指南](../../../advanced_guide/index_cn.html)
除此之外,使用文档模块根据开发者的不同背景划分了三个学习阶段:[快速上手](../../index_cn.html)[典型案例](../../../user_guides/index_cn.html)[进阶指南](../../../advanced_guide/index_cn.html)
如果您希望阅读更多场景下的应用案例,可以参考[典型案例](../../../user_guides/index_cn.html)。已经具备深度学习基础知识的用户,也可以从[进阶指南](../../../advanced_guide/index_cn.html)开始阅读。
......@@ -2,17 +2,17 @@
快速上手
########
PaddlePaddle (PArallel Distributed Deep LEarning)是一个易用、高效、灵活、可扩展的深度学习框架
PaddlePaddle (PArallel Distributed Deep LEarning)是一个易用、高效、灵活、可扩展的深度学习框架
您可参考PaddlePaddle的 `Github <https://github.com/PaddlePaddle/Paddle>`_ 了解详情,也可阅读 `版本说明 <../release_note_cn.html>`_ 了解新版本的特性
您可参考PaddlePaddle的 `Github <https://github.com/PaddlePaddle/Paddle>`_ 了解详情,也可阅读 `版本说明 <../release_note_cn.html>`_ 了解新版本的特性
让我们从学习PaddlePaddle基本概念这里开始:
- `基本概念 <../beginners_guide/basic_concept/index_cn.html>`_:介绍 Paddle的基本概念和使用方法
如果您已经掌握了飞桨的基本概念,期望可以针对实际问题建模、搭建自己网络,编程实践中提供了一些 Paddle 的使用细节供您参考:
如果您已经掌握了飞桨的基本概念,期望可以针对实际问题建模、搭建自己网络,编程实践中提供了一些 Paddle 的使用细节供您参考:
- `编程实践 <../beginners_guide/coding_practice/index_cn.html>`_
- `编程实践 <../beginners_guide/coding_practice/index_cn.html>`_:介绍如何针对实际问题建模、搭建自己网络
.. toctree::
......
......@@ -5,16 +5,15 @@ Beginner's Guide
PaddlePaddle (PArallel Distributed Deep LEarning) is a
simple, efficient and extensible deep learning framework.
Please refer to `PaddlePaddle Github <https://github.com/PaddlePaddle/Paddle>`_ for details, and `release note <../release_note_en.html>`_ for features incorporated in current version.
Please refer to `PaddlePaddle Github <https://github.com/PaddlePaddle/Paddle>`_ for details, and `Release Note <../release_note_en.html>`_ for features incorporated in current version.
For beginners of PaddlePaddle, the following documentation will tutor you about installing PaddlePaddle:
Let's start with studying basic concept of PaddlePaddle:
- `Installation Manuals <../beginners_guide/install/index_en.html>`_ :Installation on Ubuntu/CentOS/Windows/MacOS is supported.
- `Basic Concept <../beginners_guide/basic_concept/index_en.html>`_ : introduce the basic concept and usage of Paddle
If you have been armed with certain level of deep learning knowledge, and it happens to be the first time to try PaddlePaddle, the following cases of model building will expedite your learning process:
If you have mastered the basic concept of Paddle and you expect to model and build your own network according to the actual problems, you can refer to some details of the use of paddle in the Coding Practice :
- `Programming with Fluid <../beginners_guide/programming_guide/programming_guide_en.html>`_ : Core concepts and basic usage of Fluid
- `Deep Learning Basics <../beginners_guide/basics/index_en.html>`_: This section encompasses various fields of fundamental deep learning knowledge, such as image classification, customized recommendation, machine translation, and examples implemented by Fluid are provided.
- `Coding Practice <../beginners_guide/coding_practice/index_en.html>`_ : introduce how to model and build your own network for practical problems
.. toctree::
......
......@@ -16,4 +16,4 @@
advanced_guide/index_cn.rst
api_cn/index_cn.rst
faq/index_cn.rst
release_note_en.rst
release_note_cn.md
......@@ -10,4 +10,4 @@
advanced_guide/index_en.rst
api/index_en.rst
faq/index_en.rst
release_note_en.rst
\ No newline at end of file
release_note_en.md
# **使用conda安装**
# **使用Conda安装**
[Anaconda](https://www.anaconda.com/)是一个免费开源的Python和R语言的发行版本,用于计算科学,Anaconda致力于简化包管理和部署。Anaconda的包使用软件包管理系统Conda进行管理。Conda是一个开源包管理系统和环境管理系统,可在Windows、macOS和Linux上运行。
......
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
......@@ -3,80 +3,8 @@
################
本章由1篇文档组成,将指导您如何使用PaddlePaddle完成基础的计算机视觉深度学习任务
本章文档涉及大量了深度学习基础知识,也介绍了如何使用PaddlePaddle实现这些内容,请参阅以下说明了解如何使用:
内容简介
======================
您现在在看的这本书是一本“交互式”电子书 —— 每一章都可以运行在一个Jupyter Notebook里。
.. toctree::
:titlesonly:
gan/README.cn.md
我们把Jupyter、PaddlePaddle、以及各种被依赖的软件都打包进一个Docker image了。所以您不需要自己来安装各种软件,只需要安装Docker即可。对于各种Linux发行版,请参考 https://www.docker.com 。如果您使用 `Windows <https://www.docker.com/docker-windows>`_ 或者 `Mac <https://www.docker.com/docker-mac>`_,可以考虑 `给Docker更多内存和CPU资源 <http://stackoverflow.com/a/39720010/724872>`_ 。
使用方法
======================
本书默认使用CPU训练,若是要使用GPU训练,使用步骤会稍有变化,请参考下文“使用GPU训练”
使用CPU训练
>>>>>>>>>>>>
只需要在命令行窗口里运行:
.. code-block:: shell
docker run -d -p 8888:8888 paddlepaddle/book
即可从DockerHub.com下载和运行本书的Docker image。阅读和在线编辑本书请在浏览器里访问 http://localhost:8888
如果您访问DockerHub.com很慢,可以试试我们的另一个镜像docker.paddlepaddlehub.com:
::
docker run -d -p 8888:8888 docker.paddlepaddlehub.com/book
使用GPU训练
>>>>>>>>>>>>>
为了保证GPU驱动能够在镜像里面正常运行,我们推荐使用 `nvidia-docker <https://github.com/NVIDIA/nvidia-docker>`_ 来运行镜像。请先安装nvidia-docker,之后请运行:
::
nvidia-docker run -d -p 8888:8888 paddlepaddle/book:latest-gpu
或者使用国内的镜像请运行:
::
nvidia-docker run -d -p 8888:8888 docker.paddlepaddlehub.com/book:latest-gpu
还需要将以下代码
.. code-block:: python
use_cuda = False
改成:
.. code-block:: python
use_cuda = True
贡献新章节
=============
您要是能贡献新的章节那就太好了!请发Pull Requests把您写的章节加入到 :code:`pending` 下面的一个子目录里。当这一章稳定下来,我们一起把您的目录挪到根目录。
为了写作、运行、调试,您需要安装Python 2.x和Go >1.5, 并可以用 `脚本程序 <https://github.com/PaddlePaddle/book/blob/develop/.tools/convert-markdown-into-ipynb-and-test.sh>`_ 来生成新的Docker image。
**Please Note:** We also provide `English Readme <https://github.com/PaddlePaddle/book/blob/develop/README.md>`_ for PaddlePaddle book
......@@ -2,92 +2,9 @@
Computer Vision
############################
This section collects 1 document arranging from the simplest to the most challenging, which will guide you through the basic deep learning tasks in PaddlePaddle.
The documentation in this chapter covers a lot of deep learning basics and how to implement them with PaddlePaddle. See the instructions below for how to use:
Overview
======================
The book you are reading is an "interactive" e-book - each chapter can be run in a Jupyter Notebook.
.. toctree::
:titlesonly:
gan/README.md
We packaged Jupyter, PaddlePaddle, and various dependency softwares into a Docker image. It frees you from installing these softwares by yourself, and you only need to just install Docker. For various Linux versions, please refer to https://www.docker.com . If you use docker on `Windows <https://www.docker.com/docker-windows>`_ or `Mac <https://www.docker.com/docker-mac>`_ , consider `allocate more Memory and CPU resources to Docker <http://stackoverflow.com/a/39720010/724872>`_ .
Instructions
======================
This book assumes you are performing CPU training by default. If you want to use GPU training, the steps will vary slightly. Please refer to "GPU Training" below.
CPU training
>>>>>>>>>>>>
Just run these in shell:
.. code-block:: shell
docker run -d -p 8888:8888 paddlepaddle/book
It downloads the Docker image for running books from DockerHub.com.
To read and edit this book on-line, please visit http://localhost:8888 in your browser.
If the Internet connection to DockerHub.com is compromised, try our spare docker image named docker.paddlepaddlehub.com:
::
docker run -d -p 8888:8888 docker.paddlepaddlehub.com/book
GPU training
>>>>>>>>>>>>>
To ensure that the GPU driver works properly in the image, we recommend running the image with `nvidia docker <https://github.com/NVIDIA/nvidia-docker>`_ . Please install nvidia-docker first, then run:
::
nvidia-docker run -d -p 8888:8888 paddlepaddle/book:latest-gpu
Or use a image source in China to run:
::
nvidia-docker run -d -p 8888:8888 docker.paddlepaddlehub.com/book:latest-gpu
modify the following codes
.. code-block:: python
use_cuda = False
into :
.. code-block:: python
use_cuda = True
Contribute to Book
===================
We highly appreciate your original contributions of new chapters to Book! Just Pull Requests of your contributions to the sub-directory in :code:`pending` . When this chapter is endorsed, we'll gladly move it to the root directory.
For writing, running, debugging, you need to install `shell <https://github.com/PaddlePaddle/book/blob/develop/.tools/convert-markdown-into-ipynb-and-test.sh>`_ to generate Docker image。
**Please Note:** We also provide `English Readme <https://github.com/PaddlePaddle/book/blob/develop/README.md>`_ for PaddlePaddle book
......@@ -6,13 +6,23 @@
如果您已经掌握了快速上手阶段的内容,期望可以针对实际问题建模、搭建自己网络,本模块提供了一些 Paddle 的具体典型案例供您参考:
本章文档将指导您如何使用PaddlePaddle完成基础的深度学习任务
本章文档涉及大量了深度学习基础知识,也介绍了如何使用PaddlePaddle实现这些内容,请参阅以下说明了解如何使用:
内容简介
======================
- `简单案例 <../user_guides/simple_case/index_cn.html>`_ :介绍了 Paddle 的基本案例
- `计算机视觉 <../user_guides/cv_case/index_cn.html>`_ :介绍使用 Paddle 解决计算机视觉领域的案例
- `自然语言处理 <../user_guides/nlp_case/index_cn.html>`_: 介绍使用 Paddle 实现自语言处理方向的案例
- `自然语言处理 <../user_guides/nlp_case/index_cn.html>`_: 介绍使用 Paddle 实现自语言处理方向的案例
- `推荐 <../user_guides/rec_case/index_cn.html>`_:介绍如何使用 Paddle 完成推荐领域任务的案例
- `模型库 <../user_guides/models/index_cn.html>`_:介绍了 Paddle 经典的模型库
- `工具组件 <../user_guides/tools/index_cn.html>`_:介绍在 Paddle 工具组件的使用案例
......@@ -23,4 +33,71 @@
cv_case/index_cn.rst
nlp_case/index_cn.rst
rec_case/index_cn.rst
models/index_cn.rst
tools/index_cn.rst
我们把Jupyter、PaddlePaddle、以及各种被依赖的软件都打包进一个Docker image了。所以您不需要自己来安装各种软件,只需要安装Docker即可。对于各种Linux发行版,请参考 https://www.docker.com 。如果您使用 `Windows <https://www.docker.com/docker-windows>`_ 或者 `Mac <https://www.docker.com/docker-mac>`_,可以考虑 `给Docker更多内存和CPU资源 <http://stackoverflow.com/a/39720010/724872>`_ 。
使用方法
======================
本书默认使用CPU训练,若是要使用GPU训练,使用步骤会稍有变化,请参考下文“使用GPU训练”
使用CPU训练
>>>>>>>>>>>>
只需要在命令行窗口里运行:
.. code-block:: shell
docker run -d -p 8888:8888 paddlepaddle/book
即可从DockerHub.com下载和运行本书的Docker image。阅读和在线编辑本书请在浏览器里访问 http://localhost:8888
如果您访问DockerHub.com很慢,可以试试我们的另一个镜像docker.paddlepaddlehub.com:
::
docker run -d -p 8888:8888 docker.paddlepaddlehub.com/book
使用GPU训练
>>>>>>>>>>>>>
为了保证GPU驱动能够在镜像里面正常运行,我们推荐使用 `nvidia-docker <https://github.com/NVIDIA/nvidia-docker>`_ 来运行镜像。请先安装nvidia-docker,之后请运行:
::
nvidia-docker run -d -p 8888:8888 paddlepaddle/book:latest-gpu
或者使用国内的镜像请运行:
::
nvidia-docker run -d -p 8888:8888 docker.paddlepaddlehub.com/book:latest-gpu
还需要将以下代码
.. code-block:: python
use_cuda = False
改成:
.. code-block:: python
use_cuda = True
贡献新章节
=============
您要是能贡献新的章节那就太好了!请发Pull Requests把您写的章节加入到 :code:`pending` 下面的一个子目录里。当这一章稳定下来,我们一起把您的目录挪到根目录。
为了写作、运行、调试,您需要安装Python 2.x和Go >1.5, 并可以用 `脚本程序 <https://github.com/PaddlePaddle/book/blob/develop/.tools/convert-markdown-into-ipynb-and-test.sh>`_ 来生成新的Docker image。
**Please Note:** We also provide `English Readme <https://github.com/PaddlePaddle/book/blob/develop/README.md>`_ for PaddlePaddle book
......@@ -8,19 +8,22 @@ If you have got the hang of Beginner's Guide, and wish to model practical proble
you with some detailed operations:
- `LoD-Tensor Concepts <../user_guides/howto/basic_concept/index_en.html>`_ :It explains basic concepts of Fluid LoD-Tensor.
This section collects several documents arranging from the simplest to the most challenging, which will guide you through the basic deep learning tasks in PaddlePaddle.
- `Prepare Data <../user_guides/howto/prepare_data/index_en.html>`_ :This section introduces data types supported and data transmission methods when you are training your networks with Fluid.
The documentation in this chapter covers a lot of deep learning basics and how to implement them with PaddlePaddle. See the instructions below for how to use:
- `Set up Simple Model <../user_guides/howto/configure_simple_model/index_en.html>`_: This section illustrates how to model practical problems and build networks with related operators of Fluid.
- `Train Neural Networks <../user_guides/howto/training/index_en.html>`_:This section will guide you to perform single-node training, multi-node training, and save or load model variables.
Overview
======================
- `Model Evaluation and Debugging <../user_guides/howto/evaluation_and_debugging/index_en.html>`_:It introduces the model evaluation and debugging methods in Fluid
- `Simple Case <../user_guides/simple_case/index_en.html>`_ :introduces basic cases of Paddle
Reproduced classic models of multiple directions in Fluid:
- `Natural Language Processing <../user_guides/nlp_case/index_en.html>`_:introduces cases of using paddle to realize Natural Language Processing tasks
- `Fluid Model Library <../user_guides/models/index_en.html>`_
- `Recommend <../user_guides/rec_case/index_en.html>`_:introduces cases of using paddle to realize Recommend tasks
- `Models Zoo <../user_guides/models/index_en.html>`_:introduces the models zoo of Paddle
.. toctree::
:hidden:
......@@ -28,4 +31,80 @@ Reproduced classic models of multiple directions in Fluid:
simple_case/index_en.rst
nlp_case/index_en.rst
rec_case/index_en.rst
tools/index_en.rst
models/index_cn.rst
We packaged Jupyter, PaddlePaddle, and various dependency softwares into a Docker image. It frees you from installing these softwares by yourself, and you only need to just install Docker. For various Linux versions, please refer to https://www.docker.com . If you use docker on `Windows <https://www.docker.com/docker-windows>`_ or `Mac <https://www.docker.com/docker-mac>`_ , consider `allocate more Memory and CPU resources to Docker <http://stackoverflow.com/a/39720010/724872>`_ .
Instructions
======================
This book assumes you are performing CPU training by default. If you want to use GPU training, the steps will vary slightly. Please refer to "GPU Training" below.
CPU training
>>>>>>>>>>>>
Just run these in shell:
.. code-block:: shell
docker run -d -p 8888:8888 paddlepaddle/book
It downloads the Docker image for running books from DockerHub.com.
To read and edit this book on-line, please visit http://localhost:8888 in your browser.
If the Internet connection to DockerHub.com is compromised, try our spare docker image named docker.paddlepaddlehub.com:
::
docker run -d -p 8888:8888 docker.paddlepaddlehub.com/book
GPU training
>>>>>>>>>>>>>
To ensure that the GPU driver works properly in the image, we recommend running the image with `nvidia docker <https://github.com/NVIDIA/nvidia-docker>`_ . Please install nvidia-docker first, then run:
::
nvidia-docker run -d -p 8888:8888 paddlepaddle/book:latest-gpu
Or use a image source in China to run:
::
nvidia-docker run -d -p 8888:8888 docker.paddlepaddlehub.com/book:latest-gpu
modify the following codes
.. code-block:: python
use_cuda = False
into :
.. code-block:: python
use_cuda = True
Contribute to Book
===================
We highly appreciate your original contributions of new chapters to Book! Just Pull Requests of your contributions to the sub-directory in :code:`pending` . When this chapter is endorsed, we'll gladly move it to the root directory.
For writing, running, debugging, you need to install `shell <https://github.com/PaddlePaddle/book/blob/develop/.tools/convert-markdown-into-ipynb-and-test.sh>`_ to generate Docker image。
**Please Note:** We also provide `English Readme <https://github.com/PaddlePaddle/book/blob/develop/README.md>`_ for PaddlePaddle book
......@@ -2,16 +2,6 @@
自然语言处理
################
本章由3篇文档组成,将指导您如何使用PaddlePaddle完成自然语言处理领域的基础深度学习任务
本章文档涉及大量了深度学习基础知识,也介绍了如何使用PaddlePaddle实现这些内容,请参阅以下说明了解如何使用:
内容简介
======================
您现在在看的这本书是一本“交互式”电子书 —— 每一章都可以运行在一个Jupyter Notebook里。
.. toctree::
:titlesonly:
......@@ -19,67 +9,3 @@
label_semantic_roles/README.cn.md
machine_translation/README.cn.md
我们把Jupyter、PaddlePaddle、以及各种被依赖的软件都打包进一个Docker image了。所以您不需要自己来安装各种软件,只需要安装Docker即可。对于各种Linux发行版,请参考 https://www.docker.com 。如果您使用 `Windows <https://www.docker.com/docker-windows>`_ 或者 `Mac <https://www.docker.com/docker-mac>`_,可以考虑 `给Docker更多内存和CPU资源 <http://stackoverflow.com/a/39720010/724872>`_ 。
使用方法
======================
本书默认使用CPU训练,若是要使用GPU训练,使用步骤会稍有变化,请参考下文“使用GPU训练”
使用CPU训练
>>>>>>>>>>>>
只需要在命令行窗口里运行:
.. code-block:: shell
docker run -d -p 8888:8888 paddlepaddle/book
即可从DockerHub.com下载和运行本书的Docker image。阅读和在线编辑本书请在浏览器里访问 http://localhost:8888
如果您访问DockerHub.com很慢,可以试试我们的另一个镜像docker.paddlepaddlehub.com:
::
docker run -d -p 8888:8888 docker.paddlepaddlehub.com/book
使用GPU训练
>>>>>>>>>>>>>
为了保证GPU驱动能够在镜像里面正常运行,我们推荐使用 `nvidia-docker <https://github.com/NVIDIA/nvidia-docker>`_ 来运行镜像。请先安装nvidia-docker,之后请运行:
::
nvidia-docker run -d -p 8888:8888 paddlepaddle/book:latest-gpu
或者使用国内的镜像请运行:
::
nvidia-docker run -d -p 8888:8888 docker.paddlepaddlehub.com/book:latest-gpu
还需要将以下代码
.. code-block:: python
use_cuda = False
改成:
.. code-block:: python
use_cuda = True
贡献新章节
=============
您要是能贡献新的章节那就太好了!请发Pull Requests把您写的章节加入到 :code:`pending` 下面的一个子目录里。当这一章稳定下来,我们一起把您的目录挪到根目录。
为了写作、运行、调试,您需要安装Python 2.x和Go >1.5, 并可以用 `脚本程序 <https://github.com/PaddlePaddle/book/blob/develop/.tools/convert-markdown-into-ipynb-and-test.sh>`_ 来生成新的Docker image。
**Please Note:** We also provide `English Readme <https://github.com/PaddlePaddle/book/blob/develop/README.md>`_ for PaddlePaddle book
......@@ -2,15 +2,6 @@
Natural Language Processing
############################
This section collects 3 documents arranging from the simplest to the most challenging, which will guide you through the basic deep learning tasks in PaddlePaddle.
The documentation in this chapter covers a lot of deep learning basics and how to implement them with PaddlePaddle. See the instructions below for how to use:
Overview
======================
The book you are reading is an "interactive" e-book - each chapter can be run in a Jupyter Notebook.
.. toctree::
:titlesonly:
......@@ -19,78 +10,3 @@ The book you are reading is an "interactive" e-book - each chapter can be run in
label_semantic_roles/README.md
machine_translation/README.md
We packaged Jupyter, PaddlePaddle, and various dependency softwares into a Docker image. It frees you from installing these softwares by yourself, and you only need to just install Docker. For various Linux versions, please refer to https://www.docker.com . If you use docker on `Windows <https://www.docker.com/docker-windows>`_ or `Mac <https://www.docker.com/docker-mac>`_ , consider `allocate more Memory and CPU resources to Docker <http://stackoverflow.com/a/39720010/724872>`_ .
Instructions
======================
This book assumes you are performing CPU training by default. If you want to use GPU training, the steps will vary slightly. Please refer to "GPU Training" below.
CPU training
>>>>>>>>>>>>
Just run these in shell:
.. code-block:: shell
docker run -d -p 8888:8888 paddlepaddle/book
It downloads the Docker image for running books from DockerHub.com.
To read and edit this book on-line, please visit http://localhost:8888 in your browser.
If the Internet connection to DockerHub.com is compromised, try our spare docker image named docker.paddlepaddlehub.com:
::
docker run -d -p 8888:8888 docker.paddlepaddlehub.com/book
GPU training
>>>>>>>>>>>>>
To ensure that the GPU driver works properly in the image, we recommend running the image with `nvidia docker <https://github.com/NVIDIA/nvidia-docker>`_ . Please install nvidia-docker first, then run:
::
nvidia-docker run -d -p 8888:8888 paddlepaddle/book:latest-gpu
Or use a image source in China to run:
::
nvidia-docker run -d -p 8888:8888 docker.paddlepaddlehub.com/book:latest-gpu
modify the following codes
.. code-block:: python
use_cuda = False
into :
.. code-block:: python
use_cuda = True
Contribute to Book
===================
We highly appreciate your original contributions of new chapters to Book! Just Pull Requests of your contributions to the sub-directory in :code:`pending` . When this chapter is endorsed, we'll gladly move it to the root directory.
For writing, running, debugging, you need to install `shell <https://github.com/PaddlePaddle/book/blob/develop/.tools/convert-markdown-into-ipynb-and-test.sh>`_ to generate Docker image。
**Please Note:** We also provide `English Readme <https://github.com/PaddlePaddle/book/blob/develop/README.md>`_ for PaddlePaddle book
......@@ -2,81 +2,8 @@
推荐
################
本章由1篇文档组成,将指导您如何使用PaddlePaddle完成推荐领域的基础深度学习任务
本章文档涉及大量了深度学习基础知识,也介绍了如何使用PaddlePaddle实现这些内容,请参阅以下说明了解如何使用:
内容简介
======================
您现在在看的这本书是一本“交互式”电子书 —— 每一章都可以运行在一个Jupyter Notebook里。
.. toctree::
:titlesonly:
recommender_system/README.cn.md
我们把Jupyter、PaddlePaddle、以及各种被依赖的软件都打包进一个Docker image了。所以您不需要自己来安装各种软件,只需要安装Docker即可。对于各种Linux发行版,请参考 https://www.docker.com 。如果您使用 `Windows <https://www.docker.com/docker-windows>`_ 或者 `Mac <https://www.docker.com/docker-mac>`_,可以考虑 `给Docker更多内存和CPU资源 <http://stackoverflow.com/a/39720010/724872>`_ 。
使用方法
======================
本书默认使用CPU训练,若是要使用GPU训练,使用步骤会稍有变化,请参考下文“使用GPU训练”
使用CPU训练
>>>>>>>>>>>>
只需要在命令行窗口里运行:
.. code-block:: shell
docker run -d -p 8888:8888 paddlepaddle/book
即可从DockerHub.com下载和运行本书的Docker image。阅读和在线编辑本书请在浏览器里访问 http://localhost:8888
如果您访问DockerHub.com很慢,可以试试我们的另一个镜像docker.paddlepaddlehub.com:
::
docker run -d -p 8888:8888 docker.paddlepaddlehub.com/book
使用GPU训练
>>>>>>>>>>>>>
为了保证GPU驱动能够在镜像里面正常运行,我们推荐使用 `nvidia-docker <https://github.com/NVIDIA/nvidia-docker>`_ 来运行镜像。请先安装nvidia-docker,之后请运行:
::
nvidia-docker run -d -p 8888:8888 paddlepaddle/book:latest-gpu
或者使用国内的镜像请运行:
::
nvidia-docker run -d -p 8888:8888 docker.paddlepaddlehub.com/book:latest-gpu
还需要将以下代码
.. code-block:: python
use_cuda = False
改成:
.. code-block:: python
use_cuda = True
贡献新章节
=============
您要是能贡献新的章节那就太好了!请发Pull Requests把您写的章节加入到 :code:`pending` 下面的一个子目录里。当这一章稳定下来,我们一起把您的目录挪到根目录。
为了写作、运行、调试,您需要安装Python 2.x和Go >1.5, 并可以用 `脚本程序 <https://github.com/PaddlePaddle/book/blob/develop/.tools/convert-markdown-into-ipynb-and-test.sh>`_ 来生成新的Docker image。
**Please Note:** We also provide `English Readme <https://github.com/PaddlePaddle/book/blob/develop/README.md>`_ for PaddlePaddle book
......@@ -2,92 +2,9 @@
Recommend
############################
This section collects 1 document arranging from the simplest to the most challenging, which will guide you through the basic deep learning tasks in PaddlePaddle.
The documentation in this chapter covers a lot of deep learning basics and how to implement them with PaddlePaddle. See the instructions below for how to use:
Overview
======================
The book you are reading is an "interactive" e-book - each chapter can be run in a Jupyter Notebook.
.. toctree::
:titlesonly:
recommender_system/README.md
We packaged Jupyter, PaddlePaddle, and various dependency softwares into a Docker image. It frees you from installing these softwares by yourself, and you only need to just install Docker. For various Linux versions, please refer to https://www.docker.com . If you use docker on `Windows <https://www.docker.com/docker-windows>`_ or `Mac <https://www.docker.com/docker-mac>`_ , consider `allocate more Memory and CPU resources to Docker <http://stackoverflow.com/a/39720010/724872>`_ .
Instructions
======================
This book assumes you are performing CPU training by default. If you want to use GPU training, the steps will vary slightly. Please refer to "GPU Training" below.
CPU training
>>>>>>>>>>>>
Just run these in shell:
.. code-block:: shell
docker run -d -p 8888:8888 paddlepaddle/book
It downloads the Docker image for running books from DockerHub.com.
To read and edit this book on-line, please visit http://localhost:8888 in your browser.
If the Internet connection to DockerHub.com is compromised, try our spare docker image named docker.paddlepaddlehub.com:
::
docker run -d -p 8888:8888 docker.paddlepaddlehub.com/book
GPU training
>>>>>>>>>>>>>
To ensure that the GPU driver works properly in the image, we recommend running the image with `nvidia docker <https://github.com/NVIDIA/nvidia-docker>`_ . Please install nvidia-docker first, then run:
::
nvidia-docker run -d -p 8888:8888 paddlepaddle/book:latest-gpu
Or use a image source in China to run:
::
nvidia-docker run -d -p 8888:8888 docker.paddlepaddlehub.com/book:latest-gpu
modify the following codes
.. code-block:: python
use_cuda = False
into :
.. code-block:: python
use_cuda = True
Contribute to Book
===================
We highly appreciate your original contributions of new chapters to Book! Just Pull Requests of your contributions to the sub-directory in :code:`pending` . When this chapter is endorsed, we'll gladly move it to the root directory.
For writing, running, debugging, you need to install `shell <https://github.com/PaddlePaddle/book/blob/develop/.tools/convert-markdown-into-ipynb-and-test.sh>`_ to generate Docker image。
**Please Note:** We also provide `English Readme <https://github.com/PaddlePaddle/book/blob/develop/README.md>`_ for PaddlePaddle book
......@@ -2,16 +2,6 @@
简单案例
################
本章由4篇文档组成,将指导您如何使用PaddlePaddle完成基础的深度学习任务
本章文档涉及大量了深度学习基础知识,也介绍了如何使用PaddlePaddle实现这些内容,请参阅以下说明了解如何使用:
内容简介
======================
您现在在看的这本书是一本“交互式”电子书 —— 每一章都可以运行在一个Jupyter Notebook里。
.. toctree::
:titlesonly:
......@@ -19,67 +9,3 @@
recognize_digits/README.cn.md
image_classification/README.cn.md
word2vec/README.cn.md
我们把Jupyter、PaddlePaddle、以及各种被依赖的软件都打包进一个Docker image了。所以您不需要自己来安装各种软件,只需要安装Docker即可。对于各种Linux发行版,请参考 https://www.docker.com 。如果您使用 `Windows <https://www.docker.com/docker-windows>`_ 或者 `Mac <https://www.docker.com/docker-mac>`_,可以考虑 `给Docker更多内存和CPU资源 <http://stackoverflow.com/a/39720010/724872>`_ 。
使用方法
======================
本书默认使用CPU训练,若是要使用GPU训练,使用步骤会稍有变化,请参考下文“使用GPU训练”
使用CPU训练
>>>>>>>>>>>>
只需要在命令行窗口里运行:
.. code-block:: shell
docker run -d -p 8888:8888 paddlepaddle/book
即可从DockerHub.com下载和运行本书的Docker image。阅读和在线编辑本书请在浏览器里访问 http://localhost:8888
如果您访问DockerHub.com很慢,可以试试我们的另一个镜像docker.paddlepaddlehub.com:
::
docker run -d -p 8888:8888 docker.paddlepaddlehub.com/book
使用GPU训练
>>>>>>>>>>>>>
为了保证GPU驱动能够在镜像里面正常运行,我们推荐使用 `nvidia-docker <https://github.com/NVIDIA/nvidia-docker>`_ 来运行镜像。请先安装nvidia-docker,之后请运行:
::
nvidia-docker run -d -p 8888:8888 paddlepaddle/book:latest-gpu
或者使用国内的镜像请运行:
::
nvidia-docker run -d -p 8888:8888 docker.paddlepaddlehub.com/book:latest-gpu
还需要将以下代码
.. code-block:: python
use_cuda = False
改成:
.. code-block:: python
use_cuda = True
贡献新章节
=============
您要是能贡献新的章节那就太好了!请发Pull Requests把您写的章节加入到 :code:`pending` 下面的一个子目录里。当这一章稳定下来,我们一起把您的目录挪到根目录。
为了写作、运行、调试,您需要安装Python 2.x和Go >1.5, 并可以用 `脚本程序 <https://github.com/PaddlePaddle/book/blob/develop/.tools/convert-markdown-into-ipynb-and-test.sh>`_ 来生成新的Docker image。
**Please Note:** We also provide `English Readme <https://github.com/PaddlePaddle/book/blob/develop/README.md>`_ for PaddlePaddle book
......@@ -2,15 +2,6 @@
Simple Case
############################
This section collects 4 documents arranging from the simplest to the most challenging, which will guide you through the basic deep learning tasks in PaddlePaddle.
The documentation in this chapter covers a lot of deep learning basics and how to implement them with PaddlePaddle. See the instructions below for how to use:
Overview
======================
The book you are reading is an "interactive" e-book - each chapter can be run in a Jupyter Notebook.
.. toctree::
:titlesonly:
......@@ -20,77 +11,3 @@ The book you are reading is an "interactive" e-book - each chapter can be run in
image_classification/README.md
word2vec/README.md
We packaged Jupyter, PaddlePaddle, and various dependency softwares into a Docker image. It frees you from installing these softwares by yourself, and you only need to just install Docker. For various Linux versions, please refer to https://www.docker.com . If you use docker on `Windows <https://www.docker.com/docker-windows>`_ or `Mac <https://www.docker.com/docker-mac>`_ , consider `allocate more Memory and CPU resources to Docker <http://stackoverflow.com/a/39720010/724872>`_ .
Instructions
======================
This book assumes you are performing CPU training by default. If you want to use GPU training, the steps will vary slightly. Please refer to "GPU Training" below.
CPU training
>>>>>>>>>>>>
Just run these in shell:
.. code-block:: shell
docker run -d -p 8888:8888 paddlepaddle/book
It downloads the Docker image for running books from DockerHub.com.
To read and edit this book on-line, please visit http://localhost:8888 in your browser.
If the Internet connection to DockerHub.com is compromised, try our spare docker image named docker.paddlepaddlehub.com:
::
docker run -d -p 8888:8888 docker.paddlepaddlehub.com/book
GPU training
>>>>>>>>>>>>>
To ensure that the GPU driver works properly in the image, we recommend running the image with `nvidia docker <https://github.com/NVIDIA/nvidia-docker>`_ . Please install nvidia-docker first, then run:
::
nvidia-docker run -d -p 8888:8888 paddlepaddle/book:latest-gpu
Or use a image source in China to run:
::
nvidia-docker run -d -p 8888:8888 docker.paddlepaddlehub.com/book:latest-gpu
modify the following codes
.. code-block:: python
use_cuda = False
into :
.. code-block:: python
use_cuda = True
Contribute to Book
===================
We highly appreciate your original contributions of new chapters to Book! Just Pull Requests of your contributions to the sub-directory in :code:`pending` . When this chapter is endorsed, we'll gladly move it to the root directory.
For writing, running, debugging, you need to install `shell <https://github.com/PaddlePaddle/book/blob/develop/.tools/convert-markdown-into-ipynb-and-test.sh>`_ to generate Docker image。
**Please Note:** We also provide `English Readme <https://github.com/PaddlePaddle/book/blob/develop/README.md>`_ for PaddlePaddle book
.. ctr:
.. role:: raw-html-m2r(raw)
:format: html
ELASTIC CTR
......@@ -14,7 +14,8 @@ ELASTIC CTR
* `4. 查看结果 <#head4>`_
* `5. 二次开发指南 <#head5>`_
`<span id='head_1'>1. 总体概览</span>`
:raw-html-m2r:`<span id='head_1'>1. 总体概览</span>`
-------------
本项目提供了端到端的CTR训练和二次开发的解决方案,主要特点:
......@@ -65,12 +66,13 @@ ELASTIC CTR
**第5节 二次开发** 提出本一键部署方案可定制改善的部分,给出具体修改位置等
`<span id='head2'>2. 前置需求</span>`
:raw-html-m2r:`<span id='head2'>2. 前置需求</span>`
------------
运行本方案前,需要用户已经搭建好k8s集群,并安装好volcano组件。k8s环境部署比较复杂,本文不涉及。百度智能云CCE容器引擎申请后即可使用,仅以百度云上创建k8s为例。
2.1 创建k8s集群
---------------
^^^^^^^^^^^^
请参考
`百度智能云CCE容器引擎帮助文档-创建集群 <https://cloud.baidu.com/doc/CCE/GettingStarted/24.5C.E5.88.9B.E5.BB.BA.E9.9B.86.E7.BE.A4.html#.E6.93.8D.E4.BD.9C.E6.AD.A5.E9.AA.A4>`_\ ,在百度智能云上建立一个集群,节点配置需要满足如下要求
......@@ -89,7 +91,7 @@ ELASTIC CTR
创建完成后,即可参考\ `百度智能云CCE容器引擎帮助文档-查看集群 <https://cloud.baidu.com/doc/CCE/GettingStarted.html#.E6.9F.A5.E7.9C.8B.E9.9B.86.E7.BE.A4>`_\ ,查看刚刚申请的集群信息。
2.2 如何操作集群
----------------
^^^^^^^^^^^^^
集群的操作可以通过百度云web或者通过kubectl工具进行,推荐用kubectl工具。
......@@ -119,7 +121,7 @@ ELASTIC CTR
* 关于kubectl的其他信息,可以参考\ `Overview of kubectl <https://kubernetes.io/docs/reference/kubectl/overview/>`_\ 。
2.3 设置访问权限
----------------
^^^^^^^^^^
建立分布式任务需要pod间有API互相访问的权限,可以按如下步骤
......@@ -130,7 +132,7 @@ ELASTIC CTR
注意: --namespace 指定的default 为创建集群时候的名称
2.4 安装Volcano
---------------
^^^^^^^^^^
我们使用volcano作为训练阶段的批量任务管理工具。关于volcano的详细信息,请参考\ `官方网站 <https://volcano.sh/>`_\ 的Documentation。
......@@ -146,15 +148,16 @@ ELASTIC CTR
:alt: image
3.`<span id='head3'>分布式训练+Serving方案一键部署</span>`
3. :raw-html-m2r:`<span id='head3'>分布式训练+Serving方案一键部署</span>`
---------------------------------
3.1 下载部署方案脚本文件
------------------------
^^^^^^^^^^^^
请将\ `本方案所需所有脚本文件 <https://github.com/PaddlePaddle/edl/tree/develop/example/ctr/script>`_\ 下载到本地
3.2 一键部署
------------
^^^^^^^^^^^
执行以下脚本,一键将所有组件部署到k8s集群。
......@@ -169,7 +172,7 @@ ELASTIC CTR
**注**\ :以下\ **3.3-3.8节所述内容已经在一键部署脚本中包含,无需手动执行**\ 。但为方便理解,将该脚本的每一步执行过程给出说明。
3.3 选择一个node作为输出节点
----------------------------
^^^^^^^^^^^^^^^^
.. code-block:: bash
......@@ -178,7 +181,7 @@ ELASTIC CTR
这句话的意思是给这个node做一个标记,之后的文件服务和模型产出都被强制分配在这个node上进行,把NAME的一串字符替换 \$NODE_NAME即可。
3.4 启动文件服务器
------------------
^^^^^^^^^^^^^^
.. code-block:: bash
......@@ -209,7 +212,7 @@ ELASTIC CTR
3.5 启动Cube稀疏参数服务器
--------------------------
^^^^^^^^^^^^^^^^
.. code-block:: bash
......@@ -230,7 +233,7 @@ ELASTIC CTR
**注**\ :分片数量可根据稀疏字典大小灵活修改,参考5.3节。
3.6 启动Paddle Serving
----------------------
^^^^^^^^^^^^^^^
.. code-block:: bash
......@@ -259,7 +262,7 @@ ELASTIC CTR
3.7 启动Cube稀疏参数服务器配送工具
----------------------------------
^^^^^^^^^^^^^^^^^^^^^^^^
.. code-block:: bash
......@@ -288,7 +291,7 @@ ELASTIC CTR
3.8 执行Paddle CTR分布式训练
----------------------------
^^^^^^^^^^^^^^^^^^^^^^^
.. code-block:: bash
......@@ -308,10 +311,11 @@ ELASTIC CTR
:alt: image
4. `<span id='head4'>`\ 查看结果\ :raw-html-m2r:`<span>`
4. :raw-html-m2r:`<span id='head4'>`\ 查看结果\ :raw-html-m2r:`<span>`
-------------------------------------------
4.1 查看训练日志
----------------
^^^^^^^^^^^^^
百度云容器引擎CCE提供了web操作台方便查看pod的运行状态。
......@@ -334,7 +338,7 @@ pserver日志示例:
4.2 验证Paddle Serving预测结果
------------------------------
^^^^^^^^^^^^^^^^^^^
执行
......@@ -360,10 +364,11 @@ pserver日志示例:
:alt: image
5. `<span id='head5'>二次开发指南</span>`
5. :raw-html-m2r:`<span id='head5'>二次开发指南</span>`
-----------------------------
5.1 指定数据集的输入和读取方式
------------------------------
^^^^^^^^^^^^^^^^^^^
现有的数据的输入是从edldemo镜像当中的/workspace/ctr/data/download.sh目录进行下载。下载之后会解压在/workspace/ctr/data/raw文件夹当中,包含train.txt和test.txt。所有的数据的每一行通过空格隔开40个属性。
......@@ -398,7 +403,7 @@ pserver日志示例:
推荐使用百度云提供的镜像仓库,这里是说明文档\ `推送镜像到镜像仓库 <https://cloud.baidu.com/doc/CCE/s/Yjxppt74z/#%E6%8E%A8%E9%80%81%E9%95%9C%E5%83%8F%E5%88%B0%E9%95%9C%E5%83%8F%E4%BB%93%E5%BA%93>`_\
5.2 指定训练规模
----------------
^^^^^^^^^^^^^^
在ctr.yaml文件当中,我们会发现这个是在volcano的框架下定义的Job。在Job里面,我们给出了很多Pserver和Trainer的定义,在总体的Job也给出了MinAvailable数量的定义。Pserver和Trainer下面有自己的Replicas,环境变量当中有PSERVER_NUM和TRAINER_MODEL和TRAINER_NUM的数量。通常MinAvailable= PServer Num + Trainer Num,这样我们就可以启动相应的服务。
......@@ -427,7 +432,7 @@ pserver日志示例:
如上图所示
5.3 指定Cube参数服务器的分片数量和副本数量
------------------------------------------
^^^^^^^^^^^^^^^^^^^^
在cube.yaml文件当中,我们可以看到每一个Cube的节点的定义,有一个\ ``cube server pod``\ 和\ ``cube server service``\ 。如果我们需要增加cube的副本数和分片数,只需要在yaml文件中复制相关的定义和环境变量即可。
......@@ -446,7 +451,7 @@ pserver日志示例:
以上两个图片,一个是对Cube POD的定义,一个是对CubeSERVICE的定义。如果需要扩展Cube分片数量,可以复制POD和SERVICE的定义,并重命名它们。示例程序给出的是2个分片,复制之后第3个可以命名为cube-2。
5.4 Serving适配新的模型
-----------------------
^^^^^^^^^^^^^^^^^^^
在本示例中,我们如果按照5.1节的方式,修改了CTR模型训练脚本的feed数据格式,就需要相应修改Serving的代码,以适应新的feed样例字段数量和数据类型。
......@@ -462,7 +467,7 @@ pserver日志示例:
注释
----------
注1. :raw-html-m2r:`<span id='annotation_1'>Cube和Redis性能对比测试环境</span>`
-----------------------------------------------------------------------------------
......@@ -580,4 +585,4 @@ client端为基于\ `redisplusplus <https://github.com/sewenew/redis-plus-plus>`
在扩展性方面,Redis受制于单线程模型,随并发数增加,响应时间加倍增加,而总吞吐在1000qps左右即不再上涨;而Cube则随着压测并发数增加,总的qps一直上涨,说明Cube能够较好处理并发请求,具有良好的扩展能力。
RocksDB在线程数较少的时候,平均响应时间和qps慢于Redis,但是在16以及更多线程的测试当中,RocksDB提供了更快的响应时间和更大的qps。
RocksDB在线程数较少的时候,平均响应时间和qps慢于Redis,但是在16以及更多线程的测试当中,RocksDB提供了更快的响应时间和更大的qps。
\ No newline at end of file
......@@ -2,12 +2,6 @@
工具组件
################
本章由1篇文档组成,将指导您如何使用PaddlePaddle工具组件完成深度学习任务
本章文档涉及大量了深度学习基础知识,也介绍了如何使用PaddlePaddle实现这些内容,请参阅以下说明了解如何使用:
.. toctree::
:titlesonly:
......
############################
Basic Deep Learning Models
############################
This section collects 8 documents arranging from the simplest to the most challenging, which will guide you through the basic deep learning tasks in PaddlePaddle.
The documentation in this chapter covers a lot of deep learning basics and how to implement them with PaddlePaddle. See the instructions below for how to use:
Overview
======================
The book you are reading is an "interactive" e-book - each chapter can be run in a Jupyter Notebook.
.. toctree::
:titlesonly:
fit_a_line/README.md
recognize_digits/README.md
image_classification/index_en.md
word2vec/index_en.md
recommender_system/index_en.md
understand_sentiment/index_en.md
label_semantic_roles/index_en.md
machine_translation/index_en.md
We packaged Jupyter, PaddlePaddle, and various dependency softwares into a Docker image. It frees you from installing these softwares by yourself, and you only need to just install Docker. For various Linux versions, please refer to https://www.docker.com . If you use docker on `Windows <https://www.docker.com/docker-windows>`_ or `Mac <https://www.docker.com/docker-mac>`_ , consider `allocate more Memory and CPU resources to Docker <http://stackoverflow.com/a/39720010/724872>`_ .
Instructions
======================
This book assumes you are performing CPU training by default. If you want to use GPU training, the steps will vary slightly. Please refer to "GPU Training" below.
CPU training
>>>>>>>>>>>>
Just run these in shell:
.. code-block:: shell
docker run -d -p 8888:8888 paddlepaddle/book
It downloads the Docker image for running books from DockerHub.com.
To read and edit this book on-line, please visit http://localhost:8888 in your browser.
If the Internet connection to DockerHub.com is compromised, try our spare docker image named docker.paddlepaddlehub.com:
::
docker run -d -p 8888:8888 docker.paddlepaddlehub.com/book
GPU training
>>>>>>>>>>>>>
To ensure that the GPU driver works properly in the image, we recommend running the image with `nvidia docker <https://github.com/NVIDIA/nvidia-docker>`_ . Please install nvidia-docker first, then run:
::
nvidia-docker run -d -p 8888:8888 paddlepaddle/book:latest-gpu
Or use a image source in China to run:
::
nvidia-docker run -d -p 8888:8888 docker.paddlepaddlehub.com/book:latest-gpu
modify the following codes
.. code-block:: python
use_cuda = False
into :
.. code-block:: python
use_cuda = True
Contribute to Book
===================
We highly appreciate your original contributions of new chapters to Book! Just Pull Requests of your contributions to the sub-directory in :code:`pending` . When this chapter is endorsed, we'll gladly move it to the root directory.
For writing, running, debugging, you need to install `shell <https://github.com/PaddlePaddle/book/blob/develop/.tools/convert-markdown-into-ipynb-and-test.sh>`_ to generate Docker image。
**Please Note:** We also provide `English Readme <https://github.com/PaddlePaddle/book/blob/develop/README.md>`_ for PaddlePaddle book
Markdown is supported
0% .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑!
想要评论请 注册