From a038163eefd4c9868f15eaeb5be4839c8e990e9d Mon Sep 17 00:00:00 2001 From: Yu Yang Date: Thu, 25 May 2017 13:38:51 +0800 Subject: [PATCH] Follow comments --- doc/design/parameters_in_cpp.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/doc/design/parameters_in_cpp.md b/doc/design/parameters_in_cpp.md index 4c0dcc1823e..b6f99bc7d9d 100644 --- a/doc/design/parameters_in_cpp.md +++ b/doc/design/parameters_in_cpp.md @@ -1,4 +1,4 @@ -# Parameters in CPP +# Design Doc: The C++ Class `Parameters` `Parameters` is a concept we designed in Paddle V2 API. `Parameters` is a container of parameters, and make Paddle can shared parameter between topologies. We described usages of `Parameter` in [api.md](./api.md). @@ -33,6 +33,8 @@ The step by step approach for implementation Parameters in Paddle C++ core is li 3. Make `Parameters` support Multi-CPU and Multi-GPU training to prepare for sharing `Parameter` between topologies. Because we need share `Parameters` between topologies, it is `Parameters`'s response to exchange Parameters between GPUs. `GradientMachine` should not handle how to exchange Parameters because `GradientMachine` only used to train one topology and we need to support train many topologies in Paddle, i.e., there could be many GradientMachines use one `Parameters`. + * We should use a global function to exchange Parameters between GPUs, not a member function in `Parameters`. The `MultiGradientMachine` invoke this function, which uses `Parameters` as this function inputs. + * The MultiGradientMachine contains many functionalities. Extracting the Parameters exchanging logic could make MultiGradientMachine clearer and simpler. 4. Make `Parameters` as an argument for `forward/backward` function, not a data member for `GradientMachine`. For example, `forward` could be `forward(const Parameters& params, ...)` and `backward` could be `backward(Parameters* params, ...)`. After this step, Paddle could share `Parameters` between topologies. -- GitLab